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Web  Usability  Test  Findings  and  Analysis  Issues 


Jae  Soon  An 
Su-Hong  Park 
Eun-Ok  Baek 

Indiana  University  Bloomington 

Abstract 

In  this  paper,  we  discuss  findings  and  data  analysis  issues  that  resulted  from  a case  usability  test.  The  findings  relate  to  the 
generation  of  design  ideas.  Through  usability  test,  it  was  found  that  users  have  a somewhat  unified  viewpoint  regarding  the 
menus  on  the  side  areas  of  a web  page.  They  tend  to  view  them  as  a "quick  ” way  to  access  "specific  ” and  "frequently  searched  " 
information.  The  analysis  issues  are  concerned  with  analyzing  task  completion  rate,  time  taken,  and  path  taken  to  finish  the 
tasks.  When  analyzing  these  measures,  the  researcher  needs  to  consider  nature  of  the  web  site,  portal  site  in  this  case  usability 
test,  rather  than  simply  looking  at  whether  and  how  the  test  participants  found  the  answer. 

Introduction 

This  paper  discusses  findings  and  analysis  issues  derived  from  a lab  usability  test.  The  procedure  of  conducting  a lab 
usability  test  is  pretty  much  standardized  and  there  are  standard  sets  of  data  typically  collected  in  web  site  tests  such  as,  task 
completion  rate,  time  taken,  paths  taken,  and  verbal  protocol.  The  analysis  of  the  data,  however,  varies  widely  depending  on  the 
case.  When  analyzing  usability  test  data,  it  is  important  to  consider  the  nature  of  the  web  site.  Depending  on  the  nature  of  the 
site,  the  same  data  can  be  analyzed  differently  resulting  in  different  usability  indices.  In  addition,  it  is  well-known  iii  the  usability 
literature  that  deriving  design  ideas  from  usability  test  data  is  a difficult  task.  Generating  design  ideas  is  closely  related  to  how 
the  researcher  analyzes  test  data.  This  paper  describes  and  discusses  how  a web  design  team  considered  the  nature  of  the  portal 
site  in  data  analysis  and  derived  design  ideas  by  distilling  data  into  a finding  and  the  finding  into  design  ideas. 

Redesign  of  a University  Web  Site 

The  case  usability  test  this  paper  derives  data  from  was  conducted  as  a part  of  redesign  project  that  aimed  to  update  the 
existing  home  page  of  Indiana  University  Bloomington  (lUB)  [2].  The  home  page  contained  links  to  the  web  sites  that  various 
institutions  in  the  university  had  already  created.  Therefore,  it  functioned  as  a portal  to  various  web  sites  and  employed 
hierarchical  menu  structure. 

It  was  acknowledged  that  the  site  had  a high  degree  of  usability  because  a group  of  graduate  students  in  the  university 
conducted  a user-centered  design  research  and  applied  the  research  results  when  designing  the  site  in  1995  [1].  Although  usable, 
there  occurred  several  reasons  to  redesign  the  site.  First  of  all,  it  was  reported  that  prospective  students  look  at  universities’  home 
pages  a lot  when  they  consider  applying  for  schools.  Keeping  the  site  up-to-date  was  important  in  the  view  of  the  university's 
marketing  strategy.  In  addition,  information  organization  of  the  site  was  getting  messy  as  the  site  was  accommodating  publication 
requests  from  various  institutions  in  the  university  over  the  years.  For  example,  some  pages  had  too  many  links  and  some  links 
were  not  placed  in  the  page  where  users  would  expect  to  see  them.  Finally,  there  has  been  a consistent  request  that  the  site 
should  enhance  its  visual  aspects.  With  these  reasons,  lUB  Web  team  began  a redesign  project  in  January,  2000  with  an 
emphasis  on  user-centered  design  principle. 

As  the  first  step  of  the  redesign  effort,  the  design  team  created  a preliminary  prototype  outlining  link  categorization  only  and 
tested  usability  of  the  link  categorization.  Then,  the  team  produced  second  prototype  where  they  incorporated  results  of  the 
usability  test  and  included  visual  elements.  Although  the  link  categorization  was  assumed  to  be  usable,  the  team  conducted 
another  bigger  scale  usability  evaluation  on  the  second  prototype  to  ensure  usability  of  the  overall  prototype.  Usability 
evaluation  on  the  second  prototype  had  various  goals.  In  addition  to  collecting  general  usability  measures  such  as  task 
completion  rate,  time  taken  to  complete  the  tasks,  and  the  links  chosen  to  complete  the  tasks,  the  evaluation  aimed  to  measure 
effectiveness  of  new  navigation  features  and  hear  users’  opinion  about  the  site  in  general  and  visual  design  in  specific.  It  also 
targeted  to  assess  accessibility  and  speed  of  the  site.  To  achieve  such  various  goals,  it  was  necessary  to  use  various  test  methods 
such  as  a lab  usability  test,  heuristics  evaluations,  a web-based  survey,  foreign  font  display  check,  and  page  loading  speed  check. 

Lab  Usability  Test 

The  findings  and  analysis  issues  discussed  in  this  paper  resulted  mainly  from  the  lab  usability  test.  The  lab  test  employed 
typical  usability  test  methodology.  Participants  were  selected  through  purposeful  sampling.  Nine  users  of  the  current  lUB  home 
page  were  recruited  as  test  participants  considering  their  occupation,  web  use,  gender,  and  nationality.  In  the  test,  the  participants 
were  asked  to  perform  eleven  tasks  using  either  the  new  prototype  or  the  existing  web  site.  Through  random  assignment, 
participants  1,  3,  4,  5,  and  9 were  asked  to  use  the  new  prototype,  whereas  2,  6,  7,  and  8,  the  existing  site.  Through  the  eleven 
tasks,  participants  were  asked  to  search  for  typically  and  frequently  searched  information  in  lUB  site.  The  tasks  also  required  the 
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participants  to  use  new  navigational  features  and  menu  structures.  Throughout  the  test,  participants  were  asked  to  think  out  loud. 
The  test  administrator  observed  the  test  session  collecting  performance  measures  such  as  task  completion,  time  taken,  and  paths 
taken  as  well  as  the  participants’  comments  and  emotional  expressions.  She  sometimes  prompted  and  probed  think-aloud,  and 
responded  to  the  user’s  task-related  questions.  The  whole  test  sessions  were  videotaped  for  later  review.  After  the  usability  test, 
a debriefing  interview  was  conducted.  The  test  administrator  solicited  further  comments  about  the  events  during  the  test  that  she 
did  not  understand.  In  addition,  she  asked  for  comments  about  the  web  site  in  general  and  any  suggestions  for  improvement. 


Data  Analysis  Considering  Nature  of  the  Portal  Site 

Task  completion  rate,  time  taken,  and  paths  chosen  to  complete  the  tasks  are  three  classical  usability  measures.  Typically, 
successful  completion  is  considered  to  be  the  point  when  the  participant  finds  the  answer.  The  time  taken  and  the  links  chosen 
till  the  completion  point  are  typically  analyzed.  Such  typical  way  of  analysis,  however,  didn’t  seem  to  work  well  in  the  lUB  web 
site  test.  As  the  target  site  of  the  test  was  a portal  site  providing  links  to  various  external  web  sites,  it  was  inevitable  for  the 
participants  to  use  sites  other  than  the  target  site  (i.e.,  external  sites)  in  order  to  finish  the  tasks.  Indeed,  some  users  spent  much 
time  looking  at  external  pages.  Some  failed  to  find  the  answers  due  to  the  design  problems  of  the  external  sites.  Simply  looking 
at  whether  the  participant  had  found  the  answer  or  not,  and  counting  total  time  taken  didn’t  seem  to  be  appropriate  measurements. 
After  several  rounds  of  discussion,  the  team  decided  to  make  a distinction  between  the  use  of  lUB  home  pages  and  the  use  of 
other  sites  and  collect  the  three  measures  during  the  use  of  lUB  home  pages  only. 

Task  Completion 

In  the  case  of  task  completion  rate,  the  team  decided  to  consider  that  a participant  completed  a task  successfully  if  she  found 
the  link  leading  to  the  external  site  containing  the  answer  (i.e.,  correct  link),  regardless  she  found  the  answer  or  not.  The  fact  that 
the  user  had  found  the  correct  link  was  assumed  to  indicate  that  the  link  categorization  in  lUB  site  was  effective  enough  for  the 
user  to  get  to  the  right  place,  which  was  the  main  goal  of  the  site.  If  the  user  failed  to  find  the  answer  after  getting  to  the  correct 
page,  it  was  due  to  the  poor  design  of  the  external  page,  which  was  beyond  the  goal  of  the  re-design  project.  Table  2.1 
summarizes  task  completion  rate  following  this  analysis  decision.  If  the  user  found  the  correct  page  but  failed  to  find  the  answer, 
it  was  indicated  as  f/s  and  counted  as  a success  rather  than  a fail.  Table  2.2  summarizes  task  completion  rate  by  Just  considering 
whether  the  user  found  the  answer  or  not.  Note  the  difference  in  the  number  of  people  completed  the  task  successfully  between 
the  two  tables.  T test  result  on  the  difference  was  t = 1.884,  df=10,  p=  089.  Considering  small  sample  size,  the  difference  was 
almost  significant. 


<Table  2J>  Task  completion  rate  considering  the  use  oflUB  site  only 
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f/s:  the  user  found  the  correct  page  but  could  not  find  the  information  due  to  the  poor  design  of  external  sites 


<Table  2.2>  Task  completion  rate  considering  the  use  of  not  only  lUB  site  but  also  external  site 
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No.  of  people  completed  the  task  successfully  55254350555 


Time  Taken 

In  the  case  of  the  time  taken  to  finish  the  tasks,  the  team  decided  to  count  the  time  spent  before  the  users  found  the  correct 
external  site  for  the  first  time.  Here,  the  word  “for  the  first  time”  is  important  because  several  occasions  occurred  where  users 
found  the  correct  external  site  initially  but  could  not  find  the  answer  in  the  site.  Then,  they  went  back  to  lUB  site,  tried  other 
external  sites,  and  went  back  to  the  correct  site  which  they  looked  at  previously  and  found  the  answer  eventually.  In  this  case, 
the  time  taken  to  find  the  correct  external  site  for  the  first  time  seemed  to  be  an  accurate  measure  of  the  effectiveness  of  the  link 
categorization.  Table  3.1  summarizes  time  taken  to  find  the  correct  external  page  for  the  first  time  whereas  Table  3.2,  total  time 
taken  to  finish  the  tasks.  The  mean  time  taken  is  significantly  (t=  -4.318,  df=10,  p=  002)  different  depending  on  the  two  different 
ways  of  counting  time. 
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<Table  3.1>  Time  taken  to  find  the  correct  external  page  for  the  first  time  (in  second) 
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<Table  3.2>  Total  time  taken  to  find  the  answer  (in  second) 
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Paths 

When  analyzing  the  links  the  participants  have  chosen  till  they  find  the  answer  (i.e.,  path  taken),  it  is  difficult  to  derive 
general  currency  due  to  the  wide  variety  of  the  links  people  usually  choose.  It  is  often  possible  to  analyze  how  many  people 
followed  the  ideal  thread  of  links  and  how  many  did  not,  or  what  links  in  the  ideal  thread  people  did  not  choose  correctly.  Such 
analysis  provides  the  design  team  with  a general  evaluation  on  the  effectiveness  of  the  link  categorization.  However,  they  do  not 
render  much  design  ideas,  other  than  the  requirement  to  re-place  a few  links  that  people  did  not  choose  correctly  in  an  ad  hoc 
manner. 

The  team  sought  out  a more  systematic  way  to  utilize  the  path  data  and  found  out  that  a rich  set  of  design  ideas  could  emerge 
by  summarizing  the  first  link  the  participants  chose  to  complete  each  task.  Table  1 summarizes  the  first  link  participants  chose  to 
complete  the  tasks  in  the  lUB  site  test.  Almost  all  participantsichose  the  same  link  to  perform  tasks  2,  4,  5,  and  9.  To  perform 
tasks  1,  7,  8,  and  10,  participants  chose  more  than  one  link  however  there  was  one  link  that  majority  of  the  participants  (i.e.,  three 
out  of  five)  chose.  For  example,  to  perform  task  7,  three  out  of  five  users  chose  “ss”  as  the  first  link  whereas  two  users  chose 
different  links.  For  tasks  3,  6,  and  1 1,  people  chose  three  different  links  and  there  was  no  particular  link  most  participants  chose. 


<Table  J>  First  link  participants  chose  to  complete  each  task 
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Note:  Link  names  are  abbreviated.  E.g.)  '^pg''  is  '^people  & groups"  and  "ss"  is  "student  services." 

By  examining  the  degree  of  variability  in  the  choice  of  the  first  link,  the  team  could  generate  design  ideas.  The  data  indicated 
that  information  related  to  the  tasks  2,  4,  5,  and  9 were  categorized  intuitively  enough  for  most  people  to  select  correct  entry 
point.  On  the  contrary,  those  related  to  3,  6,  and  1 1 were  not.  Therefore,  information  related  to  the  tasks  3,6,  and  1 1 needed  to 
be  either  labeled  better  or  placed  in  a more  appropriate  link  hierarchy.  The  data  also  gave  ideas  as  to  where  to  put  certain 
information.  For  example,  the  data  clearly  indicated  that  users  expected  to  find  information  related  to  the  task  8 either  in  "ar"  or 
"ss"  page.  Unfortunately,  the  prototype  did  not  provide  that  information  in  either  of  the  pages. 
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Deriving  Design  Ideas  On  Side  Menus 

The  first  level  page  of  the  lUB  site  contained  ten  main  menus  and  each  menu  had  three  descriptors  underneath.  In  addition  to 
the  main  menus,  the  page  included  more  items  on  the  right-hand  side  and  the  upper  part  of  the  page,  which  the  design  team  called 
“side  menus.”  The  existing  web  site  did  not  have  side  menus.  Figure  1 and  2 depict  such  difference  in  the  menu  structure  of  the 
first  level  page  between  the  new  prototype  and  the  existing  site. 


<Figure  1 > First  level  page  *s  menu  structure  in  the  new  prototype 
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<Figure  2>  > First  level  page’s  menu  structure  in  the  existing  site 
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The  idea  of  having  side  menus  in  addition  to  main  menus  arose  from  the  team’s  desire  to  maximize  the  use  of  screen 
space.  There  have  been  research  articles  reporting  that  some  people  do  not  scroll  down  while  they  browse  menus  on  the  page.  In 
addition,  researchers  like  Jakob  Nielsen  have  been  arguing  for  the  breath  of  menu  over  the  depth. 

Initially,  the  team  did  not  have  a clear  understanding  about  the  nature  of  the  menus  appropriate  to  each  group  of  the 
menus.  Through  usability  tests  and  interviews  with  test  participants,  however,  it  was  found  out  that  users  had  a certain  and 
unified  viewpoint  on  the  side  menus.  They  tended  to  view  them  as  a “quick”  way  to  access  “specific”  and  “frequently  searched” 
information.  The  specific  and  frequently  searched  information  are  the  ones  highly  identifiable  to  most  users  of  the  site.  In  the 
case  of  a university  web  site,  access  points  to  e-mail  system,  grading  system,  or  commonly  iBed  knowledge  base  are  some 
examples.  This  finding  derived  from  the  interviews  and  the  observations  of  the  paths  test  participants  took  while  performing  the 
tasks.  Firstly,  participants  requested  that  the  information  presented  through  the  side  menus  should  be  replicated  somewhere  in 
the  main  menus.  They  wanted  to  be  sure  that  the  information  was  available  to  them  even  if  they  would  ignore  the  side  menus 
entirely.  This  indicated  that  the  main  function  of  the  side  menus  was  a way  to  access  certain  information  quickly  rather  than  a 
primary  provider  of  information.  Secondly,  the  prototype  subjected  to  the  usability  test  had  a couple  of  side  menus  whose  nature 
was  somewhat  general  such  as  “how  to  apply”  and  “computing  help.”  To  find  admission  and  computer  help  information, 
participants  used  main  menus  rather  than  side  menus  because,  they  didn’t  know  what  specific  information  they  needed  to  find  out 
yet  and  wanted  to  ensure  they  were  browsing  comprehensive  list  of  information  available  in  the  site.  This  indicated  that  the 
information  presented  through  the  side  menus  should  be  specific. 

Having  identified  the  nature  of  information  appropriate  for  the  side  menus,  the  design  team  then  discussed  kinds  of 
information  and  a naming  strategy  appropriate  for  the  side  menus.  By  examining  keyword  search  statistics  of  the  existing  lUB 
site,  the  team  found  out  that  three  most  specific  and  frequently  searched  items  were  e-mail,  INSITE,  and  Knowledge  Base, 
INSITE  was  a student  service  system  through  which  students  could  view  or  update  their  address,  class  schedules,  grades,  tuition 
and  fee  bills,  etc.  Knowledge  Base  was  a database  of  university  computing  help.  As  for  the  menu  names,  the  team  adopted  the 
actual  names  of  the  systems.  The  general  public  might  not  understand  what  the  names  meant  however  people  belonging  to  the 
community  would  understand  them.  Since  the  side  menus  were  frequently  searched  information,  targeting  people  belonging  to 
the  community  seemed  to  be  appropriate. 
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Conclusion 

Analyzing  usability  test  data  to  fulfill  true  purpose  of  the  usability  test  is  not  always  straightforward.  Design  teams  conduct 
usability  tests  usually  to  measure  the  degree  of  usability  of  their  design  (i,e,,  usability  indices)  and  gather  ideas  for  improvement 
(i,e,,  design  ideas.)  This  paper  described  how  lUB  web  team  analyzed  test  data  to  meet  such  purposes.  In  conclusion,  it  is 
important  to  consider  nature  of  the  design  when  analyzing  test  data.  In  addition,  the  process  of  distilling  findings  from  data  and 
linking  the  findings  to  design  ideas  is  an  important  part  of  analysis. 

References 

Frick,  T.,  Corry,  M.,  Hansen,  L.,  & Maynes,  B,  (1995).  Design-Research  for  the  Indiana  University  Bloomington  World 
Wide  Web:  the  “Limestone  Pages”.  [Online],  Available:  http://education,indiana,edu/ist/faculty/iuwebrep.htmlIndiana 
University,  Homepage  of  the  Indiana  University.  [Online],  Available:  http://www, indiana.edu 


A Study  on  Describing  the  Field  Identity  of  Educational 
Communications  in  Turkey 


Cengiz  Hakan  Aydin 
Anadolu  University,  Turkey 


Abstract 

Although  educational  communications  has  been  accepted  as  an  independent  field  of  study  for  nearly  40  years,  there  is  still  no 
agreement  on  its  conceptual  definition  and  boundaries  in  Turkey. 

This  study  was  conducted  by  using  the  literature,  and  the  experts’  opinions  to  describe  the  field  identity  of  educational 
communications.  The  researchers  contributing  to  theoretical  development  of  the  field  and  practitioners  developing  products  of 
educational  communications  in  Turkey  were  selected  to  form  the  expert  panel.  Then,  a three-stage  Delphi  technique  was  used  to 
collect  opinions  of  experts  who  have  been  geographically  dispersed  and  hard  to  bring  together.  Data  about  the  experts ' opinions 
on  both  the  different  aspects  of  educational  communications  and  the  structure  of  the  questionnaire  were  collected. 

The  study  has  shown  that  opinions  of  the  researchers  and  practitioners  on  these  competencies,  issues,  and  trends  are  similar 
except  the  "describing  and  introducing  the  field”  category,  and  a few  of  the  items.  In  the  further  analysis  of  the  data,  16 
professional  competencies,  10  current  issues,  and  19  future  trends  were  identified. 

Introduction 

The  term  “educational  communications”  is  used  to  refer  a field  of  study  derived  from  the  efforts  of  solving  problems  and 
needs  of  human  beings  about  learning  through  developing  technologies  by  using  theories  and  principles  of  many  other  fields.  In 
other  words,  educational  communications  is  another  name  for  the  field  of  educational  technology.  Due  to  long  history  of 
educational  television,  the  term  “educational  communications”  has  been  used  instead  of  educational  technology  for  years.  Thus, 
educational  communications  is  used  through  out  this  paper. 

Although  educational  communications  has  been  accepted  as  an  independent  field  of  study  for  more  than  40  years,  there  have 
always  been  many  discussions  and  disagreements  on  its  conceptual  definition  and  boundaries  (Hackbarth,  1996;  Heinick,  1984). 
According  to  Seattler  (1990)  it  looks  like  these  conflicts  will  always  be. 

This  situation  causes  difficulties  for  introducing  the  field  to  people  working  in  other  fields  as  well  as  new  comers.  It  also 
grounds  problems  in  coordination  between  research  and  practice  in  the  field.  These  problems  are  experienced  more  intensely  in 
Turkey  because  of  diverse  interpretations  of  the  field. 

However,  in  order  to  improve  and  extend  the  acceptance  as  a mature  field  of  study,  these  sorts  of  problems  must  be  overcome. 
By  overcoming  these  sorts  of  problems  in  a field,  theoretical  supports  might  be  provided  to  the  practices,  instructional  programs 
and  research  studies  might  be  directed,  people  working  in  the  field  might  improve  themselves,  decisions  that  might  be  effective 
future  directions  of  the  field  can  be  supported,  and  the  field  might  be  introduced  to  the  others  and  new  comers  (Marriner,  1989). 

Unfortunately  there  is  almost  no  or  a few  study  on  either  describing  the  field  identity  of  educational  communications  or 
establishing  a consensus  on  the  definition  and  boundaries  of  the  field.  The  first  and  one  of  the  significant  works  was  Alkan’s 
philosophical  thoughts  about  use  of  technology  in  education,  which  was  completed  in  1977.  Since  this  work  several  significant 
works  such  as  (^ilenti  (1983),  Giiler  (1990),  Barkan  (1994)  have  also  done.  But  all  of  them  are  based  on  translations  and 
interpretations  of  foreign  researchers’  works  and  represent  their  personal  views.  So  that,  almost  all  define  the  field  where  they 
stand  from  and  this  situation  cause  conflicting  problems. 

In  addition,  multitude  of  variables  influencing  the  field  and  mobile  nature  of  the  field  make  harder  to  come  a consensus  on 
different  aspects  of  the  field,  too. 

Another  important  reasons  of  having  no  agreement  on  the  aspects  of  the  field  might  be  that  there  are  very  few  people  work 
directly  educational  communications.  Most  of  these  people  are  separated  geographically  and  hard  to  bring  them  together. 

Delphi  technique  is  one  of  the  useful  applications  for  group  decision-making  without  face-to-face  interaction.  The  major  idea 
beneath  this  technique  is  that  “more  heads  are  better  than  one”.  Delphi  technique  is  described  as  “...  an  anonymous,  independent, 
noncompetitive  survey  of  experts  to  obtain  consensus  without  necessarily  involving  group  meetings.  The  technique  essentially 
entails  a series  of  surveys  using  the  same  experts,  each  survey  dependent  upon  the  responses  of  the  previous  one”  (Jonassen, 
Hannum,  & Tessmer,  1989).  Although  its  several  limitations,  Delphi  technique  has  been  used  for  many  years  for  especially  group 
decision-making  and  future  predictions. 

Statement  of  Purpose 

This  Study  purposes  to  determine  different  aspects  of  educational  communications  such  as  theoretical  backgrounds,  research 
topics,  emphasis  areas,  roles  and  competencies,  major  issues  and  problems,  future  direction;  therefore,  it  was  intended  to  clarify 
the  field  identity  of  educational  communications.  Sp  ecifically,  the  following  research  questions  were  addressed: 

What  kinds  of  phenomena  have  been  effective  on  evolution  of  the  field? 

What  are  the  emphasis  areas  and  research  topics  in  the  field? 

What  are  the  competencies  of  experts  of  educational  communications? 

What  are  the  major  problems,  issues  and  future  directions  in  the  field? 
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Method 

Since  this  study  requires  group  decision-making  and  future  predictions,  Delphi  technique  has  been  chosen  for  especially 
judgmental  data  collection.  In  addition  to  Delphi,  theoretical  works  and  research  studies  of  foreign  experts  are  also  used  for 
particularly  phenomenal  data  collection. 

Participants 

This  investigation  was  conducted  with  two  experts  groups.  We  hoped  that  our  results  would  be  of  interest  to  academicians 
who  either  conducted  research  or  published  a distinguished  work  (paper,  book,  etc.)  to  contribute  the  body  of  scientific 
knowledge  in  educational  communications  and  corporate  trainers  who  are  experts  work  in  either  a private  or  public  sector 
organization  and  design,  develop,  evaluate,  manage  instructional  activities. 

It  was  intended  to  reach  out  the  whole  population  since  there  are  a few  people  working  in  the  field.  Fifty  (5)  participants 
involved  in  Phase  1,  but  only  thirty  (30)  of  them  managed  to  complete  Phase  3.  Half  of  these  participants  were  researchers  (all 
academicians)  and  others  were  practitioners  (all  corporate  trainers).  Most  of  the  participants  who  dropped  the  investigation  were 
practitioners. 

Instrument  and  Treatment 

The  Delphi  instrument  unfolded  in  three  phases.  Phase  1 consisted  of  an  introductory  letter  and  a request  for  participants  to 
assist  in  offering  the  aspects  of  the  field  worth  for  investigating.  In  addition  to  these  instruments  conversations  were  conducted 
mostly  through  telephone  calls.  Submitted  statements  were  compiled,  analyzed  to  identify  the  aspects  of  educational 
communications  for  investigating.  As  a result  of  this  phase  10  aspects  were  identified  among  the  offered  ones:  History, 
definition,  theoretical  foundations,  research  topics,  emphasis  areas,  experts’  roles  and  competencies,  instructional  programs, 
major  issues  and  problems,  future  trends  and  directions,  process  of  educational  communications. 

After  having  conducted  a literature  review,  we  determined  143  statements  and  developed  a Likert  type  questionnaire 
including  these  statements  for  Phase  2.  In  Phase  2 of  the  study,  this  questionnaire  was  sent  to  each  respondent  of  Phase  1,  it  was 
asked  each  person  to  rate  each  of  the  143  statements  as  to  how  strongly  they  anticipated  the  statement.  In  the  Likert  type 
questionnaire  the  Endpoint  1 was  to  indicate  that  the  participants  strongly  disagree  the  statement,  and  Endpoint  5 to  indicate  that 
the  participants  strongly  agree  the  statement.  Also  it  was  provided  to  indicate  their  thoughts  on  the  overall  structure  of  the 
questionnaire. 

After  reaching  expected  return  rate  (17  for  each  group),  we  analyzed  the  data  and  concluded  that  most  of  these  statements 
were  phenomenal  data  and  required  no  further  investigation.  In  other  words,  the  participants  indicated  that  these  must  be  accepted 
as  they  are  so  that  they  must  not  be  included  into  the  questionnaire.  These  statements  were  excluded  from  the  questionnaire 
owing  to  the  consistency  with  the  literature.  As  a result  this  Phase,  143  discrete  statements  was  reduced  to  70  and  these  were 
categorized  into  three  groups:  competencies,  problems,  and  trends. 

In  Phase  3,  the  last  version  of  the  questionnaire  sent  to  the  determined  participants  of  Phase  1.  Unfortunately  the  ones  did  not 
respond  the  Phase  1 and  Phase  2 repeated  their  attitudes  about  responding  even  though  we  reach  most  of  them  through  phone  and 
requested  to  respond.  After  receiving  1 5 responds  for  each  group,  the  data  analyzed. 

Through  this  three-phased  Delphi  study  both  a new  standard  questionnaire  developed  and  experts’  viewpoints  were  integrated. 

Results 

In  Phase  3,  we  calculated  the  mean  score  on  all  items  to  be  3,87,  with  a standard  deviation  of  .43.  These  and  frequency  of 
scores,  in  general  terms,  indicate  that  the  average  score  on  the  70  items  leaned  toward  an  agreement -but  not  a strong  one-  side  of 
the  scale.  Also  it  can  be  observed  that  there  is  not  a big  variation  among  all  participants’  responses.  When  the  two  groups  of 
researchers  and  practitioners  were  compared,  it  was  found  that  the  researchers  were  not  only  more  agreed  on  the  statements  than 
practitioners  (mean  scores  of  4.0  versus  3.73)  but  they  also  show  a few  variations  in  their  responses  as  shown  by  the  decrease  in 
standard  deviation  (.028  versus  0.52)  (Table  1). 


Table  1:  Means  and  Standard  Deviation  of  Groups 


Groups 

Item  Mean  Score 

Standard  Deviation 

Researchers 

4.00 

0.28 

Practitioners 

3.73 

0.52 

Total 

3.87 

0.43 

The  analysis  of  the  results  revealed  that  participants  rated  38  of  70  items  with  mean  scores  higher  than  3.87.  This  finding 
might  be  interpreted  as  that  there  is  a strong  agreement  on  these  statements  related  to  the  field. 

It  was  also  compared  the  items  rated  by  the  researchers  with  those  rated  by  the  practitioners.  An  analysis  by  t test  of  non- 
independent  items  yielded  a statistically  significant  difference  on  1 1 of  the  70  items  between  the  two  groups  at  or  below  an  alpha 
level  of  .05.  Except  one  of  the  item,  related  to  the  trend  about  decrease  in  research  studies,  in  all  other  cases,  researchers  rated  the 
items  to  be  more  agreed  than  practitioners. 

Discussions 

The  results  reached  in  this  investigation  are  given  below  into  four  groups  according  to  the  research  questions. 

Phenomena  Influence  the  Evolution  of  the  Field 

This  part  heavily  depends  on  phenomenal  data  collected  from  literature.  The  literature  review  shown  that  the  term 
“educational  communications”  refer  to  a field  of  study  derived  from  the  efforts  of  solving  problems  and  needs  of  human  beings 
about  learning  through  developing  technologies  by  using  theories  and  principles  of  many  other  fields.  The  roots  of  the  field  can 
be  traced  back  to  the  works  of  Comenius.  Educational  communications  was  considered  as  a movement  until  the  mid  1960s  and 
then  it  has  accepted  as  a field  of  scientific  study.  During  the  period  of  being  considered  as  a movement,  it  was  focused  on  the 
effective  presentations  of  the  content  with  an  instructor  and  the  audio-visual  aids.  During  1950s,  with  the  influences  of 
communication,  system  and  particularly  learning  theories,  the  focus  shifted  on  development  of  instructional  materials  and 
transferring  learning  theories  into  instructional  activities.  The  previous  focus  rooted  the  educational  media  emphasis  area  and 
latter  leaded  the  development  of  instructional  design  models  and  theories  (Ely,  1996). 

Most  of  the  current  practices  and  research  studies  are  related  to  the  instructional  design.  The  results  of  Delphi  show  that  the 
highest  means  scores  observed  on  statements  related  to  instructional  design.  In  this  context,  it  might  be  claimed  that  the 
investigation  supports  the  idea  that  instructional  design  has  a significant  role  in  educational  communications. 

On  the  other  hand,  literature  reveals  that  with  the  influence  of  constructivist  approach,  most  of  the  educational 
communication  efforts  changed  toward  design  or  organization  of  learning  environments  where  learners  learn  through  interacting 
with  authentic  contexts  (Winn  & Snyder,  1 996).  During  the  Delphi  investigations  all  participants  agreed  on  the  items  related  to 
this  issue;  therefore,  this  can  be  considered  as  an  indicator  of  support  of  this  study  to  the  literature. 

Emphasis  Areas  and  Research 

This  part  also  depends  on  phenomenal  data  collected  from  literature.  Research  studies  show  us  the  topics  that  educational 
communication  specialists  are  dealing  with  (Thompson,  Simonson  & Hargrave,  1996).  Some  of  the  variables  are  motivation, 
transfer,  learning  context,  feedback,  learning  strategies,  learning  styles,  attention  focusing,  confidence,  time-on-task,  retention. 
Experts  use  these  sorts  of  variables  to  design  effective,  efficient,  and  appealing  learning  environments.  This  also  supports  the 
definition  of  educational  communications,  which  refer  to  a field  of  study  derived  from  the  efforts  of  solving  problems  and  needs 
of  human  beings  about  learning  through  developing  technologies  by  using  theories  and  principles  of  many  other  fields. 

On  the  other  hand,  there  are  so  many  emphasis  areas  in  the  field.  But,  this  might  display  the  field  too  broad  and  unorganized,  and 
might  make  harder  to  clarify  the  theoretical  boundaries  of  the  field.  In  order  to  eliminate  these  dangers,  provide  leadership  for  the 
studies,  and  make  the  introduction  of  the  field  easier,  there  might  be  a need  for  categorizing  the  emphasis  areas. 

Thus,  we  categorized  all  areas  into  three  major  emphasis  areas  after  having  examined  the  research  studies,  practices, 
developments  and  trends  of  the  field.  These  are  (a)  instructional  design,  (b)  educational  media,  and  (c)  human  resources 
development.  The  items  included  in  the  questionnaire  are  falling  into  one  of  these  areas.  This  situation  supports  this 
categorization  of  emphasis  areas. 

Competencies 

There  are  several  sources  about  the  competencies  of  educational  communication  specialists  in  the  literature.  By  using  these 
sources  and  views  of  experts  3 1 competencies  were  determined.  However  after  having  a detailed  investigation,  we  have  decided 
to  combine  related  ones  so  that  we  could  manage  to  lessen  the  number  of  competencies.  As  a result  of  these  efforts,  16  major 
competencies  for  educational  communication  specialist  were  concluded  on.  These  competencies  have  shown  similarity  with  the 
competencies  determined  for  instructional  designer  by  AECT  and  NSPI  Task  Force.  These  are: 

• Analyze  the  needs 

• Determine  the  projects  appropriate  for  instructional  design 

• Describe  the  learner  characteristics 

• Analyze  the  characteristics  of  organizational  environment 

• Conduct  task/content/job  analysis 

• Write  performance  objectives 

• Develop  achievement  tests 

• Sequence  the  objectives 

• Select  the  instructional  strategy 

• Design  the  educational  media 

• Evaluate  the  outputs  of  education,  instruction  and  training 
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• Design  the  management  systems  of  learning 

• Communicate  verbally,  visually  and  aurally 

• Consult  for  the  individual  and  career  development 

• Plan  and  monitor  his/her  own  development 

• Introduce  educational  communication  field  to  the  other  and  extend  its  applications 

On  the  other  hand,  although  instructional  design  is  a well  known  and  highly  accepted  emphasis  area  of  educational 
communications  all  over  the  World,  there  are  very  few  people  have  shown  interest  and  worked  in  the  this  field  in  Turkey.  Having 
no  Turkish  books  on  instructional  design  can  also  show  how  Turkish  scholars  pay  attention  to  this  area.  However  the  Higher 
Education  Council  required  an  instructional  design  course  for  all  the  Computer  Teaching  and  Instructional  Technology 
undergraduate  programs  of  education  colleges  but  contents  of  most  of  these  courses  are  far  from  instructional  design  and  usually 
more  related  to  the  theories  of  teaching  and  learning.  Only  in  Educational  Communications  and  Planning  Department  of  Anadolu 
University,  several  courses  in  varying  levels  (graduate,  undergraduate)  related  to  the  instructional  design  such  as  “Introduction  to 
Instructional  Design,  Instructional  Design  Models  and  Theories,  Needs  Assessment,  Instructional  Media  Design  have  been 
offered  since  1994.  It  is  our  hope  that  there  will  be  more  promising  efforts  to  show  the  power  and  the  importance  of  instructional 
design. 

Major  Problems  and  Future  Directions 

Based  on  the  data  collected  form  participants  and  the  literature,  16  major  problems  identified  and  included  the  last  version  of 
the  questionnaire.  However,  the  results  exposed  that  there  is  an  agreement  on  only  6 of  1 6 items. 

Among  these  6 problems,  the  one  related  to  the  separation  of  evaluation  as  an  individual  field  apart  from  the  educational 
communications  is  interesting.  It  can  be  noticed  in  the  literature  that  evaluation  has  been  taken  as  a separate  field  of  study  for 
years  but  there  is  a tendency  that  claims  that  evaluation  is  one  of  the  main  parts  of  instructional  design  and  cannot  be  regarded 
separate  from  this  process.  Results  of  the  questionnaire  demonstrate  that  there  is  a significant  difference  on  this  item  between  the 
groups.  Researchers  strongly  believe  in  such  a problem  while  practitioners  are  not  sure  about  the  existence  of  this  problem.  We 
interpret  this  as  practitioners  show  more  enthusiasm  and  pay  more  attention  on  evaluation  today  than  they  did  in  the  past. 

The  item  related  to  the  research  in  the  field  is  also  interesting  one.  This  item  includes  the  idea  that  the  attention  in  the  field 
focuses  much  more  on  practice  than  research.  Literature  reveals  the  existence  of  such  a problem  (Gentry  & Csete,  1991). 
However,  the  results  of  the  questionnaire  indicate  that  this  is  not  a true  statement  anymore  (mean  score  of  2.77).  In  other  words 
the  participants  do  not  agree  with  this  statement.  Though  there  is  a strong  disagreement  on  this  item  between  the  groups.  For  only 
this  item  researchers  did  not  supported  the  statement  more  strongly  than  practitioners  (means  scores  of  2.40  versus  3.13).  We 
interpreted  this  as  an  increase  in  research  studies  conducted  during  the  recent  years  and  because  of  this  researchers  believe  that 
they  do  conduct  more  research  now  compare  to  they  did  in  the  past.  In  addition,  parishioners  are  not  able  to  follow  these  studies 
as  good  as  researchers. 

Another  problem  uncovered  through  this  investigation  is  that  an  uncertainty  about  the  theoretical  boundaries  of  the  field  is 
still  continuing.  The  main  reason  of  this  problem  is  the  lack  of  enough  research  or  other  studies  on  this  topic.  Also  having  no 
professional  organization  and  publication  about  the  field  of  educational  communications  in  Turkey  supports  this  problem.  The 
field  usually  is  perceived  as  a field  of  study  deals  with  bringing  new  technological  tools  into  education.  Because  of  this  narrow 
view  investments  generally  go  to  the  production  of  new  tools.  The  wide  spread  structure  of  the  descriptive  studies  related  to  the 
field  also  supports  the  problem.  This  problem  causes  the  diverse  structure  of  instructional  programs,  misconceptions  about  the 
field  and  its  place  in  the  society,  lack  of  relationship  with  other  fields. 

We  concluded  that  the  main  reason  under  beneath  most  of  these  problems  is  lack  of  clarification  of  scope  of  the  field.  This 
main  problem  creates  more  problems.  Also  literature  is  full  of  same  ideas  on  this  problem  (e.g.  Gentry  & Csete,  1991;  Ely,  1995). 

On  the  other  hand,  the  last  version  of  the  questionnaire  included  23  statements  related  to  the  future  trends. 

Among  all  these  statements,  only  for  the  one  related  to  the  being  a leader  in  the  educational  changes  is  there  a significant 
difference  between  the  groups.  Results  indicate  that  practitioners  are  more  suspicious  than  researcher  about  future  the  roles  of  the 
field  in  educational  settings.  The  economic  problems  of  the  Country  make  graduates  of  the  educational  communication  programs 
hard  to  find  decent  Jobs.  This  situation  might  affect  the  responds  of  the  participants,  especially  practitioners. 

Participants  also  indicate  that  graduates  of  the  programs  will,  sama  as  today,  have  almost  no  chance  to  be  employed  in  public 
organizations  such  as  ministry  of  education,  etc.  The  economy  experts  are  expressing  the  misuse  of  human  resources  in  the 
governmental  and  public  organizations,  and  harmfulness  of  this  situation  on  economy.  We  think  that  if  flie  graduates  of 
educational  communications  could  find  opportunities  to  work  in  these  organizations,  there  might  be  a slight  chance  to  solve  this 
misusage.  One  of  the  emphasis  areas  of  the  field  is  human  resources  development  and  the  graduates  have  skills  and  attitudes 
toward  correct  usage  of  human  resources. 

Another  interesting  finding  is  that  participants  do  not  agree  on  the  increase  in  distance  education.  This  result  conflicts  with 
the  literature.  We  interpret  this  as  that  although  Turkey  has  a long  history,  this  technology  (distance  education)  is  not  accepted 
and  it  is  not  extended,  as  it  has  to  be. 

The  results  about  the  use  of  textbooks  in  the  future  is  also  interesting  and  conflicting  with  the  literature.  The  participants 
agree  on  that  the  textbooks  will  still  be  used  widely  in  almost  all  instructional  settings  although  literature  indicates  a gradual 
decrease  in  use  of  textbooks.  We  think  that  use  of  very  old-fashioned  instructional  approaches  in  almost  every  grade  of  public 
education  and  misleading  applications  of  technology  integration  strengthen  the  idea  that  textbooks  are  the  unique  and  most 
reliable  sources  of  knowledge  and  increased  the  fear  of  technology  usage. 


On  the  other  hand,  results  related  to  the  statements  about  different  aspects  of  instructional  design  such  as  increase  in 
applications  of  constructivist  approach,  providing  more  learner  control,  wide  spread  use  of  team  learning  show  consistency  with 
the  literature.  However,  applications  prove  that  although  experts  believe  in  the  importance  of  these  applications,  they  find 
difficult  to  apply  them  in  their  instructions  in  Turkey. 

Same  as  instructional  design  items,  participants  agree  on  the  statements  related  to  educational  media  such  as  development  of 
virtual  learning  environments,  use  of  digital  technologies,  wide  speared  use  of  computer  networks.  There  is  an  increase  in 
intensity  of  using  web  and  other  computer  related  opportunities  for  instructional  purposes.  When  these  practices  are  observed,  it 
can  be  identified  that  most  of  these  efforts  have  no  pedagogical  background  and  because  of  this  they  fail  to  help  people  learn. 
However,  there  is  a tendency  to  include  more  and  modem  pedagogical  aspects  into  instructional  practices  in  Turkey. 

Conclusions  and  Further  Investigations 

We  Strongly  believe  that  the  most  important  result  of  this  study  is  that  it  is  hard  to  draw  strict  boundaries  of  the  field  of 
educational  communications  due  to  its  peripatetic  nature  and  continues  influences  of  the  other  fields.  However,  we  propose  that 
these  sorts  of  decision-making  and/or  future  prediction  studies  should  be  conducted  regularly  to  keep  up  the  developments  in  the 
field  because  fuzzy  or  conflicting  ideas  about  the  varying  aspects  of  educational  communications  might  create  serious  problems. 
Updated  studies,  for  sure,  will  not  only  provide  help  for  practitioners  and  researchers  but  also  make  leadership  easier. 

Turkey  definitely  needs  more  of  these  sorts  of  studies  to  be  able  to  establish  a consensus  on  different  aspects  of  the  field 
including  names,  programs,  definitions,  trends,  applications,  issues,  etc.  The  first  step  might  be  a professional  organization  that 
will  have  the  mission  of  providing  leadership  in  the  field  in  Turkey.  So  that  publications  and  meetings  can  be  supported  easily  to 
provide  good  communication  channels  among  the  people  in  the  field,  standard  programs  can  be  offered  to  bring  up  new  people 
into  the  field,  and  demonstrate  effective,  efficient,  appealing  examples  of  educational  communication  practices. 

In  order  to  increase  the  validity  and  reliability  of  these  sorts  of  research  findings,  researchers  should  include  as  many  experts 
dealing  with  different  aspects  of  the  field  such  as  employers  of  educational  communication  graduates  and  foreign  experts  as 
possible.  So  that  new  ideas  might  come  up  and  stronger  agreements  might  be  achieved.  However,  Delphi  technique  is  a time 
consuming  activity.  Most  of  the  time  usually  spends  for  mailing  and  analyzing  the  responses  but  new  technological  tools  provide 
researchers  numerous  advantages.  E-mail  is  one  of  these  advantages.  E-mail  based  a Delphi  study  will  of  course  save  time.  Also 
using  some  content  analysis  tools  with  e-mail  might  provide  researchers  flexibility  of  using  different  type  of  questionnaires  such 
as  the  ones  includes  open-ended  items. 
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Abstract 

In  this  exploratory  experimental  study,  135  pre-service  teachers  developed  an  instructional  plan  for  a case  study  within  the 
MIMIC  (Multiple  Intelligent  Mentors  Instructing  Collaboratively)  computer-based  environment  Three-dimensional,  animated 
pedagogical  computer  agents,  representing  constructivist  and  instructivist  approaches  to  instructional  planning,  served  as 
instructional  mentors  within  the  environment  and  were  available  to  provide  suggestions.  The  research  design  was  comprised  of 
two  two-factor  MANOVAs  with  the  instructivist  agent  (present,  absent)  and  constructivist  agent  (present,  absent)  serving  as  the 
two  factors,  with  two  groups  of  dependent  measures  -awareness  and  attitude.  Additionally,  the  value  of  the  agents  and  overall 
differences  between  high  and  low  performers  were  investigated,  I^garding  awareness,  main  effects  for  the  presence  of  the 
constructivist  agent  indicated  that  when  the  constructivist  agent  was  present  participants  tended  to  report  a change  in  their 
perspective  of  instructional  planning,  reflected  less  on  their  thinking,  and  developed  instructional  plans  rated  as  more  constructivist 
in  underlying  pedagogy.  Regarding  attitude,  a main  effect  for  the  presence  of  the  instructivist  agent  indicated  that  when  the 
instructivist  agent  was  present,  participants  reported  a more  negative  disposition  regarding  instructional  planning.  Results  are 
discussed  in  terms  of  the  impact  on  teaching  instructional  planning  to  pre-service  teachers. 

Introduction 

In  the  field  of  instructional  design,  there  are  diverse  theories  and  approaches  to  instruction  (e,g.,  (Driscoll,  2000)),  For  pre- 
service  teachers,  the  importance  of  seeing  how  these  theories  relate  to  real  instructional  problems  is  critical.  Two  prominent  yet 
differing  approaches  to  instructional  planning  are  systematic  instructional  planning  (referred  to  here  as  instructivism),  based  on 
an  objectivist  epistemology,  and  constructivism,  based  on  an  interpretivist  epistemology  (Jonassen,  1991),  These  two 
philosophical  approaches  lead  to  different  understandings  of  human  cognition  and  affect  both  the  instruction  that  is  developed 
and  what  evaluations  are  feasible  and  appropriate  (Roblyer,  1996;  Yarusso,  1992), 

With  its  objectivist  epistemic  roots  regarding  knowledge,  the  underlying  assumption  of  traditional  instructional  planning  is 
that  knowledge  can  and  should  be  transmitted  from  teacher  to  student.  An  instructivist  approach  to  instructional  planning 
emphasizes  knowledge  transfer  and  teacher- centered  learning  environments,  where  skills  are  taught  sequentially,  incorporating 
individualized  work  with  traditional  assessment  methods  (Roblyer,  Edwards,  & Havriluk,  1997),  This  type  of  systematic 
approach  to  instruction  has  been  shown  to  be  effective  due  to  its  focus  on  clearly  identifying  goals  and  systematically  developing 
instructional  activities  and  assessment  that  lead  to  the  attainment  of  the  goals  (Reiser  & Dick,  1996), 

In  contrast,  constructivism  has  its  epistemic  roots  in  interpretivism,  which  maintains  that  knowledge  is  personally  constructed 
within  individuals  and  does  not  exist  external  to  the  individual.  The  constructivist  approach  tends  to  focus  on  more  student - 
centered  environments,  to  provide  activities  that  facilitate  knowledge  construction  and  generative  learning  (e,g,,  Wittrock,  1990), 
Driscoll  (2000)  describes  five  attributes  of  constructivist  instruction:  1)  embedding  learning  in  complex  and  realistic 
environments;  2)  providing  for  social  negotiation;  3)  supporting  multiple  perspectives  and  use  of  multiple  modes  of 
representation;  4)  encouraging  ownership  in  learning;  and,  5)  nurturing  self-awareness  of  the  knowledge  construction  process 
(pp,  382-383),  To  implement  these  features  as  part  of  the  constructivist  planning  process,  pre-service  teachers  must  learn  to 
emphasize  the  process  of  learning  more  than  the  end  product.  Constructivist  approaches  have  been  found  to  be  particularly 
beneficial  for  developing  meaningful  learning  activities  and  engaging  students  in  hiAer  order  thinking  (Jonassen,  Peck  & 
Wilson,  1999) 

One  way  to  authentically  demonstrate  these  two  distinct  approaches  to  pre-service  teachers  would  be  through  seasoned 
professionals  modeling  the  approaches  in  the  context  of  a real  instructional  situation.  Exposure  and  interaction  with  several 
experts  describing  instructional  content  matter  from  different  points  of  view  can  be  very  rewarding  for  the  learner  (Laurel,  Oren, 
& Don,  1990)  and  can  help  the  learner  to  establish  the  best  personalized  approach  to  understanding  the  content.  Further,  such 
exposure  to  multiple  pedagogical  perspectives  could  enhance  pre-service  teachers’  cognitive  flexibility  by  requiring  them  to 
independently  consider  alternative  points-of-view.  Viewing  an  instructional  problem  from  multiple  perspectives  is  also  desirable 
for  promoting  reflective  thinking  and  problem  solving,  qualities  important  for  pre-service  teachers  who  are  learning  to  be 
teaching  professionals.  Further,  as  Jonassen  (1997)  describes,  “instructional  planning  is  an  archetypal  ill-structured  problem 
because  “the  designer  is  constrained  by  circumstances,  though  in  most  design  problems,  there  are  a variety  of  solutions  each  one 
of  which  may  work  as  well  as  any  other  (p,  69).”  Given  that  more  than  one  problem-solving  path  is  possible  to  reach  a solution, 
the  ability  for  a pre-service  teacher  to  take  multiple  perspectives  on  instructional  planning  is  appropriate  and  necessary.  Overall 
while  it  may  be  beneficial  for  our  pre-service  teachers  to  see  their  role  in  the  classroom  from  multiple  pedagogical  perspectives 
(Bennett  & Spalding,  1992),  devising  this  sort  of  experiential  exposure  is  difficult  to  implement  with  human  instructors. 
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A promising  possibility  for  demonstrating  and  experiencing  different  instructional  approaches  is  through  computer-based 
agents  serving  as  a pedagogical  mentors  (Baylor,  in  press).  A software  agent  is  an  independent  computer  program  operating 
within  software  environments  such  as  operating  systems,  databases,  or  computer  networks  (Roesler  and  Hawkins,  1994).  Agents 
appear  to  have  the  characteristics  of  an  animate  being,  and  simulate  a human  relationship  by  doing  something  that  another  person 
could  otherwise  do  for  you  (Seiker,  1994).  Animated  pedagogical  agents  have  lifelike  qualities,  and  can  employ  verbal 
instructional  explanations  together  with  nonverbal  forms  of  communication  (e.g.,  gaze,  gesture,  conveying  emotion)  in 
interacting  with  the  learner.  Along  this  line,  Lester  and  colleagues  (Johnson,  Rickel,  & Lester,  2000;  Lester,  Stone,  & Stelling, 
1999)  have  suggested  that  life-like  agent  characters  are  ideal  to  serve  as  tutors,  coaches  or  guides  in  knowledge-based  learning 
environments. 

Further,  learners  treat  computer-based  agents  as  human,  even  when  the  computer  interface  is  not  explicitly  anthropomorphic 
(Reeves  &Nass,  1996). 

Building  upon  Laurel’s  (1990;  1997)  suggestion  for  agents  to  represent  different  “roles”  such  as  characters  in  a play,  the  next 
question  to  consider  is  whether  agents  could  represent  different  instructional  roles  as  pedagogical  mentors.  While  the  idea  of 
representing  multiple  instructional  roles  through  computer-based  media  has  been  implemented  in  other  research,  there  have  been 
limited  controlled  studies.  The  ETOILE  system  for  teaching  educational  psychology  principles  (Dillenbourg,  Mendelsohn,  & 
Schneider,  1994)  incorporated  five  agents,  each  labeled  after  the  teaching  styles  they  implement:  Skinner,  Bloom,  Vygotsky, 
Piaget,  and  Papert.  Each  agent  was  implemented  as  an  independent  rule  base  that  was  separated  out  from  the  content  rather  than 
being  domain-specific.  The  five  teaching  agents  implement  decreasing  level  of  directiveness:  Skinner  works  step  by  stop,  Bloom 
makes  larger  steps  but  with  close  control  of  mastery,  Vygotsky  is  based  on  participation,  Piaget  intervenes  only  to  point  out 
problems  and  Papert  does  not  interrupt  the  learner.  The  ETOILE  system  also  includes  a “coach”  agent  that  is  in  charge  of  which 
tutor  is  used  although  the  learner  may  also  select  or  remove  a tutor.  The  ETOILE  system  was  not  designed  for  the  purpose  of 
instructional  research,  but  rather  to  conceptualize  the  underlying  oigineering  principles  for  the  multiple  agents;  consequently, 
there  is  no  empirical  evidence  regarding  its  instructional  impact. 

MIMIC  (Multiple  Intelligent  Mentors  Instructing  Collaboratively)  is  an  agent-based  learning  environment  developed  for  the 
purpose  of  instructional  research  (Baylor,  1999,  2000b).  MIMIC  situates  instructional  planning  within  a specific  context:  a case 
study  of  a thirteen -year  old  girl  struggling  with  the  economics  concepts  of  supply  and  demand.  In  MIMIC,  agents  explicitly 
represent  different  perspectives  of  instructional  planning  (objectivism  and  constructivism)  and  facilitate  pre-service  teachers’ 
internalization  of  these  approaches.  The  animated  three-dimensional  pedagogical  agents  serve  as  scaffolds,  providing  cognitive 
support  to  pre-service  teachers  while  they  write  an  instructional  plan.  The  learner  has  control  over  the  amount  of  assistance  and 
when  it  will  be  provided  by  the  agent(s).  In  related  research,  participants  working  with  both  the  instructivist  and  constructivist 
pedagogical  agents  could  differentiate  between  them  and  could  explicate  the  two  theories  that  they  represented  (Baylor,  in  press). 
The  agents  were  also  reported  to  be  believable  and  useftil.  (Baylor,  in  press). 

Hietala  & Niemirepo  (1998)  suggest  that  the  same  social  factors  that  occur  in  learning  communities  with  human  beings  are 
also  influential  in  a learning  community  consisting  of  multiple  artificial  teaching  and  learning  agents.  They  refer  to  this  aspect  as 
the  need  for  pedagogical  multiplicity  of  teachers,  suggesting  that  the  many  levels  and  complexities  of  the  learning  process  might 
be  alleviated  by  providing  more  alternatives  to  the  learner  via  an  '’extended  family  of  intelligent  agents."  Essentially,  an  agent- 
based  learning  environment  such  as  MIMIC  allows  pre-service  teachers  to  figuratively  “put  on  different  hats”  and  facilitates  them 
in  switching  roles  when  needed  to  solve  an  instructional  problem.  Through  experiential  interaction  with  the  agents,  the  pre- 
service teacher  is  facilitated  in  a deep  approach  to  the  task,  focusing  on  the  meaning  of  the  instructional  planning  process  itself, 
rather  than  a surface  approach  which  would  involve  simply  writing  an  instructional  plan  following  a “recipe.”  In  this  way,  the 
pre-service  teachers'  experience  of  instructional  planning  and  the  specific  meaning  it  has  for  them  could  be  considered  as  the 
most  fundamental  aspect  of  learning  (Marton  & Booth,  1997). 

In  this  exploratory  experimental  study,  it  is  hypothesized  that  the  presence  of  the  instructivist  and  constructivist  agent(s) 
(especially  both  simultaneously)  will  impact  learners’  awareness  of  instructional  planning  by  increasing  their  reflection  during 
the  instructional  planning  process  and  changing  their  perspective  regarding  instructional  planning.  It  is  not  hypothesized  that  the 
presence  of  agents  will  affect  performance  given  that  it  is  generally  found  that  the  presence  of  agents  does  not  significantly 
improve  performance  (Dehn  & van  Mulken,  2000);  however,  it  is  predicted  that  pre-service  teachers’  transformation  of 
awareness  will  be  reflected  by  the  instructivist  or  constructivist  “flavor”  of  their  instructional  plans,  depending  on  which  agent(s) 
are  present.  In  terms  of  attitudes,  it  is  predicted  that  the  presence  of  the  agents  as  scaffolds  would  positively  influence 
participants’  dispositions,  self-efficacy  and  perceived  instrumentality  regarding  instructional  planning. 

Methods 

Sample 

The  sample  consisted  of  135  pre-service  teachers,  in  eight  sections  of  an  “Introduction  to  Educational  Technology”  course. 
As  part  of  this  required  course,  the  participants  had  already  been  taught  an  instructivist  model  of  instructional  planning  (Reiser  & 
Dick,  1996)  and  a constructivist  approach  to  instructional  planning  (Grabe  & Grabe,  2001)  with  identical  course  material  (e.g., 
lectures,  Powerpoint  slides,  assignments,  exams)  across  the  eight  sections.  Participation  in  this  study  was  a required  activity  for 
class  participants,  and  they  received  course  credit  for  participating.  The  mean  age  of  the  sample  was  19.76  years  (5D=2.13).  Of 
those  reporting  ethnicity,  84%  were  Caucasian,  4%  were  Hispanic,  1 0%  were  African  American,  and  2%  were  of  other  groups. 
Of  those  reporting  gender,  21.5%  of  the  sample  were  male  and  78.5%  were  female.  Sixty  percent  (the  majority)  of  the 
participants  were  sophomores,  27%  were  juniors  with  7%  freshman  and  6%  seniors.  In  terms  of  prior  experience  with 
instructional  planning,  participants’  mean  score  was  2.23,  pZ)=.97),  where  l==no  experience  and  5=very  much  experience. 
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indicating  that  overall  they  had  little  prior  experience.  There  were  no  significant  differences  in  age  and  GPA  among  the 
participants  in  all  conditions.  In  terms  of  ethnicity,  gender,  and  year  in  school,  chi-square  analyses  revealed  no  significant 
differences  between  the  groups. 

Multiple  Intelligent  Mentors  Instructing  Collaboratively  (MIMIC)  Environment 

From  the  learner’s  perspective,  MIMIC  consists  of  an  introduction,  a case  study,  blueprints  stage,  plan  stage,  and  assessment 
stage.  The  introduction  begins  with  the  statement  that  “We  are  pleased  that  you  have  decided  to  join  our  educational  consulting 
firm,  ‘Instruction  Inc.’  Given  your  new  skills  in  instructional  planning,  we  have  a project  for  which  we  really  need  your  help,” 
and  briefly  describes  the  case  study  situation  with  thirteen-year-old  Anna  and  her  teacher  Mr.  Lange.  Following  this,  the 
participant  is  instructed  how  to  move  throughout  the  environment: 

Through  our  computer-based  system,  you  will  be  able  to  follow  3 steps  in  devising  your  instructional  solution.  The  three 
components  of  the  project  include  the  following: 

1.  STEP  1:  Blueprints  for  Instruction:  This  is  where  you  will  first  determine  the  goals  of  the  instruction;  in  other  words 
WHAT  you  want  the  learner  to  be  able  to  do  at  the  end  of  the  process. 

2.  STEP  2:  Developing  the  Instructional  Plan:  Here  you  will  get  into  the  "heart"  of  the  planning  process,  and  develop  a 
detailed  plan  that  a teacher  can  use  to  carry  out  the  instruction  as  you  specify. 

3.  STEP  3:  Assessment:  At  this  stage  you  will  determine  how  to  measure  whether  the  learner  actually  learned  what  you 
intended. 

At  any  time  you  may  refer  back  to  the  details  of  the  case  study  to  refresh  your  memory  of  the  situation. 

You  may  go  back  and  forth  as  much  as  you  would  like  among  the  3 components.  And  when  you  are  completely  finished  with 
your  plan,  press  the  Finished  button  you'll  find  in  the  assessment  stage. 

Next,  if  participant  is  in  a 1-  or  2-agent  condition,  the  personal  Advisor(s)  (see  "Pedagogical  Agents"  section)  would 
introduce  themselves  and  their  role. 

The  environment  organized  the  participant’s  instructional  planning  processes  into  four  main  phases  which  will  be  described 
below,  each  indicated  through  large  icon-buttons:  case  study,  blueprints,  planning,  and  assessment.  At  any  time  it  is  possible  for 
the  participant  to  move  from  one  phase  to  the  other  although  it  is  not  possible  for  the  participant  to  return  to  the  introductory 
screens.  Once  the  participant  enters  the  assessment  stage,  an  additional  button  labeled  “Finished”  is  provided.  After  selecting 
“Finished”  the  participant  is  asked  “Are  you  ready  to  exit  the  application  and  go  to  the  exit  survey?”  Upon  selecting  “OK”  the 
participant  answers  post -questions. 

Case  study.  The  case  study  was  developed  for  MIMIC  given  that  it  is  difficult  to  find  existing  case  studies  that  are 
appropriate  (Ertmer  & Russell,  1995).  It  consisted  of  a description  of  Anna  and  her  problems  learning  supply  and  demand,  her 
teacher  Mr.  Lange,  and  her  school  in  Texas.  The  concept  of  supply  and  demand  was  chosen  as  it  is  relatively  domain- 
independent  of  specialty  areas  for  instruction  and  requires  less  specific  prior  knowledge.  Links  were  provided  so  that  the 
participant  could  access  Anna's  homework  that  contained  comments  from  Mr.  Lange,  and  his  personal  planning  notes  which 
included  text  and  graphics.  In  this  way,  participants  could  review  the  necessary  content  for  themselves  as  well  as  evaluate  Anna’s 
situation. 

Blueprints.  The  purpose  of  this  phase  was  listed  on-screen  as  follows:  “The  purpose  of  this  step  is  for  you  to  decide  what  you 
want  Anna  to  learn.  What  have  you  determined  to  be  the  learning  goals?  List  them  clearly  in  the  workspace  below.  For  reference 
you  may  want  to  see  the  stated  Texas  standards  and  benchmarks  regarding  supply/demand  for  eighth  graders,  with  links  below.” 
A text -box  field  was  provided  within  which  the  participant  could  list  the  instructional  goals  or  objectives.  Two  links  provided 
additional  information  regarding  Texas  standards  and  benchmarks  for  supply/demand. 

Plannin2.  The  purpose  of  this  phase  was  listed  on-screen  as  “To  develop  a detailed  instructional  plan  for  Anna.  “ A text -box 
field  was  provided  within  which  the  participant  could  enter  the  instructional  plan. 

Assessment.  The  purpose  of  this  phase  was  listed  on-screen  as  follows:  “The  purpose  of  this  phase  is  to  develop  ways  to 
determine  if  Anna  learned  what  you  initially  defined  in  the  blueprints  phase.  Please  describe  this  assessment  in  detail  in  the  space 
below.  “ A text -box  field  was  provided  within  which  the  participant  could  list  the  assessment. 

Pedagogical  Agents 

Depending  on  the  experimental  condition,  one  or  two  Microsoft  Agent  characters  (Peedy  the  Parrot  and  Merlin  the  Wizard) 
were  implemented  as  “Advisors”  to  the  participants.  Characters  were  randomly  assigned  to  represent  the  instructivist  and 
constructivist  agents  and  to  control  for  possible  differences.  The  Advisors  were  referred  to  by  gender-neutral  names  - Jan  and 
Chris.  Jan  was  always  the  instructivist  advisor,  representing  traditional  systematic  instructional  planning  including  the  problem- 
solving aspects  of  Instructional  Systems  Design  (ISD)  as  characterized  by  Dick  & Carey  (1996)  and  Reiser  & Dick  (1996).  Chris 
was  always  the  constructivist  advisor,  representing  a learner-centered  approach,  focusing  on  the  importance  of  the  context  of 
learning,  stressing  that  learning  involves  active  interaction,  and  emphasizing  the  process  rather  than  the  product  of  learning 
(Driscoll,  2000). 

The  purpose  of  the  agents  was  to  serve  as  mentors  (Baylor,  2000a)  and  to  operationalize  the  instructivist  and  constructivist 
approaches  to  instructional  planning.  When  one  or  two  agents  were  present  the  following  events  resulted:  1)  the  agent(s) 
provide(s)  an  initial  observation  upon  entering  each  of  the  four  MIMIC  planning  stages;  2)  the  agent(s)  provide(s)  reflection 
questions  to  encourage  self-evaluation  consisting  of  statements  “Make  sure  you  are  not  just  talking  about  how  you  would  do  it; 
actually  create  the  instruction  for  Mr.  Lange  (Anna’s  teacher).”,  “Actually  develop  the  content -related  activities”,  or  “Apply  the 
plan  specifically  to  the  topic  of  supply  and  demand”  every  five  minutes  upon  entering  a stage;  3)  the  agent(s)  would  provide  an 
example  of  their  instructional  plans  following  the  participant’s  development  of  an  instructional  plan;  and,  4)  the  agent(s)  would 
provide  additional  suggestions  when  selected  by  the  participant.  Agent  suggestions  were  specific  to  the  case  study  and  were 


developed  and  validated  by  experts  in  instructional  planning  together  with  the  consultation  of  an  economics  professor.  The 
available  suggestions  (specific  to  each  planning  phase)  would  appear  in  a pop-up  box  for  the  participant  to  select.  For  example,  in 
the  plan  phase,  one  available  suggestion  is  “What  is  my  role  in  the  learning  process?”  If  this  suggestion  were  selected  from  the 
instructivist  agent,  the  agent  would  reply  “You  need  to  be  in  charge  of  the  learning  process  for  Anna,  You  need  to  organize  the 
materials  for  Anna,  to  create  an  optimal  learning  environment,”  The  same  suggestion  if  requested  from  the  constructivist  agent 
would  be  “Anna  should  be  at  the  center  of  the  learning  process.  This  will  encourage  Anna’s  initiative,  get  Anna  to  think  and  to 
reflect,  and  make  the  information  real  for  Anna,”  The  blueprints  phase  had  two  suggestions,  the  plan  phase  had  five  suggestions, 
and  the  assessment  phase  had  one  suggestion.  See  a related  study  (Baylor,  in  press)  for  a complete  listing  and  description  of  all 
agent  suggestions. 

Measures 

Awareness.  Awareness  was  assessed  through  three  dependent  measures:  whether  the  participant  changed  perspective  in 
instructional  design,  amount  of  participant’s  reflective  thinking,  and  the  underlying  pedagogy  of  the  participant -designed 
instructional  plan.  To  assess  whether  use  of  the  system  changed  participants'  perspective  of  instructional  design^  they  were  first 
asked  “Did  using  this  program  change  your  perspective  of  instructional  design?  (yes/no)*'  which  was  coded  as  a 1 (yes)  or  0 (no). 
To  assess  participants’  self  reported  reflections,  they  were  asked,  “How  often  did  you  reflect  on  your  thinking  during  the 
process?”  on  a Likert  scale  of  1-3  with  “Not  at  all,”  “Several  times,”  and  “Frequently”  as  the  three  levels.  To  assess  the 
underlying  pedagogy  of  the  instructional  plans,  the  instructional  plans  were  scored  according  to  their  overall  pedagogical 
“flavor,”  on  a scale  from  1 to  10,  Given  that  certain  instructional  plan  features  are  representative  of  both  instructivist  and 
constructivist  pedagogies  (e,g,,  the  importance  of  considering  prior  knowledge),  the  focus  was  on  assessing  the  presence  (or 
absence)  of  constructivist  characteristics,  as  they  were  more  salient  and  differentiable,  A high  score  in  this  measure  indicates  that 
there  were  more  constructivist  aspects  to  the  plan  such  as  a student-centered  approach,  students’  involvement  with  constructing 
knowledge,  a focus  on  students'  reasoning/critical  thinking,  and/or  situated  learning,  A low  score  in  this  measure  indicates  that 
there  are  less  constructivist  and  more  characteristically  instructivist  elements  within  the  plan.  Two  of  the  researchers  met  and 
together  discussed  what  characterized  a score  of  1-10  for  the  presence  of  underlying  pedagogy  (where  l=not  at  all  constructivist 
and  10=highly  constructivist)  for  five  sample  instructional  plans.  Following  that,  each  researcher  independently  scored  15 
instructional  plans.  Inter-rater  reliability  between  the  two  researchers  was  determined  to  be  greater  than  ,9  for  the  fifteen 
instructional  plans.  One  of  the  researchers  then  scored  the  remainder  of  the  instructional  plans  using  the  same  rubric.  Both 
researchers  were  blind  as  to  the  conditions  of  the  participants  throughout  the  rating  process. 

Attitudes.  Attitudes  were  assessed  by  three  dependent  measures,  each  as  repeated-measures:  self-efficacy,  disposition,  and 
perceived  instrumentality.  To  assess  participants’  self-efficacy,  one  item  to  measure  the  students’  self-efficacy  beliefs  about 
instructional  planning  was  administered  before  entering  and  after  exiting  the  MIMIC  environment.  It  was  developed  based  on 
Bandura  and  Schunk's  (1981)  guidelines.  All  participants  were  asked  "How  sure  are  you  that  you  can  write  a lesson  plan?"  on  a 
scale  from  1 being  not-sure  to  9 being  very  sure.  The  test-retest  reliability  was  r=.62  (p<.00\).  To  assess  participants’  disposition 
toward  instructional  planning,  each  participant  was  asked  to  write  two  adjectives  to  "Describe  what  you  think  about  instructional 
planning,"  This  method  was  employed  to  obtain  the  participants'  personal  affect  regarding  instructional  planning  as  opposed  to 
the  response  set  that  could  bias  them  to  choose  more  favorable  adjectives  if  adjectives  were  presented  in  a list.  The  adjectives 
were  coded  according  to  three  levels:  as  -1  if  both  were  negative,  as  0 if  one  was  negative  and  the  other  positive,  and  as  +l  if  both 
were  positive.  The  items  were  coded  by  two  raters  independently.  Interrater  reliability  was  established  at  ,95,  There  were  only 
two  disagreements  about  two  sets  of  adjectives  which  were  resolved  through  discussion.  Two  adjective  pairs  were  discarded 
because  they  could  not  be  classified.  The  validity  of  this  measure  was  established  in  (Kitsantas  & Baylor,  2001)  through 
concurrent  validity  of  initial  disposition  with  initial  self-efficacy  scores,  given  that  research  has  shown  that  self-efficacious 
students  generally  have  positive  affect  (Bandura,  1986),  The  test-retest  reliability  was  r=,55,  ;?<,001,  To  assess  the  participants' 
perceived  instrumentality,  or  perceived  importance  of  instructional  planning,  the  participants  were  asked  to  rate  "How  important 
is  writing  a lesson  plan  to  you  as  a future  professional?"  on  a scale  of  1 to  5 where  1=  not  important,  2=fairly  important, 
3=important,  4=very  important,  and  5=extremely  important.  Test-retest  reliability  was  r=,83,  ;?<,001, 

Performance.  Within  MIMIC,  all  participants  developed  an  instructional  plan  to  teach  the  concepts  of  supply  and  demand  to 
Anna,  Each  instructional  plan  was  scored  according  to  a rubric  that  consisted  of  four  sub-areas.  The  four  sub-areas  of  the  rubric 
were  goals/objectives,  procedure,  assessment,  and  holistic,  the  first  three  being  aligned  with  the  major  components  of 
instructional  planning  (goals/objectives,  procedure,  and  assessment).  For  the  goals/objectives  sub-score,  the  plans  were  rated 
according  to  how  clearly  the  goals/objectives  were  stated  and  how  specifically  the  purpose  of  instruction  was  described.  For  the 
procedure  sub-score,  the  plans  were  rated  according  to  the  meaningfulness  and  effectiveness  of  the  instructional  activities, 
whether  they  were  in  a logical  sequence,  and  whether  they  addressed  the  goals  stated  in  the  blueprints  phase.  For  the  assessment 
sub-score,  the  plans  were  rated  according  to  whether  the  assessment  matched  the  goals/objectives,  and  whether  it  was  logical.  For 
the  holistic  sub-score,  the  plans  were  rated  according  to  whether  the  plan  was  overall  reasonable  and  effective.  The  overall 
performance  score  was  the  compilation  of  these  four  sub-scores  (each  rated  from  1 to  5),  with  a potential  range  of  4-20,  Two  of 
the  researchers  met  and  together  discussed  what  characterized  a score  of  1 through  5 (where  l=poor  and  5=excellent)  for  each  of 
the  four  sub-areas  for  five  sample  instructional  plans.  Following  that,  each  researcher  independently  scored  15  instructional 
plans.  Inter-rater  reliability  between  the  two  researchers  was  determined  to  be  greater  than  ,9  for  the  fifteen  instructional  plans. 
One  of  the  researchers  then  scored  the  remainder  of  the  instructional  plans  using  the  same  rubric.  Both  researchers  were  blind  as 
to  the  conditions  of  the  participants  throughout  the  rating  process. 
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A2ent  value.  Participants  in  conditions  where  agents  were  present  were  asked  to  rate  the  value  of  the  agents  in  several  areas. 
Specifically,  they  were  asked  via  Likert -scale  formatted  questions,  “Did  you  enjoy  working  with  <agent>?”  (Not  at  all  / A little  / 
Very  much/  Extremely);  “Did  you  pay  attention  when  <agent>  made  suggestions?”  (Not  at  all  / Not  usually  / Usually  / Always); 
“Overall,  was  <agent>  annoying  or  useful?”  (Extremely  annoying  / annoying  / useful  / very  useful);  and,  “Did  you  generally 
agree  with  <agent>’s  suggestions?  (yes/no)” 

Procedure 

All  participants  logged  in  to  the  MIMIC  computer  environment  and  answered  computer-based  questions  regarding  gender, 
age,  and  class  section  number.  Next,  the  participants'  perceived  instrumentality,  disposition  regarding  instructional  planning, 
prior  experience  with  instructional  planning,  and  self-efficacy  beliefs  toward  instructional  planning  were  assessed.  Following 
these  initial  measures,  the  participant  altered  the  introduction  to  the  MIMIC  environment  (see  the  MIMIC  section).  Following 
this  introduction,  and  immediately  before  entering  the  environment,  participants’  self-efficacy  regarding  the  project  was 
ascertained.  Next,  the  participants  worked  through  the  case  study,  blueprints  stage,  planning  stage,  and  assessment  stage, 
developing  an  instructional  plan.  Depending  on  the  condition  (see  “Pedagogical  Agents”  section),  0-2  agents  were  present  within 
the  environment,  serving  to  represent  instructional  planning  approaches  (objectivism  and/or  constructivism).  All  participants 
worked  independently  within  the  environment  at  their  own  pace.  Following  completion  of  the  instructional  plan  within  the 
environment,  all  participants  answered  computer-based  questions  regarding  amount  of  self-reflection,  value  of  agent(s), 
perspective  of  instructional  planning,  perceived  instrumentality,  disposition,  and  self-efficacy.  The  entire  procedure  took 
approximately  90  minutes. 

Design  and  Data  Analysis 

A three-factor  MANOVA  finstructivist  agent:  present,  absent;  constructivist  agent:  present,  absent;  agent  character:  Peedy, 
Merlin)  was  the  initial  method  used  for  data  analysis  where  agent  character  (Peedy  the  Parrot  or  Merlin  the  Wizard)  was  assigned 
as  a within-subjects  factor  to  test  for  possible  differences  in  agent  character.  After  it  was  determined  that  agent  character  did  not 
play  a factor,  that  factor  was  removed  from  further  analysis  leaving  a two-factor  MANOVA  as  the  main  method  for  data  analysis. 
The  data  was  analyzed  according  to  two  groups  of  dependent  measures:  awareness  (comprised  of  change  in  perspective, 
reflection,  and  underlying  pedagogy  of  instructional  plan),  and  attitude  (comprised  of  self-efficacy,  disposition,  and  perceived 
instrumentality),  each  of  which  was  assessed  via  a two-factor  MANOVA.  The  analysis  of  attitude  (comprised  of  self-efficacy, 
disposition,  and  perceived  instrumentality)  was  treated  as  a repeated  measures  MANOVA.  For  some  analyses  that  focused  on 
agent  combinations,  a one-way  MANOVA/ANOVA  was  also  performed  with  condition  (no  agents,  instruct! vist  only, 
constructivist  only,  both  agents)  as  the  factor.  To  analyze  participants’  value  of  the  agents,  independent-group  t-tests  were  used 
for  the  questions  comparing  one  agent  on  a particular  attribute,  and  paired-group  t-tests  or  chi-square  analysis  were  used  for  the 
questions  regarding  participants  who  received  both  agents. 

Results 

General 

There  were  no  statistically  significant  differences  among  agent  conditions  regarding  the  average  number  of  suggestions 
requested  from  the  agents.  The  average  number  of  suggestions  selected  was  10.34. 

Awareness 

The  three  dependent  measures  for  awareness  include  whether  the  participant  changed  in  perspective  regarding  instructional 
planning,  amount  of  self-reflection,  and  underlying  pedagogy  of  participant-designed  instructional  plan  and  were  assessed 
through  a two-way  MANOVA  with  instructivist  (present,  absent)  and  constructivist  (present,  absent)  as  the  between-subject 
factors. 

Perspective  of  instructional  planning.  Results  from  the  two-way  MANOVA  revealed  a main  effect  for  the  constructivist 
agent  on  change  in  perspective  in  instructional  planning,  F(l,  131)  = 9.82,p<.01,  where  M=. 71  (present)  versus  A/=.45  (absent), 
indicating  that  when  the  constructivist  agent  was  present,  participants  were  more  likely  to  report  that  MIMIC  changed  their 
perspective  of  instructional  planning.  There  were  no  other  significant  main  effects  or  interactions.  To  determine  the  relative 
differences  of  change  in  perspective  amongthe  four  agent  conditions,  a one-way  ANOVA  with  condition  (no  agents,  instructivist 
only,  constructivist  only,  both  agents)  as  the  factor  was  conducted  and  revealed  a significant  main  effect  for  agent  condition,  F(3, 
131)=3.74,  p=.01.  Post-hoc  Tukey’s  tests  indicated  that  the  constructivist-only  condition  elicited  the  most  change  in  perspective 
(M=.80)  and  was  significantly  greater  than  both  the  instruct! vist-only  condition  (A/=.47)  and  the  no-agent  conditions  (A/=.44) , 
but  was  not  significantly  greater  than  the  both-agents  condition,  which  ranked  second  in  overall  change  in  perspective  (A/=.63). 

Self-reported  reflection.  The  two-factor  MANOVA  revealed  a main  effect  for  the  constructivist  agent,  indicating  that  when 
the  constructivist  agent  was  present,  participants  reported  reflecting  less  (W=2.24),  than  when  it  was  absent  (V/=2.43),  F(l, 
131)=4.73,/?<.05.  There  were  no  significant  differences  between  high  and  low  performers  on  self-reported  reflection. 

Underlying  pedagogy  of  instructional  plan.  The  two-factor  MANOVA  indicated  a main  effect  for  the  constructivist  agent, 
F(l,131)  =11.28,  p=.00l,  where  the  presence  of  the  constructivist  agent  was  related  to  participants  developing  more 
constructivist-oriented  instructional  plans  (Vf=6.12)  than  when  it  was  absait  (^^.47).  While  there  was  not  a statistically 
significant  main  effect  for  the  instructivist  agent,  there  were  numerical  differences  showing  that  its  presence  was  associated  with 
lower  scores  ^=4.92)  than  its  absence  ^=5.72),  indicating  its  positive  relation  to  an  instructivist  underlying  pedagogy.  An 
independent  t-test  also  showed  that  high  achievers  developed  plans  that  were  significantly  more  constructivist  (Vf=6.08)  in 
approach  than  low  achievers  (A/=4.29).  The  overall  mean  for  the  underlying  level  of  pedagogy  for  all  participants  was  A^5.35, 
5D=3.01. 
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Attitude  Attitude  was  analyzed  through  a repeated-measures  two- factor  MANOVA,  with  disposition,  self-efficacy,  and 
perceived  instrumentality  as  the  dependent  measures,  each  assessed  both  before  and  after  working  in  MIMIC,  and  with  the 
instructivist  agent  and  the  constructivist  agent  as  the  two  between-subject  factors. 

Disposition  regarding  instructional  planning.  The  two- factor  repeated-measures  MANOVA  indicated  a main  effect  for  the 
presence  of  instructivist  agent  F(l,  128)-3.46,  /7<.05,  revealing  that  when  the  instructivist  agent  was  present,  participants  had 
significantly  lower  dispositions  regarding  instructional  planning  (A/=.54)  than  when  the  instructivist  agent  was  absent  (M-.15). 
Overall,  participants’  disposition  toward  instructional  planning  was  generally  positive  (where -1  represents  negative,  0 represents 
neutral,  and  1 represents  positive),  both  before  (A/-.63)  and  after  (A/=.58)  working  within  MIMIC. 

Self-efficacy.  The  two-factor  repeated-measures  MANOVA  revealed  a main  effect  for  time  of  assessment,  showing  that 
participants  overall  increased  in  self-efficacy  (A/=5.46  vs.  A/— 6.01)  as  a result  of  MIMIC,  F(  1,1 28)=  12. 54,  /?=.001.  There  were 
no  significant  main  effects  or  interactions  for  the  presence  or  absence  of  the  constructivist  and  instructivist  agents.  As  would  be 
expected,  self-efficacy  was  highly  related  to  participants'  prior  experience  with  instructional  planning.  Those  participants  with 
high  experience  writing  lesson  plans  (3,  4,  5)  as  compared  to  those  with  low  experience  (1,2;  where  l=no  experience  and  5=very 
much  experience),  had  significantly  higher  self-efficacy  both  before  (Vf=6.40  vs.  A/=4.91),  /(1 33  ) = 4.06,  /?<.001,  and  after 
(M=6.62  vs  A/=5.63),  /( 1 33)  =3. 11 , p^.OO  1 , using  MIMIC. 

Perceived  instrumentality.  The  two- factor  repeated-measures  MANOVA  did  not  show  any  significant  main  effects  or 
interactions.  Participants’  perceived  instrumentality  was  M=4.\\  for  the  pre-test,  and  A^4.18  for  the  post-test,  indicating  that 
they  believed  instructional  planning  to  be  slightly  more  important  than  “very  important’’  both  before  and  after  the  use  of  MIMIC. 

Performance  Performance  was  analyzed  through  a two-factor  MANOVA,  with  each  of  the  four  sub-scores  and  the  total 
score  as  the  dependent  measures,  with  instructivist  (present,  absent)  and  constructivist  (present,  absent)  as  the  two  between- 
subject  factors.  Results  revealed  no  main  effects  or  significant  interactions  for  the  total  performance  score  or  each  of  the  four  sub- 
scores. The  total  performance  score  ranged  from  4-20,  with  the  overall  average  of  A/=13.71,  5D=4.10.  Descriptive  statistics  for 
the  sub-scores  were  as  follows:  goals/objectives  -A/=3.38,  5D=1.32;  procedure-  M-  3.30,  SD  1.10;  assessment  — M=3.26, 
SD=\.29;  and,  holistic  - A/=3.23,  5D=1.14.  Based  on  the  total  performance  score,  participants  were  categorized  as  high 
performers  if  they  scored  in  the  top  quartile  (total  score  of  M=\6  and  above),  and  low  performers  if  they  scored  in  the  bottom 
quartile  (total  score  of  A/=10  and  below). 

Agent  value  Agent  value  was  assessed  in  several  areas,  as  listed  below. 

Enjoyment  in  working  with  agent.  Participants  tended  to  enjoy  working  with  both  agents  (M=  2.63  for  instructivist  agent  and 
M=2.64  for  constructivist  agent),  where  2=’’A  little’’  and  3=’’Very  much.’’  A non-significant  t-test  of  the  two  groups  (those  with 
constructivist  present  versus  those  with  instructivist  present)  revealed  that  participants  reported  finding  both  agents  to  be  equally 
enjoyable.  Attending  to  agent.  Participants  tended  to  pay  attention  to  both  agents  (A/=  3.05  for  instructivist  agent  and  A/=3.21  for 
constructivist  agent)  where  3=  “usually”  and  4=”always”.  A non-significant  t-test  of  the  two  groups  revealed  that  participants 
reported  paying  relatively  equal  attention  to  both  agents.  Usefulness  of  agent.  Participants  tended  to  report  that  both  agents  were 
useful  ^=3.02  for  instructivist  agent  and  A/=3.08  for  constructivist  agent)  where  3=”useful”  and  4=”very  useful”.  A non- 
significant t-test  of  the  two  groups  revealed  that  participants  reported  finding  both  agents  equally  useful.  Agreement  with  agent. 
In  terms  of  whether  participants  agreed  with  the  agents’  advisements,  answers  were  coded  as  yes=l  and  no=0,  and  results  show 
that  participants  tended  to  agree  with  both  of  the  agents’  suggestions,  M=.61  for  instructivist  and  A/=.84  for  constructivist.  A non- 
significant t-test  of  the  two  groups  revealed  that  participants  reported  not  agreeing  significantly  more  or  less  with  either  agent. 


Discussion 

Overall,  the  results  indicate  that  the  presence  of  the  constructivist  pedagogical  agent  affected  pre-service  teachers’ 
metacognitive  awareness  of  instructional  planning  in  multiple  ways:  through  a change  in  perspective,  less  reported  reflection,  and 
through  the  underlying  pedago©^  of  their  instructional  plans. 

It  is  speculated  that  increased  awareness  about  instructional  planning  would  lead  to  a richer  and  more  comprehensive 
understanding  of  the  planning  process,  leading  pre-service  teachers  to  develop  an  appreciation  for  the  process.  As  stated  by 
Marton  and  Booth  (1997),“of  prime  interest  is  the  variation  in  the  ways  in  which  people  are  capable  of  experiencing  various 
situations  or  phenomena.  If  one  becomes  aware  that  something  is  in  a certain  way,  they  also  become  aware  that  it  could  be  in 
some  other  way”  (p.207).  Eventually,  it  would  be  expected  that  this  change  in  perspective  and  understanding  of  the  depth  and 
complexity  of  instructional  planning  could  lead  to  better  performance  and/or  increased  intrinsic  motivation  related  to  the  task. 

The  presence  of  the  constructivist  agent  tended  to  change  participants’  perspective  toward  instructional  planning.  Although 
participants  had  been  introduced  to  the  constructivist  approach  as  part  of  the  course  in  which  they  were  enrolled,  it  still  may  have 
been  experienced  as  a novel  and  unique  approach.  Given  that  some  pre-service  teachers  describe  instructional  planning 
negatively,  using  adjectives  such  as  “boring,”  or  “tedious”  (Kitsantas  & Baylor,  2001),  the  constructivist  approach  may  have 
been  perceived  as  offering  something  new  and  providing  more  options  for  instructional  planning.  Further,  the  presence  of  the 
constructivist  pedagogical  agent  could  have  been  perceived  to  highlight  more  appealing  elements  of  instructional  planning  (such 
as  a student -centered  focus  or  responsibility  of  the  learner). 

While  the  two-agent  condition  (given  that  it  had  the  constructivist  agent  present)  was  more  transforming  than  the  instructivist 
only  or  no  agent  conditions,  it  was  not  found  to  lead  to  a significantly  greater  change  in  perspective  than  the  other  three  agent 
conditions,  as  was  predicted.  The  fact  that  the  presence  of  two  agents  simultaneously  was  not  perceived  as  the  most  transforming 
in  terms  of  a change  in  perspective  could  be  an  issue  of  cognitive  load.  As  Sweller  and  colleagues  suggest  (Sweller,  van 
Merrienboer,  & Paas,  1998)  “less  is  best”  in  learning  situations,  indicating  that  in  this  case  the  learners  may  be  too  focused  during 
problem  solving  to  process  suggestions  from  multiple  agents. 
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The  finding  that  the  presence  of  the  constructivist  agent  led  to  reflection  seems  at  first  incompatible  with  the  finding  that 
the  constructivist  agent  led  to  a greater  change  in  perspective.  However,  when  the  constructivist  agent  was  present,  perhaps 
participants  were  focusing  their  attention  on  its  ideas/suggestions  rather  than  reflecting  on  their  own  cognitive  processes.  In  other 
words,  it  seems  viable  that  the  presence  of  constructivist  agent  facilitated  pre-service  teachers  to  think  more  (i.e.,  change 
perspective),  but  not  necessarily  to  reflect  more.  While  there  is  strong  evidence  that  reflection  during  instructional  activities  is 
important  (Chi  & VanLehn,  1991;  VanLehn,  Jones,  & Chi,  1992),  there  is  less  information  regarding  the  relative  value  of 
reflection  as  compared  to  awareness.  Future  research  should  include  an  open-ended  follow-up  question  to  determine  what  pre- 
service  teachers  actually  meant  bv  reporting  less  reflection. 

The  presence  of  the  constructivist  agent  was  also  associated  with  participants’  developing  more  constructivist-oriented 
instructional  plans,  reflecting  a “trickle  down”  effect  of  the  agent’s  pedagogical  beliefs  to  the  participants.  Although  there  was  not 
a main  effect  where  the  presence  of  the  instructivist  agent  was  related  to  lower  underlying  pedagogy  score  (thus  indicating  a more 
instructivist-oriented  underlying  pedagogy  in  the  instructional  plans),  there  were  numerical  differences  showing  that  the  presence 
of  the  instructivist  was  associated  with  lower  scores  in  this  area,  indicating  that  in  both  cases  (presence  of  instructivist  and 
presence  of  constructivist)  the  pre-service  teachers  internalized  the  agent’s  suggestions  and  translated  them  in  their  instructional 
plans. 

Overall,  the  agents  were  perceived  by  the  pre-service  teachers  to  be  valuable  as  mentors.  Participants  reported  neither  agent  to 
be  “better”  or  “worse”  in  any  of  the  following  aspects:  enjoyment  of  working  with  agent;  paying  attention  to  agent;  perceived 
usefulness  of  agent;  and,  credibility  of  agent.  Further,  the  pre-service  teachers  who  received  both  agents  were  equally  split  as  to 
which  agent  made  them  think  the  most,  which  thought  the  most  like  them,  and  which  of  the  two  agents  they  would  choose  to 
assist  them  (if  could  choose  only  one).  Agents  were  rated  as  equally  useful,  they  also  were  paid  equal  attention,  and  were  equally 
enjoyable  with  which  to  work. 

While  there  were  no  explicit  differences  between  the  agents  in  terms  of  value,  there  were  differences  in  the  effect  of  the  agents  on 
attitude.  Contrary  to  what  was  hypothesized,  it  was  found  that  the  presence  of  the  instructivist  agent  led  participants  to  report 
significantly  lower  dispositions  regarding  instructional  planning.  Given  that  the  instructivist  agent  represents  a systematic 
approach,  perhaps  students  felt  it  was  too  prescriptive,  and  made  the  instruction  and/or  planning  process  seem  tedious  or  boring. 
While  there  were  no  main  effects  on  self-efficacy  for  the  presence  of  the  agents,  self-efficacy  increased  for  participants  using  the 
MIMIC  system,  suggesting  a practice  effect.  A reason  that  may  explain  why  there  were  no  effects  of  the  agents  on  perceived 
importance  of  instructional  planning  could  be  that  participants  started  with  already-high  ratings  (rating  it  on  average  as  slightly 
more  important  than  “very  important”). 

There  were  no  main  effects  of  the  agents*  presence  for  the  overall  performance  score  which  was  expected  given  that  related 
research  has  not  provided  evidence  that  animated  agents  improve  learning  (Dehn  & van  Mulken,  2000).  Further,  the  MIMIC 
agents  provided  suggestions  regarding  the  underlying  pedagogic  rationale  for  different  aspects  of  the  planning  process,  not 
solutions.  While  these  advisements  were  content-specific,  they  did  not  specifically  prescribe  or  show  the  students  exact 
implementation. 

In  terms  of  the  overall  implications,  there  is  preliminary  evidence  to  suggest  that  the  exposure  to  constructivism  as  an 
instructional  planning  process  adds  richness,  diversity,  meaning  and  interest.  While  the  instructivist  approach  adds  substance  and 
structure  to  the  process,  it  may  negatively  affect  disposition.  If  only  one  perspective  could  be  provided  to  enrich  their  awareness 
of  instructional  planning,  exposure  to  the  constructivist  approaches  may  be  most  beneficial,  especially  if  the  pre-service  teachers 
already  have  a strong  foundation  in  instructivist  approaches  to  instructional  planning. 

Future  research  could  implement  the  study  with  more  advanced  pre-service  teachers  or  instructional  designers,  to  determine 
how  the  agents  impact  them  in  terms  of  awareness  of  and  attitude  towards  instructional  planning.  Another  way  to  explore  a 
change  in  awareness  could  be  through  including  epistemology  profiles  to  determine  if  pre-service  teachers  epistemic  beliefs 
change  as  a result  of  using  the  system.  The  role  of  reflection  needs  to  be  further  investigated  through  more  open-ended  questions 
and  to  systematically  evaluate  the  agents  for  their  self-regulatory  features  to  determine  what  promotes  monitoring  and  evaluation 
and  how  they  relate  to  what  the  participant  terms  “reflection.”  Cognitive  load  as  an  explanation  for  the  impact  of  two  agents 
needs  to  be  further  examined  with  more  advanced  students,  who  may  be  able  to  better  manage  receiving  advisements  from 
multiple  agents. 

Overall,  this  study  validated  the  effectiveness  of  an  agent-based  approach  as  a research  process  to  investigate  teaching  and 
learning  by  simulating  human-like  mentoring  via  pedagogical  agents  (Baylor,  in  press).  The  instructivist  and  constructivist 
agents  within  MIMIC  provided  an  indirect  and  meaningful  way  to  investigate  students*  affect  and  beliefs  toward  instructional 
planning. 
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Abstract 

The  Instructional  Planning  Self- Reflective  Tool  (IPSRT)  and  the  Constructivist  Planning  Self-Reflective  Tool  (CPSRT)  were 
developed  to  facilitate  self-regulation  during  instructional  planning  for  pre-service  teachers.  We  developed  the  IPSRT  tool  to 
promote  the  adaptation  of  traditional  instructional  planning  methods  and  the  CPSRT  tool  to  promote  the  use  of  effective 
constructivist  planning  principles  for  pre-service  teachers.  This  paper  will  present  the  two  tools  and  provide  suggestions  based 
on  empirical  research  for  instructors  of  teacher  preparation  programs  on  how  to  assist  pre-service  and  in-service  teachers  to 
become  self-regulated  instructional  planners. 


Introduction 

Instructional  planning  is  a key  element  of  the  teaching  process.  Teacher  preparation  programs  spent  a significant  number  of 
hours  instructing  pre-service  teachers  how  to  write  effective  instructional  plans.  Two  major  theoretical  perspectives,  the 
instructivist  and  the  constructivist  approaches  to  instructional  planning,  are  implemented  by  colleges  of  education  to  prepare 
future  teachers.  However,  writing  an  instructional  plan  for  a specific  student  population  requires  more  than  knowing  the  essential 
elements  that  constitute  an  effective  instructional  plan.  It  requires  the  teacher  to  use  self-regulatory  processes,  especially  self- 
monitoring  and  self-evaluation  that  will  enable  him/her  through  reflection  to  tailor  the  instructional  plan  to  the  needs  of  the 
particular  group  of  students. 


Description  of  the  Two  Tools 

IPSRT  (Instructional  Planning  Self- Reflective  Tool) 

We  developed  the  Instructional  Planning  Self-Reflective  Tool  (IPSRT)  based  on  both  research  on  self-regulated  learning 
(e.g.,  Zimmerman,  2000)  and  the  Reiser  & Dick  (1996)  instructional  planning  model.  It  was  designed  to  facilitate  teachers’  self- 
monitoring and  self-evaluation  during  instructional  planning.  See  Baylor,  Kitsantas  & Chung  (2001)  for  the  complete  tool. 

The  major  headings  for  the  IPSRT  were  determined  according  to  the  Reiser  & Dick  (1996)  instructional  model;  instructional 
goal,  objectives,  materials/preparation,  learner  characteristics,  procedure,  and  assessment.  Under  the  heading  for  objectives,  there 
are  specific  subheadings  for  the  four  basic  components  of  an  instructional  objective:  audience,  behavior,  condition  and  degree. 
Under  the  heading  for  procedure,  listed  are  the  six  subheadings  of  instructional  elements:  motivating  students,  informing  students 
of  objectives,  helping  students  recall  prerequisites  and  presenting  information  and  examples,  providing  practice  and  feedback  and 
summarizing  the  lesson.  These  are  considered  the  necessary  steps  for  developing  instructional  activities.  An  additional  section 
was  added  for  users  to  self-evaluate  the  quality  of  the  overall  instructional  plan.  Under  each  subheading,  the  IPSRT  consists  of 
questions  with  prompts  to  remind  the  users  of  what  should  be  included  in  a traditional  instructional  plan. 

We  initially  tested  the  IPSRT  with  175  pre-service  teachers,  who  were  enrolled  in  seven  sections  of  an  “Introduction  to 
Educational  Technology”  course  and  used  the  IPSRT  tool  to  construct  several  instructional  plans.  These  pre-service  teachers 
were  asked  what  was  helpful  about  using  the  tool  for  instructional  planning.  Eighty-percent  of  participants  reported  that  the 
IPSRT  was  useful  for  self-monitoring,  75%  stated  it  was  useful  for  self-evaluation,  and  25%  mentioned  that  it  was  beneficial  for 
organization  (Baylor,  Kitsantas  & Chung,  2001).  Given  that  the  IPSRT 

was  developed  specifically  for  self-monitoring  and  evaluation,  the  results  supported  its  value  as  a cognitive  tool  in  these  two 
areas. 

The  Constructivist  Planning  Self- Reflective  Tool  (CPSRT) 

We  designed  the  Constructivist  Planning  Self-Reflective  Tool  (CPSRT)  on  the  basis  of  research  on  self-regulated  learning 
(Zimmerman,  2000)  in  association  with  the  constructivist  approach  to  instructional  planning  (Jonnasen,  1999).  It  was  developed 
as  a “Learning  Support  Plan”  to  support  pre-service  teachers  in  defining  the  key  characteristics  of  an  effective  constructivist 
learning  environment.  See  Kitsantas,  Baylor  & Hu  (in  press)  for  the  complete  tool. 

It  is  organized  according  to  three  phases.  The  first  phase  “Before”  (for  activities  prior  to  the  implementation  of  instruction), 
includes  the  instructional  purpose,  and  defines  learning  activities  with  both  required  and  desirable  characteristics.  The  required 
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characteristics  of  learning  activities  refer  to  constructivist  features  such  as  the  importance  of  cognitive  activity  and  the  focus  on 
the  learning  process  as  opposed  to  acquiring  specific  knowledge.  The  desirable  characteristics  contain  the  attributes  such  as  ill- 
structured  tasks,  definition  by  learner  and  the  social  environment  for  the  activities.  The  second  phase  “During”  (for  activities 
during  the  implementation  of  instruction),  describes  the  role  of  the  student  and  the  instructor,  e.g,,  students  should  be  active  in  the 
learning  process,  students  should  take  the  major  responsibility  for  their  learning,  and  teachers  act  as  facilitators  of  the  learning 
process.  The  third  phase  “After”  (for  activities  following  the  instruction),  comprises  the  assessment  of  students’  learning,  which 
also  emphasizes  the  activity-based  nature  of  the  constructivist  approach  by  examining  if  the  assessment  measures  the 
instructional  goals  and  involves  some  sort  of  performance  by  the  learner.  This  tool  does  not  specify  what  needs  to  be  included  in 
the  instructional  plan,  but  rather  provides  suggestions  for  consideration  within  the  constructivist  framework. 

We  tested  the  CPSRT  tool  with  approximately  150  pre-service  teachers,  who  were  registered  in  eight  sections  of  an 
“Introduction  to  Educational  Technology”  course.  The  results  of  the  evaluation  show  that  this  tool  is  constructive  for  self- 
evaluation,  organization,  monitoring,  and  cognitive  flexibility.  Thirty  eight  percent  of  participants  reported  that  the  CPSRT  was 
useful  for  monitoring,  38%  stated  it  was  convenient  for  self-evaluation,  and  33%  mentioned  that  it  was  effective  for  organization, 
and  31%  commented  that  it  is  beneficial  for  cognitive  flexibility  (Kitsantas,  Baylor  & Hu,  in  press).  As  compared  to  the  IPSRT, 
the  CPSRT  is  valuable  for  promoting  cognitive  flexibility  (Baylor  & Kitsantas,  2001),  which  is  a critical  component  for  the 
constructivist  planning  approach. 

Recommendations  for  use 

Based  on  empirical  research,  the  following  recommendations  are  presented  for  instructors  using  the  two  tools  for  instructivist 
and/or  constructivist  approaches  to  instructional  planning. 

IPSRT  is  useful  for  pre- service  teachers  who  have  limited  experience  in  teaching  because  it  provides  them  with  systematic 
procedures  to  follow. 

Pre-service  teachers  may  not  be  very  familiar  with  the  instructional  elements  that  are  necessary  for  an  effective  instructional 
plan.  The  IPSRT  is  good  for  traditional  instructional  planning  because  it  is  well  structured,  and  the  organization  provided  by  the 
IPSRT  tool  is  consistent  with  the  step  -by-step  nature  of  traditional  classroom  instruction.  The  IPSRT  can  serve  as  a checklist  for 
inexperienced  pre-service  teachers  to  make  sure  that  they  have  included  all  the  necessary  instructional  elements  of  a instructional 
plan. 

The  IPSRT  includes  a number  (f  questions  with  check  boxes  for  the  pre-service  teacher  to  review  whether  each  area  is 
covered  in  his/her  current  instructional  plan.  For  example,  when  writing  an  instructional  plan,  pre-service  teachers  do  not  always 
include  the  four  major  components  of  instructional  objectives.  The  questions  in  the  objective  part  of  the  IPSRT  tool  not  only 
remind  the  users  of  these  critical  sections  of  an  instructional  objective  but  also  give  pre-service  teachers  a recap  on  the  important 
characteristics  of  these  elements.  See  the  following  excerpt: 

OBJECTIVE(s): 

• Are  all  four  of  the  following  components  present  for  each  objective? 

□ yes  Gno 

1 . Audience 

• Does  this  component  state  who  will  be  doing  the  performance?  □ yes  Qno 

2.  Behavior 

• Is  the  behavior  measurable  and  observable?  □ yes  Qno 

3.  Condition 

• Is  the  context  for  the  behavior  specified?  □ yes  Qno 

4.  Degree 

• Does  this  component  clarify  how  well/to  what  extent  the  performance  must  be  done? 

□ yes  Gno 

The  IPSRT  tool  is  especially  helpful  when  pre-service  teachers  are  concerned  or  having  difficulty  with  writing  instructional 
plans.  Instructors  of  pre-service  teachers  can  provide  the  IPSRT  tool  as  a job  aid  for  their  in-class  or  take-home  instructional 
planning  practice.  With  the  tool  in  their  hands,  the  instructor  can  review  the  instructional  elements  with  the  pre-service  teachers 
and  then  explain  how  they  can  use  the  tool  to  monitor  and  reflect  on  their  own  instructional  plan  composition.  Once  they 
understand  how  to  use  the  tool,  they  should  be  able  to  focus  on  the  content  of  the  instructional  plan  instead  of  unnecessarily 
worrying  about  missing  any  instructional  elements  in  the  structure.  Further,  it  has  been  shown  that  the  use  of  the  IPSRT  is  best 
introduced  and  used  prior  to  introduction  of  the  CPSRT  to  serve  as  a strong  foundation  in  systematic  methods  of  instruction 
(Baylor  & Kitsantas,  2001). 

The  IPSRT  enhances  pre-service  teachers ' intrinsic  motivation  for  instructional  planning 

Pre-service  teachers  often  have  limited  knowledge  and  skills  writing  instructional  plans,  and  are  much  less  confident  than 
experienced  teachers.  We  found  in  an  experimental  study  with  the  IPSRT  (Kitsantas  & Baylor,  in  press)  that  pre-service  teachers 
tend  to  have  negative  attitudes  initially  about  instructional  planning.  During  the  experiment  with  175  pre-service  teachers, 
partially  designed  to  assess  the  pre-service  teachers’  dispositions  towards  instructional  planning,  participants  were  asked  to  list 
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two  adjectives  to  describe  their  thoughts  regarding  instructional  planning.  We  found  that  participants  who  did  not  use  the  IPSRT 
were  slightly  negative  towards  instructional  planning  whereas  participants  who  used  the  IPSRT  were  positive.  Some  of  the 
positive  adjectives  used  by  the  participants  were  “organized/’  “helpful,”  “important,”  while  some  of  the  negative  adjectives  were 
“time-consuming,”  “tedious,”  “boring,”  This  finding  is  a powerful  empirical  evidence  of  the  positive  effect  of  using  the  IPSRT 
tool  to  improve  pre-service  teachers’  disposition  towards  instructional  planning. 

Pre-service  teachers’  self-efficacy,  defined  as  the  degree  to  which  they  feel  competent  in  writing  an  instructional  plan,  is  also 
affected  by  the  use  of  the  IPSRT.  Pre-service  teachers  that  initially  reported  high  self-efficacy  beliefs  to  write  an  instructional 
plan,  following  use  of  the  IPSRT,  realized  that  instructional  planning  is  a far  more  complex  activity  than  they  originally  thought. 
On  the  other  hand,  pre-service  teachers  initially  reporting  low  self-efficacy  beliefs  to  write  an  instructional  plan,  felt  more 
confident  after  using  the  IPSRT  (Kitsantas  & Baylor,  in  press).  These  findings  suggest  that  this  tool  serves  as  a reflective 
mechanism  for  high  self-efficacious  learners  who  believe  that  they  have  the  skills  to  write  effective  instructional  plans,  and 
enhances  the  motivation  of  those  who  believe  that  they  lack  the  skills  to  succeed  in  this  task. 

With  the  IPSRT  tool  serving  as  a job  aid  or  cognitive  tool,  pre-service  teachers  may  expend  more  mental  effort  to  writing  the 
content  of  the  instructional  plan,  allowing  for  the  development  of  more  effective  and  creative  instructional  plans.  Furthermore, 
the  improvement  in  their  performance  in  instructional  planning  as  well  as  the  self-evaluation  conducted  by  pre-service  teachers 
using  the  IPSRT  will  positively  reinforce  the  pre-service  teachers,  and  may  make  them  feel  more  confident  and  motivated  in 
carrying  out  this  task  in  the  future. 

The  IPSRT  promotes  a greater  metacognitive  awareness  of  the  complexity  of  instructional  planning 

The  IPSRT  is  also  beneficial  to  pre-service  teachers  because  it  functions  as  a guide  for  users  to  monitor  and  evaluate  their 
performance,  thereby  improving  metacognitive  awareness.  The  prompt  questions  in  each  subheading  help  the  pre-service 
teachers  examine  the  adequacy,  clarity  and  accuracy  of  their  instructional  plan  content. 

From  a micro-perspective,  pre-service  teachers  may  have  difficulties  applying  the  necessary  skills  within  each  instructional 
planning  phase  (e.g.,  writing  objectives,  formulating  test  items).  From  a macro-perspective,  pre-service  teachers  do  not  always 
understand  how  the  phases  are  interrelated  and  interconnected  in  the  process  of  instructional  development.  Frequently  pre-service 
teachers  see  the  process  as  incremental  (e.g.,  the  individual  phases  or  tasks)  and  feil  to  see  the  global  aspect  of  the  overall 
planning  model.  For  example,  when  evaluating  their  objectives  with  the  IPSRT,  it  guides  them  on  the  micro  level  in  writing 
learning  objectives  using  the  four  components  such  as  audience,  behavior,  condition  and  degree,  while  at  the  same  time  relating 
this  particular  micro-level  part  to  the  macro-level  structure  of  instructional  goal  section  by  asking  “Does  each  objective  derive 
directly  and  logically  from  one  of  the  instructional  goals?”  Pre-service  teachers  improve  their  critical  thinking  skills  by 
alternating  between  the  two  levels.  Therefore,  the  IPSRT  is  designed  to  facilitate  the  use  of  self-regulation  strategies  from  both 
the  macro  and  micro  perspective,  listing  specific  strategies  within  each  component  of  the  instructional  plan  and  strategies  referred 
to  the  overall  connectivity  and  holistic  value  of  the  instructiohal  plan  respectively. 

The  CPSRT  provides  exposure  to  constructivist  methods  of  instruction  for  pre-service  teachers. 

Constructivism  represents  a very  different  approach  to  instructional  planning.  Most  of  the  pre-service  teachers  are  novice  to 
the  constructivist  principles  about  instruction.  The  structure  of  the  CPSRT  tool  and  the  prompt  questions  in  each  phase  are 
representative  of  typical  constructivist  instruction  characteristics.  The  CPSRT  can  thus  figuratively  serve  as  a “menu  of  ideas”  for 
inexperienced  pre-service  teachers  to  select  appropriate  instructional  elements  of  a constructivist  instructional  plan.  Not  only  does 
the  CPSRT  provide  exposure  to  constructivist  methods,  but  also  it  was  found  to  facilitate  self-efficacy  in  the  process  (Baylor  & 
Kitsantas,  2001). 

The  following  excerpt  illustrates  that  in  a constructivist  approach  learners  need  to  be  cognitively  active  and  take 
responsibility  for  the  learning  process  whereas  the  instructor  should  try  to  facilitate  the  learning  process  instead  of  directly 
imparting  knowledge. 

Role  of  Student 

• Are  the  students 

o Engaged  and  cognitively  active?  n Yes  □ No 

o Taking  responsibility  for  learning?  □ Yes  □ No 

Role  of  Instructor 

• Is  the  instructor 

o Facilitating  learning  rather  than  directly  teaching? 
n Yes  n No 

o Encouraging  student  ownership  of  the  process?  □ Yes  □ No 
The  CPSRT  promotes  cognitive  flexibility,  a key  component  of  the  constructivist  approach 

In  contrast  with  the  IPSRT  tool,  which  provides  a checklist  for  necessary  elements  of  a instructional  plan,  the  CPSRT  tool 
offers  a reservoir  of  suggestions  from  which  users  can  choose  (Baylor  & Kitsantas,  2001).  The  prompt  questions  in  this  tool  do 
not  provide  definitive  answers,  but  rather  offer  a space  for  users  to  expand  their  creativity  and  imagination  within  the  framework 
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of  constructivist  principles.  The  flexibility  provided  by  the  CPSRT  tool  is  consistent  with  the  less-structured  and  activity-based 
characteristics  of  the  constructivist  instruction. 

The  following  excerpt  about  learning  activities  illustrates  the  flexibility  of  the  CPSRT  tool.  An  activity  will  be  considered 
desirable  for  constructivist  learning  if  any  or  all  of  the  boxes  are  checked.  The  list  of  the  check  box  questions  provides  various 
options  for  the  user  to  choose  or/and  build  upon.  Pre-service  teachers  will  not  feel  limited  by  having  to  come  up  with  only  one 
specific  type  of  activity.  Instead,  they  conveniently  have  a list  of  different  types  of  appropriate  characteristics  for  the  instructional 
activities. 


Desirable  Characteristics 

Are  the  activity(ies): 

• Ill-structured  tasks?  n Yes  n No 

• Complex?  n Yes  □ No 

• Multi-disciplinary?  □ Yes  □ No 

• Involving  cognitive  conflict?  □ Yes  □ No 

• Including  discussion  and/or  collaboration?  n Yes  □ No 

Conclusions 

We  found  that  the  two  tools  improve  instructional  planning  performance  for  plans  developed  according  to  the  systematic 
(IPSRT)  and  constructivist  (CPSRT)  approaches  (Baylor  & Kitsantas,  2001).  Further,  anecdotal  evidence  of  the  value  of  the 
tools  includes  comments  from  the  undergraduate  pre-service  teachers  who,  after  using  the  tools,  wondered  why  they  were  not 
provided  earlier  in  the  course! 

While  both  tools  assist  in  dl  phases  of  instructional  planning,  they  place  a strong  emphasis  on  assessment.  Given  that 
assessment  is  strongly  emphasized  as  a critical  factor  in  teacher- education  programs,  these  tools  encourage  pre-service  teachers 
to  seriously  consider  assessment,  the  last  phase  of  the  instructional  plan.  For  example,  the  IPSRT  assists  them  to  monitor  and 
self-evaluate  their  assessment  procedures,  e.g.,  does  the  assessment  reflect  the  objectives  of  the  instructional  plan?  The  CPSRT 
similarly  requires  them  to  directly  link  the  assessment  to  the  instructional  purpose  and  involve  the  learner  in  appropriate 
performance(s). 

Overall,  the  IPSRT  and  the  CPSRT  are  useful  cognitive  tools  for  pre-service  or  possibly  in-service  teachers  to  promote  the 
effectiveness  of  their  instructional  planning  in  a self-regulated  approach.  If  utilized  appropriately,  these  tools  can  bring  about 
increased  motivation,  intellectual  challenge,  reduced  workload,  and  improved  performance  for  both  pre-service  and  in-service 
teachers. 
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Abstract 

In  J999,  the  University  of  West  Florida  launched  an  online  Instructional  Technology  master's  program.  Students  enrolled  in 
this  online  program  can  be  divided  into  two  groups:  J)  ''local " students  who,  for  various  reasons,  prefer  the  online  courses,  and 
2)  students  at  a geographical  distance.  The  purpose  of  this  study  was  to  identify  factors  influencing  the  satisfaction  of  these 
students  with  the  online  courses.  A second  purpose  was  to  ascertain  any  difference  in  satisfaction  levels  between  the  two  groups. 
The  Biner  instrument  (1993)  was  modified  to  accommodate  questions  relating  to  online  courses.  Fifty^two  respondents  from  a 
sample  of 200  participants  completed  the  online  survey.  The  results  indicated  student  satisfaction  in  online  courses  is  influenced 
by  three  constructs:  instructor  variables,  course  management,  and  technical  issues.  The  statistical  analysis  did  not  reveal 
significant  differences  in  satisfaction  between  the  two  groups.  However^  when  the  researchers  compared  the  differences  in 
distribution  of  responses  between  the  two  groups,  some  interesting  differences  were  found. 

Introduction 

Distance  learning  is  defined  as  instruction  where  “students  and  teachers  are  separated  by  distance  and  sometimes  by  time” 
(Moore  & Kearsley,  1996,  p.  1).  Many  higher  education  institutions  today  are  either  offering  online  courses  and  degree  programs 
or  are  planning  such  initiatives.  In  1997-98,  34%  of  postsecondary  educational  institutions  offered  distance  education  courses 
and  20%  planned  on  offering  distance  courses  by  2000.  Of  these  institutions,  77%  indicated  they  used  the  Internet  as  one  of 
many  instructional  delivery  modes  (National  Center  for  Education  Statistics  [NCES],  1999),  Many  universities  now  offer  a wide 
range  of  online  courses  and  degree  programs  (Laws,  1996).  For  example,  in  1994-95,  51%  of  postsecondary  educational 
institutions  in  the  U.S.  offered  more  than  ten  distance  education  courses;  only  4%  did  not  offer  this  type  of  courses  (NCES, 
1998). 

Enrollment  in  these  courses  has  increased  dramatically  in  the  1990s  (Neeley,  Niemi,  & Ehrhard,  1998).  In  the  academic  year 
1994-95,  formal  online  student  enrollment  was  758,640  (NCES,  1998).  By  1997-98,  that  number  had  increased  to  1,661,100 
(NCES,  1999).  The  growth  in  distance  education  is  largely  credited  to  the  availability  of  technology -enhanced  instruction 
(Hobbs  & Christianson,  1997). 

Historically,  retention  of  distance  learners  has  been  problematic  with  dropout  rates  disproportionably  high  compared  to 
traditional  course  settings  (Richards  & Ridley,  1997;  Wetzel,  Radtke,  & Stem,  1 994).  A dropout  rate  of  30  to  50  percent  was  not 
uncommon  (Moore  & Kearsley,  1996).  Students  may  experience  feelings  of  isolation  in  distance  courses  compared  to  prior  face- 
to-face  educational  experiences  (Shaw  & Polovina,  1999)  because  of  limited  contact  with  instructors  and  fellow  students.  The 
result  of  this  isolation  can  be  unfinished  courses  or  degree  programs  (Keegan,  1990). 

Student  satisfaction  in  traditional  learning  environments  has  been  overlooked  in  the  past  (Astin,  1993)  and  has  not  been 
explored  sufficiently  (DeBourgh,  1999;  Navarro  & Shoemaker,  2000).  Student  satisfaction  has  also  not  been  given  the  proper 
attention  in  distance  learning  environments  (Biner,  Dean,  & Mellinger,  1994).  Many  current  distance  learners  are  “non- 
traditional  students” — ^adults  who  have  important  commitments  such  as  raising  a family  and  maintaining  full-time  employment 
(Richards  & Ridley,  1997).  Non-traditional  learners  may  differ  from  traditional  learners  in  reporting  satisfying  experiences. 
According  to  Donohue  and  Wong  (1997),  further  research  should  be  conducted  to  investigate  causes  of  satisfaction  in  non- 
traditional  students.  Richards  and  Ridley  (1997)  also  suggest  further  research  is  necessary  to  study  factors  affecting  student 
enrollment  and  satisfaction. 

Prior  studies  in  classroom-based  courses  have  shown  there  is  a high  correlation  between  student  satisfaction  and  retention 
(Astin,  1993;  Edwards  & Waters,  1982).  Studies  in  which  distance  learners  were  the  target  population  have  yielded  similar 
results  (Bailey  et  al.,  1 998). 

Many  studies  comparing  distance  education  to  traditional  face-to-face  instruction  have  focused  on  factors  such  as  attrition, 
effectiveness,  locus  of  control,  different  media,  and  student  achievement  (Bailey,  Bauman,  & Lata,  1998;  Navarro  & Shoemaker, 
2000;  Richards  & Ridley,  1997;  Sankaran,  Sankaran,  & Bui,  2000;  Schutte,  1996;  White,  1999;  Wideman  & Owston,  1999). 
However,  research  comparing  “local”  distance  learners  (those  who  could  attend  on-campus  classes)  to  students  who  are 
geographically  “distant”  is  limited. 

In  August  of  1999,  the  University  of  West  Florida  (UWF)  launched  an  online  Master  of  Education  degree  program  in 
instructional  technology.  Faculty  members  who  teach  these  online  courses  employ  a full  complement  of  tools  and  strategies. 
Students  enrolled  in  this  online  program  can  be  divided  into  two  groups:  1)  local  students,  and  2)  students  at  a geographical 
distance.  The  purpose  of  this  study  was  to  identify  factors  influencing  the  satisfaction  of  these  students  with  the  online  courses. 
A second  purpose  was  to  identify  any  difference  in  satisfaction  levels  between  the  two  student  groups. 


Review  of  Literature 

Student  Satisfaction 

Most  college  students  spend  considerable  time,  money,  and  effort  in  obtaining  a quality  education  and  perceive  their 
postsecondary  educational  experiences  as  being  of  high  value  (Knox,  Lindsay,  & Kolb,  1993).  Satisfaction  is  an  important 
“intermediate  outcome”  (Astin,  1993,  p.  278).  Student  satisfaction  is  important  because  it  influences  the  student’s  level  of 
motivation  (Chute,  Thompson,  & Hancock,  1 999;  Donohue  & Wong,  1 997)  which  is  an  important  psychological  factor  in  student 
success  (American  Psychological  Association  [APA],  1997).  Bean  and  Bradley  (1986)  found  student  satisfaction  has  a 
significant  effect  on  performance.  Conversely,  performance  does  not  affect  student  satisfaction. 

According  to  experts,  satisfaction  is  a good  predictor  of  academic  success  (Donohue  & Wong,  1997)  and  retention  (Astin,  1993; 
Edwards  & Waters,  1982).  Elliott  (1999)  notes  postsecondary  educational  institutions  must  retain  existing  students  in  order  to 
achieve  the  goal  of  maximum  growth.  Therefore,  educational  institutions  must  focus  on  student  satisfaction  in  order  to  increase 
retention  (Astin,  1 993).  A postsecondary  educational  institution  may  also  use  student  satisfaction  as  one  measure  of  its  success 
(Knox  etal.,  1993). 

Student  satisfaction  can  be  defined  as  the  student’s  perception  pertaining  to  the  college  experience  and  perceived  value  of  the 
education  received  while  attending  an  educational  institution  (Astin,  1993).  However,  a problem  exists  with  measurement  of  this 
important  outcome  (Williams  & Ceci,  1997).  Course  evaluations,  which  usually  intend  to  measure  the  student’s  satisfaction  with 
a course,  may  not  be  valid  instruments.  For  example,  in  a study  students  rated  the  instructor’s  content  knowledge  based  on 
perceptions  of  enthusiasm  and  on  presentation  style.  They  rated  the  course  based  on  how  much  they  thought  they  had  learned, 
which  did  not  actually  correlate  with  the  amount  they  had  learned.  The  researchers  also  reported  the  overall  course  rating  was 
strongly  correlated  with  the  final  grade  received  in  the  course.  Despite  these  problems,  surveys  administered  to  distance  learners 
after  a course  has  been  completed  can  give  evaluators  valuable  information  pertaining  to  satisfactory  or  unsatisfactory  aspects.  In 
turn,  this  information  can  then  be  used  to  improve  the  course  or  program  (Chute  et  al.,  1999). 

Distance  Education 

Many  advantages  and  disadvantages  exist  for  distance  learners.  A key  advantage  is  convenience  and  flexibility  for  learners. 
This  is  particularly  true  for  adults  who  must  schedule  coursework  around  family  and  career  obligations.  Distance  education 
courses  are  often  self-paced.  With  the  use  of  asynchronous  communication  tools,  learners  have  access  to  content,  instructors  and 
classmates  at  all  hours  (Belanger  & Jordan,  2000).  Another  advantage  for  online  learners  is  they  have  more  time  to  reflect  and 
formulate  their  responses  in  chat  rooms  or  threaded  discussions  compared  to  learners  in  a classroom-based  course  (Moore  & 
Kearsley,  1 996).  This  particularly  benefits  students  who  may  be  reluctant  to  speak  in  a classroom  setting. 

Students  with  limited  access  to  higher  educational  opportunities  also  benefit  from  distance  education.  Learners  who  live  in 
remote  or  rural  areas  or  who  are  restricted  in  mobility  can  access  online  courses.  Others  may  need  access  to  specialized  courses, 
degree  programs,  or  professional  certificates  not  available  in  the  area  in  which  they  live.  Some  students  may  not  want  to  attend 
the  local  colleges  or  universities,  and  distance  education  gives  them  a choice  of  institutions  to  attend.  Another  advantage 
includes  the  increased  access  to  experts  in  the  field  via  telecommunications  (Belanger  & Jordan,  2000;  Hara  & Kling,  2000). 

Disadvantages  for  the  learners  are  loss  of  direct  interaction  with  the  instructor  and  possible  loss  of  motivation  to  complete  the 
course  or  program  (Belanger  & Jordan,  2000).  When  students  are  not  familiar  with  the  technology  used  in  the  course,  it  can  be 
difficult  for  them  to  catch  up  with  the  rest  of  the  group  (Vrasidas  & Mclsaac,  1999).  It  can  also  be  difficult  to  enable  effective 
group  collaboration  and  discussion.  Access  to  resources  such  as  the  campus  library  can  be  problematic  as  well.  In  addition,  the 
potential  for  disruptive  technical  problems  is  an  important  factor  (Belanger  & Jordan,  2000). 

Software  programs  used  to  facilitate  collaborative  learning  have  been  used  successfully  in  online  courses.  Navarro  (2000) 
reports  many  students  are  highly  satisfied  with  online  courses.  Hiltz  (1993)  reports  that  communications  software  increased  the 
quality  of  instruction,  raised  students’  level  of  motivation  due  to  greater  access  to  instructors,  and  increased  their  satisfaction  with 
outcomes.  Powers,  Davis,  and  Torrence  (1999)  also  report  high  student  satisfaction  with  their  level  of  involvement  in  a graduate 
instructional  technology  course. 

In  a study  by  Bower  and  Kamata  (2000),  84%  of  students  indicated  they  were  highly  satisfied  or  satisfied  with  their  online 
courses  experience.  Richards  and  Ridley  (1997)  found  the  majority  of  students  who  completed  online  courses  were  satisfied  with 
their  experiences  and  rated  courses  comparable  to  classroom-based  course.  In  a study  comparing  an  online  and  traditional 
introductory  psychology  course,  researchers  found  students  in  the  online  course  were  satisfied  with  the  course  and  rated  the 
communication  with  the  instructor  as  better  than  in  a classroom-based  course.  However,  the  students  in  the  classroom-based 
course  indicated  they  were  overall  more  satisfied  than  students  in  the  online  course  (Maki,  Maki,  Patterson,  & Whittaker,  2000). 

Some  studies  have  reported  decreased  student  satisfaction  in  online  courses.  Online  students  have  reported  needing  to  work 
harder  in  an  online  course  compared  to  a course  in  the  traditional  setting  (Maki  et  al,  2000).  Students  have  also  reported  anxiety, 
confusion,  and  frustration  with  online  courses  (Hara  & Kling,  2000).  Some  students  reported  feeling  isolated  and  they  had 
problems  overcoming  the  distance  (Wegerif,  1998). 

Factors  Contributing  to  Student  Satisfaction 

In  traditional  settings,  areas  associated  with  student  satisfaction  are  student  characteristics,  quality  of  relationships  with 
faculty,  curriculum  and  instruction,  student  life,  support  services,  resources,  and  facilities.  A study  with  undergraduate  students 
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by  Astin  (1993)  identified  the  following  factors  as  most  important:  contact  time  with  faculty  members  and  administrators, 
availability  of  career  advisors,  student  social  life  on  campus,  and  overall  relationships  with  faculty  and  administrators.  Bean  and 
Bradley  (1986)  concluded  the  best  predictors  of  student  satisfaction  are  academic  integration,  institutional  fit,  quality  and 
usefulness  of  education,  social  life,  and  difficulty  of  program. 

The  instructor  is  the  main  predictor  in  course  satisfaction  (Finaly -Neumann,  1994;  Williams  & Ceci,  1997).  Student 
satisfaction  is  highly  correlated  with  the  performance  of  the  instructor,  particularly  with  his  or  her  availability  and  response  time 
(DeBourgh,  1 999;  Hiltz,  1 993).  Instructors  must  be  available  if  students  have  questions  and  must  be  flexible  (Moore  & Kearsley, 
1996).  The  instructor  not  only  becomes  a facilitator  of  learning  but  also  a motivator  for  the  student. 

The  instructor’s  feedback  is  a key  factor  in  satisfaction  with  the  instructional  environment  (Finaly-Neumann,  1994). 
Feedback  on  assignments  must  be  given  in  a timely  manner  to  keep  learners  involved  and  motivated  (Smith  & Dillon,  1999). 
Communication  must  be  on  a regular  basis  (Mood,  1 995).  Otherwise,  students  can  experience  a great  level  of  frustration  (Hara  & 
Kling,  2000).  Distance  learners  can  experience  feelings  of  isolation,  and  high  levels  of  frustration  and  anxiety  if  communication 
and  interaction  between  the  different  parties  are  lacking  (Mood,  1 995). 

Mood  (1995)  reports  that  course  goals  and  objectives  should  be  clearly  communicated  to  the  students  at  the  beginning  of  the 
course.  If  students  know  what  is  expected  of  them,  their  levels  of  anxiety  can  be  reduced.  Instructors  should  encourage  student 
participation,  provide  updated  information,  and  monitor  student  progress.  Students  should  also  have  opportunities  to  become 
self-directed  learners  and  structure  their  own  learning  experiences  (Wegerif,  1998). 

Students  must  have  access  to  reliable  equipment  (Belanger  & Jordan,  2000).  Students  with  limited  access  are  at  a 
considerable  disadvantage  to  learners  who  have  unlimited  access  (Wegerif,  1 998).  Access  is  one  of  the  most  important  factors 
influencing  student  satisfaction  (Bower  & Kamata,  2000).  Online  learners  must  be  familiar  with  the  technology  used  in  the 
course  in  order  to  be  successful  (Belanger  & Jordan,  2000).  Students  who  experience  frustrations  with  technology  in  a course 
report  lower  satisfaction  levels  (Chong,  1998;  Hara  & Kling,  2000). 

Navigational  components  are  also  important  issues  in  the  online  environment.  Learners  should  be  able  to  move  within  the 
course  Web  site  without  getting  lost  (Aggarwal,  2000).  Hyperlinks  must  work  properly  or  students  will  experience  frustration 
(Harrison,  1999). 

Learning  environments  in  which  social  interaction  and  collaboration  are  allowed  and  encouraged  lead  to  positive  leaning 
outcomes  (APA,  1997).  Collaborative  learning  tools  can  improve  student  satisfaction  in  the  online  learning  environment  (Bonk, 
1998;  Gunawardena  & Zittle,  1998).  These  tools  allow  for  group  work  and  immediate  feedback.  Students  are  able  to  share 
viewpoints  and  discuss  them  with  one  another  in  a virtual  environment,  thereby  gaining  insights  and  perspectives  they  otherwise 
would  not  have  been  exposed  to.  This  type  of  environment  allows  for  social  interaction  and  creates  meaningful,  active  learning 
experiences  (Bonk,  1 998). 

Methodology 

The  University  of  West  Florida  was  founded  in  1963  and  is  located  in  the  Florida  Panhandle.  The  main  campus  is  located  in 
Pensacola,  Florida.  In  1993,  the  Fort  Walton  Beach  campus  was  established  approximately  50  miles  east  of  Pensacola.  The 
university  offers  many  courses  utilizing  two  modes  of  distance  learning  including  online  courses  and  interactive  video  courses. 

Total  university  enrollment  is  approximately  9,000  students. 

In  2000-01,  the  university’s  total  enrollment  was  8,517  students.  Graduate  and  doctoral  students  make  up  16%  of  the  student 
body.  The  College  of  Professional  Studies  has  791  masters  and  doctoral  students  (University  of  West  Florida,  2001). 

Sample 

The  sample  used  in  this  study  was  drawn  from  a pool  of  all  graduate  distance  learners  (507  students)  at  this  university.  The 
researchers  decided  to  split  the  population  in  two  groups:  “local”  and  geographical  “distant”  students.  Zip  codes  of  students’ 
residences  were  used  to  make  the  distinction  between  the  two  groups  of  students.  A “local”  student  was  defined  as  anyone  with  a 
zip  code  starting  with  “325”.  A “distant”  student  was  defined  as  anyone  with  any  other  zip  code.  The  use  of  this  criteria  resulted 
of  a sample  of  363  “local”  and  144  “distant”  students.  A total  of  100  students  were  randomly  selected  from  each  of  the  two 
groups. 

Instrument 

The  Telecourse  Evaluation  Questionnaire  (Biner,  1993)  has  a total  of  42  questions.  This  instrument  measures  student 
attitudes  toward  televised  distance  education  and  addresses  three  factors:  (1)  instruction  and  instructor,  (2)  technology,  and  (3) 
course  management.  With  permission,  the  researchers  modified  the  survey  to  address  issues  related  to  the  online  environment  and 
student  satisfaction.  In  order  to  eliminate  neutral  responses,  participants  were  asked  to  indicate  their  level  of  satisfaction  on  a 4- 
point  Likert  scale  ranging  from  “1  = strongly  disagree”  to  “4  = strongly  agree.”  The  researchers  added  several  questions  relating 
to  general  information  such  as  age,  major,  final  course  grade,  hours  per  week  spent  on  the  course,  and  Internet  access  issues.  The 
researchers  also  added  four  open-ended  questions,  asking  the  most  and  least  satisfying  aspects  of  the  course,  Eason  for 
enrollment,  and  factors  that  could  improve  student  satisfaction.  Because  this  survey  was  significantly  modified  to  adapt  the 
technology  used  in  Web-based  courses,  the  researchers  performed  a reliability  analysis  after  the  data  collection  phase. 
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Data  Collection 

The  online  survey  was  constructed  and  uploaded  to  a server.  A personalized  e-mail  was  sent  to  the  participants  with 
instructions  on  accessing  and  completing  the  survey.  The  researchers  provided  participants  with  the  purpose  of  the  research,  a 
statement  addressing  confidentiality  and  voluntary  participation,  and  contact  information  of  one  of  the  researchers.  Initially,  the 
return  rate  was  21.5%.  After  six  days,  all  participants  were  reminded  via  e-mail  to  complete  the  survey,  which  resulted  in  a final 
response  rate  of  26%  (52  participants). 

Data  Analysis 

The  data  was  examined  for  statistical  assumptions  (e.g.  sample  size  and  missing  data,  linearity,  multicollinearity,  singularity, 
univariate  and  multivariate  outliers).  None  of  the  cases  had  missing  values.  Each  group  was  examined  separately  for  outliers. 
Examination  of  the  scatterplots  revealed  no  univariate  outliers.  A Mahalanobis  distance  test,  ? ^ =22.46  at  p < .001,  was 
performed  and  no  multivariate  outliers  were  detected.  In  order  to  examine  for  linearity,  several  bivariate  scatterplots  were 
generated  and  examined.  All  of  the  scatterplots  revealed  abnormalities  between  the  variables  due  to  the  instrument  being  a 4- 
point  Likert  scale. 

The  Pearson  correlation  coefficients  were  examined  in  a correlation  matrix  in  order  to  determine  if  multicollinearity  existed. 
Many  correlation  coefficients  exceeded  .50  and  the  highest  correlation  coefficient  detected  in  this  matrix  was  .91.  The 
collinearity  diagnostic  demonstrated  variance  proportions  were  below  .64  and  this  leads  to  the  conclusion  that  no 
multicollinearity  existed  between  any  of  the  dependent  variables.  Each  dependent  variable  was  an  independent  measure, 
therefore  ruling  out  singularity. 

A confirmatory  factor  analysis  was  performed  to  subtract  factors  relevant  to  student  satisfaction  as  identified  in  the  literature 
and  to  examine  the  construct  validity  of  the  satisfaction  survey.  The  researchers  expected  six  factors  with  high  subscale  loadings 
for  the  online  course  satisfaction  survey.  An  initial  examination  of  the  data  revealed  eight  dimensions  which  had  eigenvalues 
greater  than  1.  The  examination  of  the  scree  plot,  however,  indicated  the  instrument  has  only  three  components. 

The  factor  loadings  on  the  instructor/instruction  satisfaction  dimension  were  satisfactory  and  explained  50. 12%  of  variance.  The 
other  two  components  had  several  complex  loadings.  A possible  explanation  is  students  associated  many  of  the  course 
management  aspects  with  instructional  issues  and  the  online  learners  might  have  associated  technology  aspects  with  factors 
outside  the  course.  However,  these  results  indicate  the  online  course  satisfaction  survey  is  a true  measure  of  satisfaction. 

Descriptive  measures  were  calculated  for  each  item  on  the  online  course  satisfaction  survey.  A /-test  for  independent  samples 
with  an  alpha  level  of  .05  was  performed  to  determine  if  differences  in  responses  between  groups  were  statistically  significant. 
The  survey  was  then  collapsed  into  six  subscales:  instructor,  technology,  course  management,  course  Web  site,  interactivity,  and 
general  issues.  Respondents’  scores  were  summarized  and  divided  by  the  number  of  questions  in  each  of  the  subscales  and  a 
multivariate  analysis  of  variance  (MANOVA)  was  performed.  Then,  the  subscales  where  collapsed  again  without  dividing  the 
computed  scores  in  order  to  avoid  losing  variance  and  a MANOVA  was  performed  a second  time  to  detect  statistically  significant 
differences  between  the  two  groups.  The  data  analysis  also  involved  open  coding  of  qualitative  data  generated  with  the  use  of 
open-ended  questions. 

Results  and  Discussion 

The  internal  consistency  reliability  of  the  online  course  satisfaction  survey  was  determined  using  the  Cronbach  alpha 
coefficient.  The  overall  reliability  of  the  modified  instrument  was  high  (.97). 

General  Survey  Responses 

Fifty-two  students  who  had  completed  at  least  one  online  course  responded  to  the  Web-based  survey.  Of  the  respondents, 
69%  were  female.  The  majority  of  learners  were  between  40  and  49  years  of  age.  All  but  two  respondents  were  Education 
majors.  Most  respondents  (53.8%)  lived  within  a 30-mile  radius  from  a UWF  campus.  Twenty-one  percent  indicated  they  enroll 
only  in  distance  learning  courses. 

Half  of  the  participants  had  taken  only  one  or  two  online  courses  at  the  time  of  completion  of  the  survey.  The  participants 
spent  an  average  of  12  hours  per  week  working  on  course  activities.  The  hrge  majority  of  participants  (86.5%)  accessed  the 
course  materials  from  home. 

Forty-six  percent  of  the  respondents  indicated  that  they  would  have  been  able  to  take  a course  if  it  had  not  been  offered 
online.  When  asked  why  they  enrolled  in  an  online  course,  the  majority  of  participants  indicated  availability  and  degree 
requirements  were  the  most  important  reasons,  followed  by  convenience,  and  out-of-town  travel  during  the  semester.  These 
findings  are  consistent  with  results  of  previous  studies,  in  which  students’  reasons  for  enrolling  in  an  online  course  were 
convenience,  the  ability  to  take  a course  which  would  otherwise  not  have  been  available  to  them,  or  fulfilling  degree  or  certificate 
requirements  (Bower  & Kamata,  2000). 

According  to  respondents,  Ihe  three  most  satisfying  aspects  were  acquisition  or  improvement  of  technology  skills, 
instructor’s  support  and  responsiveness,  and  flexibility.  The  top  two  least  satisfying  aspects  were  lack  of  face-to-face  contact 
with  the  instructor  and  students,  and  heavy  workload.  When  asked  what  would  increase  satisfaction  in  the  online  course,  several 
students  responded  that  they  could  think  of  nothing,  and  they  were  satisfied  and  enjoyed  the  course.  Some  students  indicated  that 
the  university  should  ensure  students  have  the  necessary  computer  skills  before  allowing  them  into  online  courses. 
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Overall  online  course  experience 

Eighty-four  percent  of  respondents  indicated  they  were  satisfied  with  the  online  course.  Fifty-six  percent  were  more  satisfied 
with  the  online  course  than  with  a classroom-based  course,  and  73.1%  were  satisfied  with  the  course  workload.  Eighty-one 
percent  stated  they  would  enroll  in  another  online  course.  Ninety  percent  were  satisfied  with  their  final  grade,  and  90%  received 
a final  grade  of  “B”  or  higher. 

Ninety  percent  of  respondents  strongly  agreed  or  agreed  with  positive  statements  about  their  satisfaction  with  aspects  of 
online  courses  such  as  quality  of  lessons,  instructor’s  content  knowledge,  opportunities  to  participate,  reliability  of  the 
university’s  server,  course  registration  procedures,  and  external  hyperlinks  used  in  the  course.  Satisfaction  with  the  instructor, 
reliability  of  computer  equipment,  Internet  connection,  administrative  issues,  access  to  resources,  and  course  web  ate  was 
indicated  by  more  than  80%  of  the  respondents. 

Less  than  25%  of  respondents  indicated  dissatisfaction  with  instructor  feedback  and  teaching  methods,  Internet 
communication  tools,  availability  of  course  mentor,  Web  site’s  organizational  structure,  and  personal  familiarity  of  technology 
tools  used  in  the  course.  However,  58%  did  not  agree  there  was  more  interaction  between  all  involved  parties  in  the  online 
course  and  48%  strongly  disagreed  or  disagreed  they  participated  more  in  the  online  course  than  in  a traditional  classroom 
setting.  In  addition,  31%  were  not  satisfied  with  the  effectiveness  of  communication  in  the  online  course. 

Differences  Between  Local  and  Distant  Learners 

The  “distant”  group  consisted  of  more  females  than  the  “local”  group:  81%  and  61%  respectively.  Of  “local”  learners,  90% 
lived  within  30  miles  from  a university  campus.  None  of  the  “distant”  students  lived  closer  than  31  miles  to  a campus  and  the 
majority  of  this  group  (52%)  lived  between  31  to  100  miles  from  a campus.  In  the  “local”  group,  13%  indicated  they  only 
enrolled  in  distance  education  courses,  whereas  one  third  of  individuals  in  the  “distant”  group  were  enrolled  in  only  distance 
learning  courses. 

None  of  the  “local”  students  had  taken  more  than  nine  online  courses.  In  comparison,  20%  of  “distant”  students  had  taken 
more  than  nine  of  these  courses.  Almost  two  thirds  of  “distant”  students  and  one  third  of  “local”  students  would  not  have  been 
able  to  take  the  course  had  it  not  been  offered  online.  The  majority  of  “local”  students  spent  more  time  working  on  the  course. 
Of  “distant”  students,  67%  spent  between  one  to  ten  hours  per  week  working  on  the  course,  whereas  54.9  % of  “local”  students 
spent  more  than  ten  hours. 

The  majority  of  “local”  learners  (45.2%)  did  not  experience  difficulties  accessing  the  Internet.  In  contrast,  the  majority  of 
“distant”  learners  (42.9%)  had  problems  between  one  to  three  times.  Of  students  who  answered  what  grade  they  had  received  in 
the  course,  only  two  students  received  a grade  C.  These  two  students  were  “distant”  students.  All  other  students  received  a grade 
B or  better. 

Overall  online  course  experience 

The  statistical  analysis  revealed  no  statistically  significant  differences  between  the  means  of  the  two  groups  at  the  p < .05 
level.  The  standard  deviations  are  relatively  minor.  Variables  with  a correlation  coefficient  between  .60  and  .80  are  considered  to 
have  a strong  relationship,  whereas  variables  with  a correlation  coefficient  between  .80  and  1.00  have  a very  strong  relationship. 
In  Group  1,  there  were  six  relationships  with  a correlation  coefficient  higher  than  .60;  in  Group  2,  six  relationships  with  a 
correlation  coefficient  above  .60  and  nine  relationships  above  .80  were  detected. 

When  the  researchers  compared  the  differences  in  distribution  of  responses  by  percentages  (“strongly  disagree”  and 
“disagree”  versus  “agree”  and  “strongly  disagree”)  between  the  two  groups,  some  interesting  differences  were  found.  “Local” 
students  were  generally  more  satisfied  than  “distant”  students  with  a few  exceptions.  “Distant”  students  disagreed  far  less  with 
the  statement  there  was  more  interaction  between  all  involved  parties  in  the  online  course. 

The  “distant”  group  was  also  more  satisfied  with  the  online  course  compared  to  a classroom-based  course.  The  technology  used 
in  the  course  was  more  familiar  to  them  and  they  were  more  satisfied  with  the  use  of  threaded  online  discussions  or  forums.  In 
addition,  they  were  slightly  less  negative  about  the  statement  they  participated  more  in  the  online  course  than  in  a traditional 
classroom  setting  and  were  slightly  more  satisfied  with  the  instructor’s  use  of  various  teaching  methods  and  techniques. 

In  general,  “local”  students  were  more  satisfied  with  the  opportunities  given  to  them  by  the  instructor  to  participate  in  the 
course,  the  instructor’s  communication  skills,  and  the  instructor’s  organization  and  preparation.  They  also  indicated  the 
instructor  made  them  feel  more  like  they  were  part  of  the  class  and  belonged,  and  they  were  more  satisfied  with  the  instructor’s 
encouragement.  They  agreed  more  with  the  statement  the  Web  site  was  consistent  and  well  designed  and  their  Internet  service 
provider  was  reliable.  Interestingly  enough,  they  were  slightly  more  satisfied  with  the  university’s  role  in  helping  them  get 
started  in  the  course  and  with  the  accessibility  of  departmental  program  personnel.  The  “local”  students  were  also  more  satisfied 
with  their  final  grade  in  the  course  and  with  the  quality  of  the  weekly  lessons. 

Conclusions 

Students  in  online  courses  face  a number  of  obstacles.  Online  students  who  are  geographically  distant  would  theoretically 
have  more  to  overcome  than  those  who  live  near  the  institution  and  have  ready  access  to  the  instructor,  peers,  and  physical 
campus  resources  such  as  the  library  and  computing  center.  This  study  found  no  statistically  significant  levels  of  satisfaction 
between  local  learners  and  truly  distant  learners  in  online  courses.  Truly  distant  learners  did  not  experience  less  satisfaction  in 
their  online  courses.  Perhaps  they  experience  more  satisfaction  than  one  would  expect  because  they  are  accustomed  to  the 
technology  and  the  environment.  Perhaps  they  have  extra  motivation  because  they  have  no  choice:  they  would  not  otherwise  be 
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able  to  take  the  required  courses  for  their  program  of  study  and  are  therefore  content  with  the  online  environment.  The  only 
alternative  would  be  driving  long  distances  to  physical  campuses  to  have  the  educational  opportunities  their  local  counterparts 
take  for  granted.  This  issue  certainly  requires  further  investigation. 

Interesting  conclusions  we  might  draw  from  learners’  responses  to  the  open-ended  questions  are  that  the  university  should 
consider  more  preparation  of  students  for  online  environment.  This  could  be  in  the  form  of  policies  and  procedures,  orientations, 
and  checklists  of  required  equipment  and  skills.  One  thing  an  institution  can  provide  for  its  distance  learners  is  a handbook  for 
distance  students  with  basic  institutional  information,  policies  and  procedures,  and  technological  skills  and  requirements  (Hardy, 
1999).  This  is  particularly  important  for  learners  who  are  being  introduced  to  the  online  learning  environment.  Many  of  the 
respondents  in  this  study  had  completed  only  one  online  course. 

According  to  the  participants  in  this  study,  the  limited  face-to-face  interaction  was  a drawback.  Perhaps  it  would  benefit  the 
students  to  schedule  an  in-person  meeting  at  the  beginning  of  the  course,  even  though  it  would  be  potentially  inconvenient  for 
students.  Because  limited  face-to-face  contact  was  the  most  frequently  cited  issue  that  limited  satisfaction,  a one-time  meeting 
and  orientation  for  all  class  participants  could  address  this  problem. 

One  recommendation  for  further  research  is  the  investigation  of  differences  in  satisfaction  between  students  who  are  enrolled 
strictly  in  online  courses  and  others  who  take  a mixture  of  Web-based  and  classroom-based  courses.  The  sample  in  this  study 
was  not  large  enough  to  perform  a statistical  analysis  because  only  1 1 students  were  true  distance  learners,  A small  sample  size 
such  as  this  could  be  used  for  an  in-depth  case  study  with  structured  interviews, 

Moore  and  Kearsley  (1996)  warn  student  satisfaction  is  not  correlated  with  actual  student  achievement.  However,  the  fact 
that  satisfaction  is  a contributing  factor  in  motivation,  which,  in  turn,  is  a predicting  factor  of  student  success,  is  reason  enough  to 
be  concerned  about  the  levels  of  satisfaction  students  experience  in  online  courses  and  degree  programs.  The  increase  in 
numbers  of  online  courses  offered  at  postsecondary  institutions  and  the  rising  enrollment  in  these  courses  and  programs  should 
encourage  researchers  to  investigate  student  satisfaction. 
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Abstract 

This  study  was  conducted  to  examine  the  effects  of  instruction  administered  through  written  and  visual  symbol  systems  on  the 
achievement,  confidence,  attitudes,  time-on- task  and  retention  of  formal  and  distance  education  students. 

The  sample  of  the  study  consisted  of  161  undergraduate  students  from  formal  and  distance  education  settings.  Subjects  were 
divided  into  three  groups  and  asked  to  study  the  materials  during  a week.  First  group  studied  the  materials  that  included  only 
written  symbol  systems,  second  group  visual  symbol  systems,  and  the  third  group  both  written  and  visual  symbol  systems.  After 
studying  the  materials,  the  subjects  completed  an  achievement  test  and  a Likert  type  attitude  scale.  Two  weeks  later,  an  identical 
achievement  test  was  administered  again  to  determine  the  retention  of  learning. 

Results  indicated  that  different  forms  of  symbol  systems  had  different  effects  on  student’s  achievement,  confidence,  and 
retention  of  learning.  Attitudes  of  subjects  were  positive  toward  the  symbol  systems  that  they  studied.  Especially,  distance 
education  students  were  positive  toward  both  to  written  and  visual  symbol  systems  while  formal  education  students  were  positive 
only  to  written  symbol  systems. 

Introduction 

Symbol  systems  in  information  processing  are  defined  as  symbolic  representations  of  information  and  requires  interrelation 
of  varying  symbol  forms  such  as  letters,  numbers,  formulas,  figures,  notes,  graphics,  photographs,  etc  (Goodman,  1976). 
Selection  of  symbol  systems  to  use  with  different  media  depends  on  the  nature  of  media  and  its  technological  limitations. 
Students’  selection,  processing,  storage  and  recall  of  verbal,  visual  or  aural  symbol  systems  vary  due  to  their  individual 
differences. 

Media  in  learning  process  show  differences  in  cognitive  information  processing  in  relation  to  their  capabilities,  technologies 
and  symbol  systems  (Kozma,  1991).  In  other  words,  an  individual  utilize  varying  schemata  for  defining,  processing  and  storing 
symbol  systems  relevant  to  the  characteristics  of  media. 

The  schemata  in  memory  are  coded  either  verbally  or  visually  according  to  characteristics  of  symbol  systems  that  carry 
information  and  recalled  back  to  be  used  in  process  of  learning  new  information.  Learning  occurs  with  processing  of  prior  and 
new  information  in  memory  as  an  integrated  entity.  However,  in  some  situations,  new  information  may  not  fit  into  an  existing 
schema  and  either  a new  one,  appropriate  for  new  information,  is  constructed  or  the  existing  schema  is  modified  in  a way  that 
new  information  can  suit.  This  modification  to  new  information  might  be  possible  through  gaining  attention  of  students.  Thus, 
symbol  systems  in  instructional  materials  must  be  designed  carefully  to  acquire  students’  attention  (Wittrock,  1990). 

In  the  light  of  this  view  and  explanations  of  cognitive  psychology  about  information  processing,  a shift  happened  in  media 
comparison  studies  toward  studying  how  learning  occurs  according  to  individual  characteristics  of  students.  So  that  it  can  be 
revealed  how  to  implement  instruction  according  to  available  technological  facilities. 

However,  researchers  in  the  field  of  educational  technology  have  not  reached  a common  consensus  on  which  and  how  attributes 
of  different  media  influence  learning  although  debates  have  been  continuing  for  many  decades.  Thus,  more  detailed  studies  are 
still  needed  to  clarify  this  issue.  This  need  is  more  crucial  in  the  field  of  distance  education  that  naturally  depends  on  heavily 
mediated  learning  activities.  The  quantity  and  quality  of  research  studies  have  done  on  effects  of  media  on  distance  learning 
urges  the  need. 

This  study  was  conducted  to  examine  the  effects  of  instruction  administered  through  written  and  visual  symbol  systems  on 
the  achievement,  confidence,  attitudes,  time-on-task  and  retention  of  formal  and  distance  education  students.  It  is  believed  that 
results  of  the  study  may  help  especially  distance  learning  designers  and  instructors  provide  more  effective,  efficient  and 
appealing  instructional  media  to  their  students. 

Purpose  of  Study 

In  this  investigation,  it  was  sought  to  clarify  the  effects  of  using  different  media  in  both  distance  and  formal  education  on 
learning  outcomes.  In  other  words,  this  study  purposed  to  determine  which  symbol  systems  provide  better  achievement,  higher 
confidence,  lesser  time-on-task,  positive  attitudes,  more  retention  of  learning,  when  used  in  distance  and  formal  education 
settings.  Specifically,  the  following  research  questions  were  addressed: 

1.  In  which  way  is  the  use  of  different  media  and  symbol  systems  effective  on  achievement  of  students  in  distance 
education  and  formal  education? 

2.  How  do  distance  and  formal  education  students’  attitudes  toward  the  media  they  use  and  the  content  they  encounter 
differentiate? 


3.  How  does  the  use  of  different  symbol  systems  influence  confidence  level  toward  achievement  and  retention  in 
distance  education  and  formal  education  settings? 

4.  Does  the  time  students  spend  to  complete  the  task  differ  according  to  instructional  strategy  (distance  and  formal) 
and  symbol  systems? 

5.  In  which  way  is  the  use  of  different  symbol  systems  effective  on  the  retention  of  learning? 


Methodology 

This  investigation  included  a 2X3  factorial  analysis.  The  factors  can  be  seen  in  Table  1,  The  effects  of  these  dependent  variables 
on  formal  and  distance  students’  achievements,  levels  of  self-confidence,  academic  attitudes,  time-on-tasks  and  retention  of 
learning,  which  is  determined  at  the  second  test  administered  two  weeks  later  the  first  one,  are  investigated  in  this  study. 


Table  L Factors  and  Quantity  of  Participants 


Symbol  Systems 

Written 

Visual 

Written  and  Visual 

Distance 

n = 28 

n = 31 

oo 

CM 

II 

c 

Formal 

n = 25 

n-24 

n-25 

53 

55 

53 

The  sample  of  study  consisted  of  161  sophomore  students  from  formal  and  distance  education  settings,  87  of  these  were 
formal  education  students  majoring  in  Economics  and  Business  Programs  of  Anadolu  University,  Others  were  also  studying  in 
economics  and  business  but  they  were  all  in  the  distance  education  programs  of  the  University, 

A 138  minutes  long  video  program  and  a 50  pages  textbook  on  same  topic,  anthropology  were  used  in  as  the  instructional 
materials  of  this  study.  Subjects  were  divided  into  three  groups  and  asked  to  study  the  materials  during  a week.  First  group 
studied  the  materials  that  included  only  written  symbol  systems,  second  group  visual  symbol  systems,  and  the  third  group  both 
written  and  visual  symbol  systems.  It  was  also  asked  students  record  the  time  they  spent  on  studying  the  materials.  After  studying 
the  materials,  the  subjects  completed  an  achievement  test  and  a Likert  type  attitude  scale.  Two  weeks  later,  an  identical 
achievement  test  was  administered  again  to  determine  the  retention  of  learning.  In  addition  to  these,  students  indicated  their  grade 
expectations  on  both  achievement  tests.  These  were  used  to  determine  the  self-confidence  level  of  students. 

In  the  process  of  data  analysis,  means,  standard  deviation,  Pearson  correlation,  two  way  variance  analysis  and  Fisher  LSD 
test  were  used.  Using  MS  Excel  and  Systat  software  completed  all  these  statistical  processes.  For  data  analysis  ,05  Alpha  level 
determined  but  it  was  also  indicated  when  a ,01  Alpha  level  determined. 

In  order  to  determine  the  effects  of  using  different  symbol  systems  (written,  visual  or  combination  of  these)  on  distance  and 
formal  education  students,  five  different  instruments  about  achievement,  self-confidence,  attitudes,  time-on-task  and  retention  of 
learning  were  developed, 

• Achievement  test:  After  studying  the  materials  participants  took  a 50  items  achievement  test,  50%  of  these  items  were 
intended  to  measure  students  recalling  abilities  others  were  comprehension  abilities, 

• Self-confidence:  The  students  were  asked  to  write  down  what  grade  they  expect  to  get  in  the  exam  before  they  started 
to  answer  the  questions.  These  expectations  were  used  as  their  confidence  levels, 

• Time-on-task:  It  was  demanded  to  keep  a record  of  time  spent  for  studying  the  materials.  So  that,  whether  a statically 
significant  difference  are  there  between  in  time  spent  for  studying  and  instructional  strategy  (distance  versus  formal)  or 
symbol  systems  through  a two-way  variance  analysis, 

• Attitudes:  A 30  item  Likert  type  instrument  were  developed  to  determine  the  students’  attitudes  toward  symbol 
systems  and  the  topic  they  studied,  1 5 of  the  items  were  related  to  the  instructional  media  and  others  were  about  the 
content, 

• Retention  of  learning:  Two  weeks  later,  an  identical  achievement  test  was  administered  again  to  determine  the 
retention  of  learning.  Same  as  achievement  test,  it  was  intended  to  determine  if  there  is  any  difference  between  in  the 
recalling  and  comprehension  levels  of  students. 

Results  and  Discussion 

The  results  reached  in  this  investigation  are  given  below  into  four  groups. 
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Achievement 

Results  indicated  that  students  in  face-to-face  education  achieved  more  than  the  students  in  distance  education.  While  the 
groups  studying  only  with  the  written  symbol  systems  in  face-to-face  education  are  more  successful,  in  distance  education  the 
groups  studying  with  both  written  and  visual  symbol  systems  together  are  more  successful.  This  finding  might  be  implying  that 
students  use  their  usual  learning  styles  when  they  are  presented  the  content  in  a different  material.  In  other  words,  using 
textbooks  in  face-to-face  education  as  a basic  resource  results  in  students’  preferring  the  textbooks  and  written  symbol  systems. 
On  the  other  hand,  the  fact  that  students  using  both  written  and  visual  symbol  systems  in  distance  education  are  more  successful 
comparing  the  other  two  groups  may  be  caused  by  their  familiarity  to  the  media  and  determination  of  which  symbol  systems  to 
use  (Weinstein  and  Mayer,  1985), 

While  the  remote  memorization  part  of  the  achievement  test  imply  almost  the  same  results,  the  comprehension  part  of  it,  on 
the  other  hand,  yields  a significant  difference.  In  the  comprehension  part  of  the  achievement  test,  those  who  use  both  written  and 
visual  symbol  systems  in  both  traditional  and  distance  learning  are  more  successful  than  other  groups.  Research  findings  related 
to  the  effects  of  presentation  of  content  via  different  materials  and  different  symbol  systems  support  the  literature  in  general.  For 
example,  Bagget  and  Ehrenfeucht  (1983)  believed  the  strength  of  the  use  of  verbal  and  visual  symbol  systems  together  in 
increasing  the  achievement.  Also,  Pezdek  and  Hartman  (1983)  suggest  that  using  audio-visual  information  together  increases  the 
achievement,  while  Pluss,  Leutner,  Chu,  and  Mayer  (1998)  suggest  visual  and  verbal  elaboration  in  information  processing 
increases  the  achievement.  But  these  findings  have  been  gathered  from  the  learners  in  traditional  education.  On  the  other  hand, 
findings  of  this  research  shows  that  the  students  who  use  only  the  written  symbol  systems  in  face-to-face  education  scored  the 
highest  points  on  the  achievement  test.  This  might  be  implying  that  the  students  in  traditional  learning  usually  prefer  verbal 
learning  habits  in  order  to  create  connections  in  between  their  previous  and  new  learning. 

Confidence 

Confidence  score  means  of  the  students  in  face-to-face  education  are  higher  than  those  who  are  in  distance  education. 
Correlation  between  achievement  scores  and  confidence  scores  was  found  to  be  positive  and  significant. 

Research  findings,  on  the  other  hand,  show  that  there  is  a difference  between  successful  groups  and  confidence  levels.  From 
the  achievement  perspective,  the  group  using  written  symbol  systems  in  traditional  education  is  more  successful  than  the  group 
using  both  written  and  visual  symbol  systems  in  distance  education.  From  the  confidence  perspective,  the  groups  using  only 
visual  symbol  systems  had  higher  confidence  levels  in  both  traditional  and  distance  education.  This  might  be  an  indication  of 
students’  perception  of  visual  symbol  systems  easier  and  finding  the  presentation  of  content  more  attractive, 

Salomon  (1979,  1984)  compared  the  printed  materials  and  television  in  his  researches,,  and  suggested  that  television  is 
perceived  much  easier  compared  to  printed  materials,  but  still  the  learning  is  not  that  strong.  Similarly,  Cennamo,  Savenye  and 
Smith  (1991)  suggest  that  the  way  students  perceive  the  medium  will  make  their  learning  either  easier  or  more  difficult 
depending  on  their  abilities.  In  this  research  also,  findings  support  that  groups  using  only  the  visual  symbol  systems  had  higher 
confidence  levels  since  they  perceived  the  medium  much  easier. 


Attitudes 

Attitude  score  means  of  students  in  distance  education  are  rather  close  to  the  means  of  students  in  face-to-face  education. 
Overall  attitude  scores  for  all  students  are  generally  high.  It  can  be  said  that  students’  attitudes  toward  the  instructional  media  and 
content  are  positive. 

In  the  media  part  of  the  attitude  scale,  both  the  written  symbol  systems  and  written-visual  symbol  systems  groups  showed 
more  positive  attitudes  compared  to  only  visual  symbol  systems  groups.  This  finding  yields  important  clues  for  instructional 
designers,  teachers,  program  producers  and  textbook  writers.  Because  adaptation  to  any  content  can  only  be  possible  by  focusing 
on  the  media.  This  requires  the  design  of  learning  materials  and  symbol  systems  in  a way  that  they  will  get  their  attention  and 
attract  them  (Wittrock,  1990). 

In  the  content  part  of  the  attitude  scale,  only  written  symbol  systems  groups  and  written-visual  symbol  systems  groups  in 
distance  education  exhibit  more  positive  attitudes  in  understanding  the  content.  This  shows  that  in  distance  education,  only  the 
visual  symbol  systems  are  not  enough  to  affect  achievement.  This  finding  supports  Pezdek  and  Stevens  (1984), 

On  the  other  hand,  groups  studying  only  with  the  written  symbol  systems  in  traditional  education,  exhibit  more  positive 
attitudes  in  understanding  of  the  content.  This  implies  that  the  book  is  the  most  effective  material  for  understanding  the  content. 
This  finding  supports  Salomon  (1984).  According  to  him,  printed  materials  are  perceived  to  be  requiring  more  efforts  and 
investment. 

Time-on -task 

The  time  students  spent  on  completing  the  instructional  materials  are  significant  in  both  teaching  method  and  symbol  systems 
variables.  Distance  education  students  spent  more  time  than  the  traditional  groups.  Groups  using  written  and  visual  symbol 
systems  together  spent  more  time  than  those  using  only  written  or  only  visual  symbol  systems.  To  see  how  these  two  variables 
effect  each  other,  Fisher  LSD  test  processed  and  according  to  the  results,  distance  learning  groups  in  both  written  and  visual 
symbol  systems  compared  to  other  five  groups.  Traditional  face-to-face  group  spend  less  time  to  complete  the  material  than 
other  groups. 
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Looking  at  their  achievement  score  means,  results  show  that  there  is  a positive  correlation  between  time-on-task  and 
students’  achievement  in  distance  written  and  visual  groups.  While  the  traditional  written  symbol  group  use  less  time  for 
learning,  still  they  are  more  successful  comparing  to  other  groups.  The  reason  for  this  success  of  students  in  this  group  might  be 
related  to  their  ability  levels  and  entry  learning  behavior  levels.  Different  researches  in  the  literature  suggest  that  learning  content 
from  different  materials  and  symbol  systems  might  change  as  a result  of  their  entry  behaviors  and  abilities  of  students.(Eckhardt, 
Wood  and  Jacobwitz,  1991;  Van  der  Molen  and  Van  Der  Voort,  1997;  1998).  This  effects  student  achievement  either  positively 
or  negatively. 

Future  research  is  needed  to  see  whether  the  findings  are  correlated  to  entry  behaviors  and  abilities. 

Result  of  the  retention  of  learning  test  was  conducted  two  weeks  after  the  achievemet  test.  The  test  indicated  that  each 
group’s  achievement  points  were  decreased.  Formal  students  were  more  successful  than  the  distance  students  at  the  retention 
test.  If  we  look  at  the  symbol  systems’  point  of  view,  formal  group  students  who  have  used  symbol  systems  were  more 
succesful  than  the  other  groups.  Distance  education  students  who  have  used  the  verbal  and  video  symbol  systems  were  more 
succesful  than  the  other  groups. 

Recall  sub  division  of  the  retention  test’s  statistical  results  indicated  that  there  were  some  differantiation  between  the 
achievement  tests’  sub  divisions.  Formal-verbal  was  the  most  successful  group  in  the  context  of  achievement -recall.  In  conrats 
verbal-visual  group  were  the  most  succesful  in  vebal-visual  context.  Those  results  supports  that  the  idea  of  visual  knowledge  can 
be  stayed  more  than  a week  in  memory  (Bagget  and  Ehrenfeucht,  1983),  and  if  visual-verbal  knowledge  can  be  presented 
together  those  knowledge  effects  the  students’  achievement  (Plass,  Lautner,  Chun,  and  Mayer,  1998). 

Formal  written  group  is  seen  the  most  successful  group  in  the  comprehension  sub  division  of  retention  test  while  the  formal 
written  and  visual  group  is  the  most  successful  in  the  comprehension  sub  division  of  achievement  test.  On  the  other  hand,  the 
formal  education  learners’  interaction  level  with  symbol  systems  reveals  that  formal  written  group  is  more  successful  in 
retention-comprehension  while  formal  written  and  visual  group  leads  in  achievement-comprehension.  These  findings  can  be 
interpreted  as  that  formal  education  learners  use  mostly  use  verbal  strategies  in  the  mental  information  processing. 

The  responds  of  students  related  to  confidence  show  no  significant  difference  neither  for  instructional  strategy  nor  symbol 
systems.  Decreases  can  be  seen  in  the  relationship  between  students’  actual  grades  and  indicated  confidence  responses,  and 
retention-confidence  level  of  each  group  compare  to  achievement -confidence.  Positive  and  significant  relation  between  in  the 
retention  test  grades  and  in  confidence  grades  is  also  observed  same  as  relation  between  achievement  and  confidence. 

Another  decrease  can  also  be  noticed  in  each  groups’  means  of  confidence  level  related  to  retention  test,  when  achievement- 
confidence  means  of  the  groups  and  the  means  of  confidence  level  related  to  retention  test  are  compared.  Same  as  achievement 
test  confidence  means,  groups  studied  only  visual  symbol  systems  got  better  confidence  scores  than  others.  According  to  these 
findings,  it  can  be  claimed  that  visual  materials  are  perceived  easier  than  printed  materials. 

In  the  light  of  all  the  findings  indicated  above,  it  can  be  told  that  use  of  different  symbol  systems  for  learners  in  formal  and 
distance  education  settings  shows  assorted  effects  on  learners’  achievement,  attitudes,  confidence,  time-on-task  and  retention  of 
learning.  So  that,  in  order  to  enable  learners  with  diverse  characteristics  to  get  benefits  from  various  symbol  systems, 
instructional  media  must  be  designed  and  utilized  appropriately. 

Suggestions 

According  to  the  results  of  this  investigation  and  the  experiences  gained  during  the  study,  following  suggestions  are  offered 
for  both  practitioners  and  researchers. 

Different  symbol  systems  do  influence  the  achievement  in  different  ways.  So  that,  hstructional  designers  (practitioners) 
should  pay  attention  to  learners’  individual  characteristics  and  distinctiveness  of  symbol  systems  in  order  to  provide  effective, 
efficient  and  appealing  distance  learning  opportunities  as  well  as  formal  learning  practices  in  every  phases  of  instructional  design 
process. 

As  with  any  research  effort,  this  one  raised  a number  of  compelling  questions  worth  further  exploration.  First,  distance 
learners’  individual  characteristics  such  as  age,  sex,  learning  style  might  influence  the  effective  use  of  different  symbol  systems. 
Thus,  people  who  are  interested  in  symbol  systems  (researchers)  in  distance  education  can  investigate  the  relationship  between 
these  characteristics  and  symbol  systems. 

Second,  effects  of  learners’  prerequisite  skills  about  instructional  content  on  distance  learners’  symbol  system  preferences  is 
another  topic  worth  further  investigation.  Third,  the  instructional  content  of  this  study  was  a social  science,  anthropology.  The 
effects  of  symbol  systems  might  differ  on  another  instructional  content.  In  other  words  similar  investigations  should  be  conducted 
in  different  content  areas  such  as  sciences,  mathematics. 

Forth,  another  point  is  that  sometimes  learners  might  find  the  content  more  important  than  the  symbol  system  or  vice  versa. 
In  another  investigation  this  point  might  be  examined  to  clarify  in  distance  education. 
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Abstract 

The  Carl  D.  Perkins  Vocational  and  Technical  Education  Act  of  1998  made  the  community  colleges  the  point  of  access  for 
training  and  education  for  special  populations.  Physical  barriers  are  obvious  problems  that  limit  access  for  individuals  with 
disabilities,  and  community  colleges  have  made  reasonable  accommodations  for  individuals  with  disabilities.  However,  there  are 
on-line  barriers  that  limit  Web-based  content  that  have  often  been  overlooked.  This  study  examined  the  content  accessibility  of 
community  college  home  pages.  A total  of  253  community  college  home  pages  were  evaluated  for  content  accessibility.  Only 
22.1%  of  the  community  college  home  pages  were  accessible  to  individuals  with  disabilities. 


Introduction 

Access  and  opportunity  have  become  the  hallmarks  of  post-secondary  education.  The  community  college  extends  far  beyond 
the  traditional,  limited  freshmen-sophomore  experience  and  provides  a setting  where  almost  anyone  can  learn  (Parlinchak,  1998). 
Community  colleges  serve  all  citizens  and  provide  a range  of  services  that  support  special  populations.  As  the  number  of 
students  continues  to  increase,  especially  among  special  populations,  so  does  the  need  for  support  programs  and  services. 

The  Americans  with  Disabilities  Act  (ADA)  of  1990  provides  the  same  civil  rights  protection  to  individuals  with  disabilities 
that  apply  as  a result  of  race,  gender,  national  origin,  and  religion  (Button  & Wobschall,  1994).  Title  III  of  the  ADA  directs  that 
public  facilities  make  reasonable  modifications  to  control  discrimination  and  support  accessibility  in  policies,  practices,  and 
procedures  (Council  for  Exceptional  Children,  1994).  As  a result  of  this  landmark  legislation,  accessibility  alterations  such  as 
providing  ramps  to  elevated  areas  and  providing  accessible  signage  through  height  adjustments  and  raised  lettering  have  become 
commonplace  across  the  United  States. 

The  Perkins  Vocational  Act  of  1984  called  attention  to  America’s  need  to  support  individuals  who  were  less  fortunate  by 
birth  or  economic  circumstances.  The  Act  underscored  the  need  for  improving  vocational  programs  and  serving  special 
populations  of  students.  The  Act  created  an  awareness  of  the  population  of  people  that  had  gone  unnoticed  with  little  or  no 
training.  This  Act  made  community  colleges  the  point  of  access  for  training  and  education  for  special  populations. 

The  World  Wide  Web  (WWW)  has  become  an  invaluable  resource  for  many  people  with  disabilities.  Accessibility  across 
platforms  and  geographic  distance  makes  the  WWW  an  ideal  universal  tool  for  gathering  and  disseminating  information  (Heflich 
& Edybum,  1998).  In  fact,  it  is  estimated  that  34.4%  of  community  colleges  use  the  Internet  to  disseminate  training  and 
educational  programs  to  special  populations  (Gibson,  2000).  Wong  (1997)  discussed  using  the  Internet  for  increased  self- 
advocacy  by  individuals  with  physical  impairments.  It  is  ironic,  however,  that  while  technological  developments  have  enhanced 
and  provided  new  exciting  opportunities  for  the  WWW,  they  have,  at  the  same  time,  complicated  and  limited  the  accessibility  of 
the  content  and  resources  for  individuals  with  disabilities. 

Physical  barriers  are  obvious  accessibility  concerns.  Web  page  developers  need  to  be  just  as  aware  that  on-line  barriers  can 
create  significant  problems  for  some  users.  The  Americans  with  Disabilities  Act  requires  that  all  organizations  make  reasonable 
accommodations  fbr  individuals  with  disabilities.  Even  though  there  has  not  been  a judicial  ruling  on  WWW  accommodations  for 
individuals  with  disabilities,  home  page  developers  should  work  towards  designing  and  building  Web  sites  that  are  accessible  to 
all  individuals.  It  is  important  that  Web  page  developers  use  and  follow  standards  that  allow  accessibility  to  all  WWW  users. 

A variety  of  disabilities  can  reduce  accessibility  to  the  WWW.  Visual,  hearing,  movement,  cognitive,  speech,  and  other 
impairments  can  limit  availability  of  information.  Assisted  technologies  or  accessibility  aids,  such  as  Braille  output  systems, 
keyboard  modification,  screen  enlargement  utilities,  voice  output  utilities,  and  other  technologies  allow  individuals  with 
disabilities  to  access  information  on  the  WWW.  However,  because  of  the  complexity  of  many  Internet  resources,  some 
information  cannot  be  accessed  with  these  aids.  Developers  of  accessibility  aids  continue  to  identify  and  develop  features  that  can 
overcome  some  of  these  barriers,  but  there  are  many  simple  strategies  that  Web  page  developers  can  use  with  very  little  effort 
that  would  make  their  services  more  accessible. 

The  Trace  Research  and  Development  Center  at  the  University  of  Wisconsin  at  Madison  produced  the  Unified  Web  Site 
Accessibility  Guidelines  (Trace  Research  and  Development  Center,  1998).  This  information  was  transferred  to  the  Web 
Accessibility  Initiative  (WAI)  of  the  World  Wide  Web  Consortium  (W3C)  and  used  to  produce  the  Web  Content  Accessibility 
Guidelines  1.0  (Chisholm  & Vanderheiden,  1999a).  The  primary  goal  of  the  guidelines  is  to  promote  content  accessibility.  The 
guidelines  do  not  discourage  content  developers  from  using  images,  video,  and  other  multimedia  tools;  rather,  they  explain  how 
to  make  multimedia  content  more  accessible  to  a wide  audience. 
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The  Web  Content  Accessibility  Guidelines  1.0  (Chisholm  & Vanderheiden,  1999a)  is  organized  around  two  general  themes 
and  14  guidelines  or  general  principles  of  accessible  design.  The  themes  are  (a)  ensuring  graceful  transformation  and  (b)  making 
content  understandable  and  navigable.  The  document  provides  the  rationale  behind  the  guidelines  and  includes  some  groups  of 
users  who  benefit  when  they  are  applied  to  Web  pages.  In  addition,  a list  of  checkpoints  is  provided  that  explains  how  the 
guidelines  apply  to  typical  content  development  scenarios.  Each  checkpoint  is  specific  enough  to  be  verified,  while  general 
enough  to  allow  Web  developers  freedom  to  use  appropriate  strategies. 

The  guidelines  that  primarily  address  the  theme  of  ensuring  graceful  transformation.  Guidelines  1 through  11,  assist  Web 
page  developers  in  producing  sites  that  remain  accessible  despite  constraints  confronted  by  people  with  disabilities.  For  example. 
Guideline  1 states  that  Web  developers  should  provide  equivalent  alternatives  to  auditory  and  visual  content.  Text  can  be 
rendered  in  ways  that  are  available  to  almost  all  browsing  devices  and  accessible  to  all  users,  but  auditory  and  visual  content  are 
not..  Guidelines  12  through  14  primarily  address  the  theme  of  making  content  understandable  and  navigable.  This  includes 
providing  navigation  tools  and  orientation  information  in  pages  with  maximize  accessibility  and  usability.  Not  all  users  can  make 
use  of  visual  clues  such  as  image  maps  or  graphical  information,  but  with  orientation  information,  users  can  understand  many  of 
these  graphical  images.  The  Web  Content  Accessibility  Guidelines  1.0  document  provides  much  more  detail  in  developing 
content  accessible  Web  pages  (Chisholm  & Vanderheiden,  1 999a). 

Building  Web  sites  that  comply  with  standards  for  accessibility  should  be  a high  priority  for  Web  page  developers.  To  date, 
little  research  has  documented  the  extent  to  which  accessibility  goals  have  been  reached.  The  purpose  of  this  study  is  to  examine 
the  accessibility  of  community  college  home  pages  and  provide  information  on  making  them  accessible  (if  they  are  not)  to 
individuals  with  disabilities. 

Method 

To  examine  the  accessibility  of  community  college  home  pages  a descriptive  study  was  conducted.  The  sampling  technique 
used  to  select  community  college  Web  sites  and  the  evaluation  procedures  are  discussed  in  the  following  section. 

Sampling 

The  population  Web  sites  for  this  study  was  community  colleges  located  in  the  United  States.  A list  of  720  community 
college  Web  sites  was  generated  using  the  search  engine  go.com  (2000).  A random  sample  of  260  community  college  home 
pages  was  selected  for  content  accessibility  evaluation  in  this  study. 

Procedures 

Each  home  page  was  analyzed  using  the  software  package  Bobby  3.2  (Center  for  Applied  Special  Technology,  2000),  which 
allows  researchers  and  other  professionals  to  evaluate  Web  pages  in  accordance  with  the  W3C  Web  Accessibility  Initiative's 
guidelines.  Bobby  3.2  produces  a summary  report  that  consists  of  (a)  the  number  of  Priority  1,  Priority  2,  and  Priority  3 access 
errors,  (b)  user  check  data,  (c)  the  types  of  accessibility  errors,  and  (d)  the  ease  in  correcting  the  accessibility  error.  Priority 
1 access  errors  are  problems  that  seriously  affect  the  page’s  usability  by  people  with  disabilities  and  the  Center  for  Applied 
Special  Technology  (CAST)  strongly  suggest  that  Web  developer  correct  these  errors.  For  a page  to  obtain  Bobby  Approved 
rating,  the  home  page  cannot  contain  any  Priority  1 errors.  Priority  2 access  errors  are  considered  important  for  access  but  are  not 
as  vital  as  Priority  1 . Priority  3 access  errors  are  third-tier  access  problems  that  a Web  developer  should  consider  correcting. 

Some  accessibility  errors  cannot  be  confirmed  using  Bobby  3.2,  but  Bobby  3.2  provides  user  check  data  that  informs  the  user 
that  manual  examination  and  human  judgment  are  required  for  examining  a specific  area  of  the  home  page.  For  example,  when 
different  font  colors  are  detected,  Bobby  3.2  identifies  multiple  color  fonts  and  reports  this  as  a user  check,  meaning  that  it  may 
potentially  be  an  accessibility  problem.  In  this  study  the  user  check  data  was  not  manually  examined  but  will  be  reported  as 
potential  accessibility  problems. 

For  a full  description  of  the  types  of  access  errors  see  the  Techniques  for  Web  Content  Accessibility  Guidelines  1.0 
(Chisholm  & Vanderheiden,  1999b).  In  this  study  only  the  initial  home  page  was  evaluated;  that  is,  no  links  from  the  home  page 
within  the  domain  were  evaluated.  Scores  for  each  home  page  were  tabulated  and  further  analyzed. 

Results 

Of  the  260  community  college  home  pages  randomly  selected  for  this  study,  only  253  pages  were  available  for  evaluation. 
Approximately  three-fourths  (77.1%)  of  the  home  pages  (n=195)  were  not  approved  by  Bobby  3.2  (2000)  as  content  accessible. 
This  indicates  that  at  least  one  Priority  1 error  (seriously  affects  accessibility)  was  detected  on  these  pages.  There  was  an  average 
of  1.01  Priority  1 accessibility  errors  on  the  community  college  home  pages.  In  addition,  the  average  number  of  potential  Priority 
1 accessibility  errors  was  8.48. 

There  were  three  types  of  Priority  1 accessibility  errors  detected  on  the  home  pages.  Most  of  the  community  college  home 
pages  (64.2%)  did  not  provide  alternative  text  for  all  images.  A few  of  the  home  pages  did  not  provide  alternative  text  for  image 
map  hot-spots  (17.3%)  and  did  not  provide  alternative  text  for  each  applet  (5.5%).  All  the  Priority  1 accessibility  errors  were 
rated  as  easy  to  correct. 

Almost  all  the  home  pages  (99.2%)  did  not  identify  the  language  of  the  text.  Approximately  90%  of  all  community  colleges 
home  pages  (a)  did  not  specify  a logical  tab  order  among  form  controls,  links,  and  object,  (b)  did  not  provide  keyboard  shortcuts 
to  links,  (c)  did  not  provide  a descriptive  title  to  links,  and  (d)  used  deprecated  (i.e.,  included  elements  that  have  been  replaced  by 
newer  elements)  language  features.  Using  tables  in  home  pages  create  additional  types  of  accessibility  problems.  Community 
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college  home  pages  used  tables  to  format  text  documents  in  columns  (77.2%),  did  not  provide  a linear  text  alternative  for  tables 
(81 .9%),  and  did  not  provide  a summary  and  caption  for  tables  (77.6%).  Many  of  the  home  pages  used  movement  in  their  images 
(78.7%). 

Using  color  on  home  pages  can  create  problems  in  differentiating  items  on  the  page.  Most  of  the  pages  needed  examining  for 
foreground  and  background  colors  contrast  (92.1%)  and  used  color  fonts  to  convey  information  (87.4%).  The  majority  of  sites 
did  not  use  an  extended  description  to  convey  information  beyond  what  was  in  the  alternative  text  (84.2%).  Again,  the  inclusion 
of  tables  on  home  pages  could  create  potential  accessibility  problems.  Most  of  the  home  pages  needed  to  be  examined  for  the  use 
of  structural  markup  to  identify  their  hierarchy  and  relationship  (80.7%)  and  examined  for  the  presence  of  headers  for  the  table 
rows  and  columns  (72.4%).  When  scripts  are  used  to  convey  information  or  functionality,  alternative  content  needs  to  be 
provided  (54.3%). 

Discussion 

Community  colleges  have  played  an  important  role  in  the  training  and  education  of  individuals  with  disabilities.  This  study 
provides  empirical  evidence  that  most  community  college  home  pages  are  not  accessible  to  individuals  with  disabilities.  With 
very  little  effort  all  the  home  pages  could  easily  be  corrected  to  eliminate  the  more  severe  Priority  1 accessibility  errors. 

Web  developers  at  community  colleges  need  to  examine  their  Web  sites  for  accessibility  problems.  It  is  strongly 
recommended  that  validation  methods  be  used  in  the  early  stages  of  Web  development,  which  will  help  make  problems  easier  to 
correct  and  assist  developers  in  avoiding  many  accessibility  problems.  There  are  two  suggested  methods  of  validating  a Web 
page  for  accessibility  (Chisholm  & Vanderheiden,  1999a).  First,  automatic  tools  are  available  for  scanning  the  site  and  providing 
data.  Bobby  and  other  validation  services  should  be  used  to  provide  information  concerning  accessibility  problems.  Automatic 
tools  are  convenient  but  do  not  identify  all  accessibility  issues,  therefore  it  is  recommended  that  each  site  be  examined  by  a 
knowledgeable  individual  and  individuals  with  disabilities  to  ensure  clarity  of  language  and  ease  of  navigation.  The  processes  of 
rapid  prototyping  and  formative  evaluation  have  been  used  for  many  years  to  help  develop  educational  software  and  have 
recently  proven  to  be  useful  in  the  process  of  Web  site  development  (Corry,  Frick,  & Hansen,  1997).  Expert  and  novice  users 
with  disabilities  should  be  invited  to  view  home  pages  and  provide  feedback  about  accessibility  or  usability  problems  and  their 
severity. 

Community  colleges  are  leaders  in  educating  special  populations.  The  WWW  has  become  an  invaluable  resource  for 
notifying  special  population  students  of  services  available  at  their  institution.  Creating  home  pages  that  are  accessible  to  a diverse 
group  of  users  would  insure  the  universality  of  the  WW  W. 
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Generative  Strategies  for  Reading  Comprehension 
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Abstract 

Reading  management  programs  have  become  very  popular  in  elementary  schools.  Students  select  a title  from  a prescribed 
reading  list  then  are  tested  for  recall  of facts  and  events  in  the  story.  Students  are  motivated  to  read  as  many  titles  as  possible 
since  they  must  achieve  a certain  score  to  win  prizes.  Although  there  have  been  reports  that  this  increases  voluntary  independent 
reading,  programs  like  Accelerated  Reader®  do  little  to  increase  comprehension  or  enhance  reading  instruction.  Generative 
strategies  use  integration  and  organizational  activities  to  increase  both  recall  and  comprehension  of  textual  information.  For  this 
study,  students  were  assigned  two  different  book  titles  for  independent  reading.  Using  thinking  maps,  the  students  analyzed  the 
information  in  the  books,  then  used  these  to  generate  test  questions  for  a mock  Accelerated  Reader  test.  The  results  show 
marginal  differences  between  pre  and  posttest  scores,  however  there  was  a significant  improvement  when  students  tested  each 
other  with  their  own  test  items.  In  addition,  students ' test  items  were  aligned  with  the  Levels  for  Thinking  model  used  by  the 
North  Carolina  Department  of  Public  Instruction  for  end-of-grade  tests  for  higher  level  thinking.  Half  of  the  student-generated 
test  items  matched  higher  level  thinking  categories.  Students  were  able  to  generate  multiple  choice  test  items,  distractors,  and 
correct  answers  at  a high  level  of  thinking.  Three  models  were  used  to  compare  student-generated  and  criterion  reference  test 
items:  Generative  strategies  (Wittrock),  Thinking  Maps  (Hyerle),  and  Dimensions  for  Thinking  (Marzano). 


Introduction 

The  Accelerated  Reader©(AR)  program  is  becoming  increasingly  popular  in  elementary,  middle  schools,  and  high  school 
reading  programs.  Currently  there  are  approximately  43,000  schools  across  the  United  States  (Education  Commission  of  the 
States,  1999)  that  use  the  AR  software.  Students  choose  from  a list  of  titles  that  have  been  selected  for  their  prescribed  reading 
level  then  take  quizzes  to  show  mastery  of  content.  Although  these  reading-management  programs  have  contributed  to  an 
increase  in  voluntary  reading,  there  are  few  reports  (Mathis,  1996;  Turner,  1993)  suggesting  that  AR  programs  enhance 
comprehension  or  are  useful  as  a tool  for  reading  instruction.  Most  of  the  AR  quizzes  appear  to  measure  recall  of  factual 
information  only.  Generative  learning  theory  supports  the  use  of  cognitive  strategies  to  promote  improved  comprehension  in 
addition  to  recall  of  facts.  There  are  a number  of  activities  that  could  be  considered  generative.  These  include  developing  test 
questions,  writing  summaries  from  a passage  of  text,  elaboration  by  developing  a multimedia  presentation,  and  analysis  of 
content  by  generating  charts  and  tables  (Morrison,  Ross,  & Kemp,  2001). 


Purpose  of  the  Study 

Voluntary  independent  reading  may  be  increased  through  computerized  reading-management  programs,  but  the  tests  used  by 
these  programs  are  largely  designed  to  measure  recall  of  main  ideas,  characters,  settings,  and  sequence  of  events  in  the  story.  If 
the  reader  does  not  gain  a conceptual  understanding  of  ideas  and  principles  communicated  in  the  books,  tests  similar  to  those 
administered  through  the  reading- management  programs,  may  not  reveal  deficiencies  in  comprehension.  Research  in  generative 
learning  theories  have  provided  much  evidence  to  support  the  use  of  generative  strategies  to  increase  comprehension  of  ideas  and 
concepts,  in  addition  to  recall  of  facts.  Comprehension  of  textual  materials  read  during  sustained  voluntary  reading  sessions  may 
be  increased  by  combining  two  types  of  generative  strategies. 

First  there  are  “organizational”  strategies  for  the  analysis  and  interpretation  of  textual  information.  Through  the  use  of 
organizational  charts,  students  analyze  the  ideas  presented  in  a reading  passage.  Sentences,  paragraphs,  and  chapters  are  distilled 
into  basic  ideas  and  concepts.  The  product  of  this  analysis  is  displayed  through  visual  tools  such  as  bubble  maps  and  flowcharts. 

Second,  there  are  “integration”  strategies  that  use  the  integrative  approach  with  student -generated  test  questions.  Using  their 
own  thinking  maps  to  interpret  reading  passages,  students  could  generate  test  items  consistent  with  higher  level  thinking 
processes  similar  to  Dimensions  in  Thinking  and  those  recommended  by  the  North  Carolina  State  Department  of  Public 
Instruction  for  end  of  grade  tests.  Students  should  be  able  to  connect  what  they  already  know  about  a topic  with  the  new  ideas 
and  concepts  read  in  the  text.  Thus,  three  questions  were  generated  to  further  investigate  these  strategies  for  teaching  reading  and 
writing  comprehension;  (1)  can  students  generate  test  items  consistent  with  higher  level  thinking  processes  similar  to  those 
recommended  by  NCDPI  and  Marzano’s  model  for  higher  level  thinking,  (2)  will  the  use  of  organizational  strategies  for 
generating  original  reading  comprehension  questions  significantly  affect  student  performance  on  a criterion  referenced  test  for 
main  idea,  sequence  of  events,  fact  versus  opinion,  and  cause/effect  factors,  and  (3)  will  students  perform  better  on  a criterion 
referenced  test  than  their  own  student  -generated  test  for  reading  comprehension? 
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Review  of  the  Literature 

Generative  Learning  Theory  (GLT)  first  appeared  in  the  literature  in  the  early  1970’s  (Wittrock,  1974).  Since  then,  reports  on 
the  effective  use  of  generative  strategies  for  improving  recall  and  comprehension  skills  for  reading  (Dunlap,  1999;  Volk  & 
Ritchie,  1999;  Wittrock  & Alesandrini,  1990;  Wittrock,  1991)  have  continued  to  be  published.  GLT  is  based  on  brain  research 
which  suggests  that  neural  processes  for  learning  are  deeper  and  more  lasting  when  connections  are  made  between  prior 
knowledge  and  new  information.  Wittrock  (1992)  defines  generative  learning  as  a process  that  leads  learners  to  see  relationships, 
(1)  across  concepts  and  (2)  between  prior  learning  and  new  information.  Evidence  that  these  connections  are  meaningful  can  be 
found  students’  writings  of  summaries,  metaphors,  paraphrases,  and  outlines.  These  strategies  all  fit  in  one  category  for 
generative  learning  known  as  integration  strategies  (Volk  & Ritchie,  1999).  In  a second  type  of  generative  strategy,  students  may 
use  tools  for  analysis  of  textual  material  for  the  purpose  of  seeing  relationships  between  ideas,  concepts,  or  events  in  a reading 
passage.  A variety  of  tools  can  be  used  to  accomplish  this  analytic  type  of  processing.  These  include  students’  generated  charts, 
tables,  graphs,  and  concept  maps  and  are  known  as  organizational  strategies. 

Thinking  Maps 

The  use  of  visual  tools  for  organizing  ideas  and  concepts  found  in  textual  information  has  been  reported  as  useful  for  helping 
students  translate  what  they  have  read  into  graphic  images.  These  are  in  the  form  of  thinking  maps.  Hyerle  (2000)  has  organized 
these  thinking  maps  into  eight  primitive  formats,  circle  map,  bubble  map,  double  bubble-map,  tree  map,  brace  map,  flow  map, 
multi-flow  map,  and  bridge  map.  Each  of  the  primitives  is  unique  in  form  and  purpose.  There  are  a number  of  benefits  for  using 
the  maps.  By  organizing  information  into  the  appropriate  map,  students  are  able  to  increase  memory  of  factual  information,  gain 
deeper  conceptual  understanding,  communicate  abstract  concepts,  and  enhance  creativity  for  perception-taking. 

Dimensions  for  Thinking 

North  Carolina  has  built  higher  order  thinking  skills  into  classroom  activities  in  all  content  areas  for  K:12  schools 
(Houghton,  1994).  {Thinking  Skill  Levels . 

Available:  [http://www.ceap.wcu.edu/Houghton/Leamer/Think94/homeNCthink94.html]).  These  activities  were  developed  along 
with  accountability  measures  adopted  by  the  state  in  the  form  of  End-Of-Grade  Testing  (North  Carolina,  1992-1993). 

Activities  for  the  classroom  were  planned  around  thinking  skills  levels  developed  by  Robert  Marzano  (1988,  1992).  Marzano’s 
model  is  a framework  for  higher  level  thinking  applied  to  specific  strategies  for  the  classroom.  It  contains  eight  categories: 
focusing,  information  gathering,  remembering,  organizing,  analyzing,  generating,  integrating,  evaluating.  NCDPI  curriculum 
specialists  reduced  these  eight  categories  to  seven  by  collapsing  the  subcategories  for  focusing,  information  gathering,  and 
remembering  to  one  category,  “knowledge”.  Table  1 shows  how  generative  strategies,  as  defined  by  Morrison,  et  al.,  (2001), 
compare  with  Hyerle’s  Thinking  Maps  and  the  NCDPI’ s adaptation  of  Marzano’s  model.  Dimensions  for  Thinking. 


Table  1 Generative  strategies,  Thinking  Maps,  and  Dimensions  for  Thinking  are  compared  as  generative  learning  models. 


Morrison,  et  al. 
Generative 
Strategies 
(based  on 
Wittrock) 

David  Hyerle 
Thinking  Maps 

NCDPI  adapted  from 
Marzano’s 

Dimensions  for 
Thinking 

Strategies  for  Teaching 

Recall 

Context/frame  of 
reference 

Knowledge 

>Using  drill  and  practice  software,  repetitive  games  with 
feedback 

>Circle  map  for  brainstorming,  defining  words 
List  attributes  or  list  the  steps  in  a procedure 

Organizational 

Describing  qualities 

Compare  and 

contrast 

Classification 

Whole/part 

Sequencing 

Organizing 

Analyzing 

>Using  productivity  software  to  generate  charts,  graphs,  or 
tables 

>Tree  map  for  organizing  topics 

>Organize  notes  from  lecture 

>Bubble  maps  for  organizing  and  identifying  key 

components  from  information 

>Classifying  groups  of  items  into  categories  on  the  basis  of 
attributes. 

Elaboration 

Cause  and  effect 
Analogies 

Applying 

>Using  different  media  forms  to  generate  an  expanded 
version  of  concept  or  idea 

Evaluating 

>Using  flow  maps  to  sequence  story  parts,  analyze  and 
prioritize  event  and  identify  cause  and  effect  relationships 
>Judging  the  value  or  logic  of  ideas 
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Integration 

Whole/part 

Generating 

>Use  word  processor  or  webpage  editor  to  generate 

Integrating 

summaries,  outlines,  and  analogies 

Analogies 

>Bridge  maps  for  comprehending  analogies,  similes,  and 
metaphors 

Cause  and  effect 

>New  information  and  prior  knowledge  are  connected, 
combined  and  incorporated  to  generate  a cohesive  statement 
with  new  understanding. 

Sample 

The  sample  consisted  of  twenty  4**"  grade  students  from  a suburban  elementary  school.  There  were  19  Caucasian  children  and 
1 African  American  child.  None  of  the  students  were  eligible  for  free  or  reduced  lunch  programs,  thus  it  was  expected  that 
students  came  from  middle  to  high  SES  environments.  None  of  the  students  had  been  administered  the  AR  leading  test  for 
reading  material  used  in  the  study,  therefore  few,  if  any,  of  the  children  had  read  the  books  used  for  the  treatment. 


Method  and  Materials 

Students  were  assigned  two  titles.  The  Velveteen  Rabbit,  and  The  Lion,  the  Witch,  and  the  Wardrobe.  Each  child  had  his  or 
her  own  copies  purchased  with  funding  from  a university/school  partnership  grant.  Students  were  introduced  to  the  researcher 
and  told  they  would  read  the  books,  then  as  a special  project,  would  create  original  Accelerated  Reader®  tests  for  other  students 
in  the  school.  They  were  also  told  that  they  should  generate  thinking  maps  related  to  the  content  of  the  books.  Prior  to  this  study, 
the  students  and  the  classroom  teacher  had  all  received  instruction  in  the  use  of  thinking  maps  for  organizing  and  interpreting 
information,  and  were  familiar  with  the  eight  types  of  thinking  maps  as  described  by  Hyerle  (2000) 

The  first  treatment 

After  each  child  had  read  The  Velveteen  Rabbit,  students  were  administered  a reading  comprehension  test  developed  by  the 
researcher  and  the  classroom  teacher.  After  the  tests  were  scored,  the  teacher  prompted  students  as  they  brainstormed  and 
generated  phrases  and  key  terms  for  the  thinking  maps.  Inspiration®,  a flowcharting  software  designed  for  elementary  through 
high  school  age  students,  was  used  to  display  the  maps  on  a wall  screen.  Hard  copies  of  the  maps  were  printed  and  returned  to  the 
students.  Students  were  then  instructed  to  use  their  thinking  maps  as  a guide  for  writing  the  questions  that  could  be  used  for  the 
AR  reading  tests.  These  activities  were  spread  over  several  days.  Students  were  also  given  skills  instruction  in  how  to  use  the 
Inspiration®  software  and  how  to  develop  tests  using  the  authoring  program,  HyperStudio®.  Many  of  the  students  WCTe  familiar 
with  both  software  packages  and  needed  less  skills  training  than  had  been  anticipated  by  researcher.  After  the  test  items  had  been 
written,  students  were  administered  a posttest.  Since  the  Velveteen  Rabbit  was  below  the  recommended  reading  level  for  this  4* 
grade  class,  (AR  suggests  grades  1 -3  for  this  title),  difference  between  the  pre  and  posttest  was  not  significant.  The  purpose  in  the 
first  treatment  was  to  investigate  possible  technical  problems  and  to  determine  the  computer  skill  level  of  the  students.  The  first 
treatment  was  actually  a pilot  test  for  treatment  number  2. 

The  second  treatment 

Students  were  given  an  independent  reading  assignment  for  The  Lion,  the  Witch,  and  the  Wardrobe.  Each  child  had  his  or  her 
own  copy  of  the  book  and  was  given  the  freedom  to  read  the  book  independently  at  home,  or  in  their  school  classroom.  After  a 
period  of  two  weeks,  all  the  children  had  completed  the  book  and  were  given  a pretest  created  by  the  researcher.  Items  for  this 
test  were  categorized  by  (1 ) Main  Idea,  (2)  Sequence  of  Events,  (3)  Fact  vs.  Opinion,  and  (4)  Cause  and  Effect.  Items  were 
developed  to  measure  higher  levels  of  comprehension  and  were  aligned  with  the  North  Carolina  Department  of  Public 
Instruction’s  Model  for  Thinking  Skill  Levels  (NCDPI, 

Available  online:  [http://www.ceap.wcu.edu/Houghton/LeamerA'hink94/NCmarzanoThink.html]).  The  Thinking  Skills  Levels 
model  is  also  used  to  develop  End  of  Grade  (EOG)  tests  that  determine  promotion  or  retention  for  grades  4 and  8.  In  addition  to 
aligning  test  items  with  Thinking  Skill  Levels,  the  test  was  reviewed  by  the  classroom  teacher.  The  researcher  was  confident  in 
her  ability  to  evaluate  the  test  as  reliable  for  measuring  reading  comprehension.  There  were  two  reasons  for  this;  first,  she  had 
worked  as  a trainer  in  the  area  of  student  assessment,  and  second,  she  was  the  designated  coordinator  for  professional 
development  in  her  building.  Items  have  been  aligned  with  higher  level  thinking  processes  according  to  Marzano’s  Model  for 
Dimensions  of  Thinking  in  Appendix  I. 

After  the  pre  test  was  completed  and  scored,  students  were  assigned  thinking  maps  to  analyze  the  content  of  the  book. 
Students  used  bubble  maps  and  flowcharts  to  present  the  visual  images  for  concepts  related  to  book  characters,  factors  for  cause 
and  effect,  and  sequence  of  events.  The  maps  were  generated  in  two  formats;  first,  the  researcher  developed  the  maps  using 
Inspiration®  as  the  students  dictated  related  concepts  and  ideas  for  the  maps,  and  second  as  an  independent  activity  using  paper 
and  pencil. 

After  maps  were  completed,  students  were  given  the  assignment  to  “write  the  test  questions”  for  the  AR  test  program.  This 
was  an  independent  activity  in  which  students  wrote  the  questions  at  their  desk  using  paper  and  pencil.  They  were  told  that  this 
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would  be  their  official  “AR  test”  and  could  be  used  by  others  in  the  school  to  test  their  reading  comprehension  of  The  Lion,  the 
Witch,  and  the  Wardrobe.  Students  were  also  instructed  to  use  information  generated  in  the  thinking  maps  to  write  their  test 
questions.  Each  student  wrote  5 to  7 test  questions  for  the  final  “Accelerated  Reader”  test  that  would  be  generated  using  the 
HyperStudio®  software  in  the  school’s  computer  lab. 

Following  this,  over  a three-week  period,  students  were  divided  into  small  groups  and  taken  to  the  school  computer  lab.  The 
researcher  and  the  classroom  teacher  monitored  students  as  they  entered  questions  into  the  HyperStudio  “reading  test”.  The  test 
was  a template  prepared  by  the  researcher  and  saved  to  disks  for  each  student  to  use  independently.  Students  were  encouraged  to 
enter  as  many  questions  as  time  permitted  and  to  create  elaborated  versions  of  their  test  with  reward  buttons  linked  to  “correct 
answers”.  Time  limitations  allowed  only  two  visits  to  the  computer  lab,  thus,  the  average  number  of  test  items  completed  by 
each  student  was  four  questions. 

Following  the  computer  lab  sessions,  a posttest  was  administered  and  scored  by  the  researcher.  The  items  in  the  posttest  were 
identical  to  those  in  the  pre  test  for  measuring  reading  comprehension  of  (1)  Main  idea,  (2)  Sequence  of  Events,  (3)  Fact  vs. 
Opinion,  and  (4)  Cause  and  Effect. 

Analysis  of  Data 

The  content  of  student  questions  (See  Appendix  II)  was  categorized  by  levels  of  thinking  defined  by  NCDPI  model  for 
Dimension  for  Thinking,  which  is  also  used  as  a framework  for  test  items  written  for  EOG  Tests.  Each  item  was  matched  to  one 
of  the  following:  knowledge,  organizing,  analyzing,  applying,  evaluating,  generating,  or  integrating.  These  categories  represent 
thinking  processes  that  would  be  required  by  the  test  taker  as  well  as  the  test  giver.  It  is  assumed  that  students  who  generated  the 
test  item  for  a multiple  choice  test,  must  first  write  the  test  question,  then  be  able  to  identify  the  correct  answer  as  well  as 
generate  several  distracter  responses.  Thus,  students  would  engage  in  generative  strategies  for  developing  test  questions,  and  at 
much  higher  levels  than  simple  recall  of  facts. 

The  posttests  were  scored  by  number  of  correct  responses.  SPSS  was  used  to  calculate  differences  in  means  for  pre  and 
posttests.  T test  for  paired  samples  was  used  to  determine  significant  differences  between  the  pretest  and  posttests  generated  by 
the  researcher  and  posttests  generated  by  students. 

Results  and  Discussion 

Test  items  developed  by  the  researcher  were  aligned  with  the  NCDPI  recommendations  for  test  development  for  higher  levels 
of  reading  comprehension.  As  can  be  seen  in  Appendix  II,  most  of  the  items  could  be  categorized  as  organizational.  Generative 
strategies  for  organization  help  students  analyze  information,  see  relationships,  compare  and  contrast  concepts,  and  restructure  a 
large  amount  of  information  into  appropriate  categories.  Compared  to  the  researcher’s  posttest,  student -generated  items  for  the 
posttest  contained  more  recall  type  questions  but  also  included  items  for  analysis,  organization,  application,  evaluation,  and 
integration. 

Differences  in  pre  and  post  test  scores 

There  was  some  variation  in  the  spread  of  the  scores  for  the  researcher’s  pre  and  posttest.  There  was  very  little  variation  in 
the  scores  for  the  student -generated  posttest  scores.  As  can  be  seen  in  the  box  charts  in  Figure  1,  test  scores  on  the  student - 
generated  test  were  tightly  clustered  around  the  mean  value  85.  Scores  were  more  varied  in  the  pre  test  (m  = 71 .5,  sd  = 16.7)  than 
in  the  post  test  (m  = 84.7,  sd  = 13.5) 
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Figure  1 Box  charts  showing  spread  of  scores  around  the  means  for  pretest  and  both  posttests. 
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20 
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Since  pre  and  posttests  were  administered  to  the  same  sample  of  students,  there  should  be  a strong  positive  correlation 
between  test  scores  on  the  pre  test  with  the  scores  on  the  researcher’s  posttest  as  well  as  a strong  positive  correlation  between  the 
pre  test  and  the  student -generated  posttest.  The  paired  sample  statistics  (See  Table  2)  revealed  a significant  correlation  (p  < .05) 
between  the  pre  test  and  post  tests  (r  = .76,  df  = 19;  r =.  77,  df  = 19). 

Table  2 Paired  Samples  Correlations 


N 

Correlation 

Sig. 

Pair  1 

MYPRE& 

MYPOST 

20 

.761 

.000 

Pair  2 

MYPRE& 

STUDPOST 

20 

.773 

.000 

As  can  be  seen  in  Table  3,  the  t of  1.74  for  differences  in  the  researcher’s  pre  and  posttest  was  not  significant  at  the  .05  level, 
however,  there  was  a significant  difference  in  means  between  the  pre  test  and  the  student -generated  post  test  (t  = 5.57,  df=19,  p < 
.000). 
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Table  3 Paired  Samples  Test 


Paired 

Differences 

Mean 

Std,  Deviation 

Std, 

Mean 

95% 

ErrorConfidence 
Interval  of  the 
Difference 

t 

df 

Sig,  (2- 
tailed) 

Pair  1 

MYPRE  - 
MYPOST 

-4.25 

10.88 

2.43 

Lower  Upper 

-9.34  .844 

-1,74 

19 

,097 

Pair  2 

MYPRE  - 
STUDPOS 

-13.25 

10.62 

2.37 

-18.22  -8,27 

-5.57* 

19 

.000 

Conclusions 

Close  to  half  the  student -generated  test  items  were  written  to  measure  recall  of  basic  facts.  These  test  items  would  be  aligned 
with  the  students’  thinking  processes  for  recall  of  who,  what,  when,  and  where  did  this  happen  in  the  story,  however,  over  half 
the  students’  tests  questions  were  written  at  a higher  level  than  simple  recall  of  facts.  Since  students  developed  the  items 
independently,  this  study  provides  strong  evidence  to  suggest  that  elementary  children  are  able  to  use  thinking  maps  or  other 
visual  tools  with  generative  writing  strategies  for  deeper  processing  of  textual  information.  From  their  own  analyses,  students 
were  able  to  generate  test  questions  that  could  be  used  to  measure  the  higher  level  thinking  processes  of  their  peers.  In  addition, 
students  were  able  to  evaluate  the  accuracy  of  correct  responses  and  determine  the  suitability  of  distracters  for  a reading 
comprehension  test.  It  cannot  be  determined  from  this  study,  the  full  effect  of  the  thinking  maps  as  visual  tools  for  interpreting 
and  analyzing  textual  information.  Further  study  is  planned  to  investigate  differences  in  the  quality  of  generated  test  items  with 
and  without  the  use  of  the  thinking  maps.  In  addition,  there  should  be  an  investigation  of  the  effect  of  computerized  software  for 
generating  maps  when  compared  to  simple  maps  produced  by  paper  and  pencil.  Volk  and  Ritchie  (1999)  found  significant  effects 
from  the  interaction  of  groups  with  the  use  of  generative  strategies  using  manipulatives.  Students  for  this  study  worked  in  a 
whole  group  session  facilitated  by  the  teacher  and  the  researcher  to  generate  thinking  maps.  Their  questions,  however,  were 
generated  independently  using  the  completed  thinking  maps  as  a blueprint  to  construct  the  items.  Further  study  in  students’  use  of 
generative  strategies  to  write  test  questions,  with  or  without  electronic  thinking  maps,  may  reveal  similar  interaction  effects  on 
their  performance  on  the  posttest. 

The  bubble  format  was  the  thinking  map  of  choice  for  the  students.  Hyerle  (2000)  defines  the  bubble  map  as  an  analytic  tool. 
This  type  of  thinking  and  mental  processing  should  help  the  student  to  develop  a deeper  conceptual  understanding  of  a reading 
passage.  For  example,  students  used  the  bubble  map  to  analyze  the  personalities  of  all  the  characters  in  The  Lion,  the  Witch,  and 
the  Wardrobe.  Each  character’s  personal  traits  were  defined,  discussed,  and  then  identified  as  villain,  hero,  good,  evil, 
protagonist,  etc.  For  some  of  the  maps,  students  developed  a flowchart  to  analyze  the  sequence  of  events.  During  the  whole  class 
sessions,  students  carefully  analyzed  the  order  of  events  for  the  story.  From  this  kind  of  mental  processing,  it  would  be  expected 
that  students  would  write  more  test  items  for  organizing  and  analyzing  ideas.  It  is  surprising,  then,  to  discover  that  the  students’ 
test  items  were  in  the  categories  for  applying,  evaluating  and  integrating  ideas.  For  example,  one  student  wrote,  “9.  Why  did  Mr. 
Tumnus  get  made  into  stone?”.  Since  the  story  did  not  clearly  explain  the  full  reason,  the  reader  would  need  to  make  certain 
assumptions  based  on  the  character’s  actions  in  relation  to  the  villain’s  personality  traits.  In  addition,  the  reader  would  need  to 
evaluate  certain  moral  justifications  that  were  not  explicitly  stated  in  the  story.  This  would,  more  likely,  require  thinking 
processes  for  integration  of  new  information  with  prior  experiences  and  synthesis  of  ideas  than  organization  and  sequence  of 
events. 

Writing  test  questions  with  accurate  answers  and  appropriate  distracters  would  also  help  students  evaluate  the  concepts  and 
events  in  the  story.  Since  the  test  items  were  generated  independently,  it  may  have  resulted  in  students’  greater  use  of  generative 
strategies  for  integration  of  new  ideas  with  prior  knowledge  and  across  the  various  ideas  rather  than  organization  of  events. 
Integration  strategies  increase  students’  ability  to  see  relationships  and  connections  between  ideas  and  concepts.  This  type  of 
mental  processing  would  certainly  result  in  students’  ability  to  develop  questions  for  application  of  concepts  and  synthesis  of 
themes  and  ideas  across  the  entire  book.  In  answer  to  the  first  question  in  this  study,  a qualitative  evaluation  of  students’  test 
questions  would  suggest  elementary  age  children  can  generate  test  items  consistent  with  higher  level  thinking  processes. 

A paired  sample  t test  was  used  to  measure  the  differences  in  means  between  the  pretest  and  posttest  developed  by  the 
researcher.  It  was  hoped  that  the  use  of  the  generative  strategies  would  have  a more  significant  effect  on  students’  test  scores, 
however,  differences  were  marginal  at  the  .05  level  (t  ~ 1.74,  df  ~ 19,  p < .09).  In  retrospect,  I would  set  the  significance  level  at 
.10  for  two  reasons.  First,  this  study  was  not  in  a controlled  laboratory  setting  nor  were  students  tested  for  reading  ability  and 
cognitive  capacity  prior  to  the  treatment.  Similar  to  the  study  by  Volk  and  Ritchie  (1999),  this  was  a pilot  study  with  many 
variables  that  might  interfere  with  the  results  of  students’  performance.  Second,  scores  from  matched  pairs  are  less  likely  to 
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reveal  differences  based  on  the  effect  of  the  treatment.  According  to  Popham  and  Sirotnik  (1992),  with  corresponding  scores, 
there  is  a tendency,  that  differences  in  means  are  less  likely  to  be  significantly  different. 

There  was  a significant  difference  between  the  means  from  the  pre  test  and  means  from  the  student -generated  post  test  (t  = 
5.57,  df  - 19,  p < .000).  Using  paper  and  pencil,  each  student  had  written  4 to  5 items,  and  then  entered  these  into  the 
HyperStudio®  stack  for  the  mock  AR  test.  From  this  pool  of  test  questions,  20  items  were  selected.  These  were  copied  and  pasted 
into  a word-processed  document  and  administered  to  the  students  as  a paper  and  pencil  test.  Conditions  for  the  student -generated 
posttest  were  identical  to  those  of  the  researcher’s  posttest.  T test  statistics  for  paired  samples  showed  a significant  correlation 
between  the  data  from  the  pre  test  and  the  data  from  the  posttests.  Students  who  performed  poorly  on  the  pre  test,  performed  at  a 
corresponding  level  on  both  posttests,  thus  it  is  likely  that  the  student-generated  posttest  was  reliable.  In  answer  to  question  2, 
this  would  suggest  that  the  use  of  organizational  strategies  for  generating  original  reading  comprehension  questions  does  have  a 
positive  effect  for  student  performance.  Further,  the  effect  of  the  treatment  appears  to  improve  scores  for  a criterion  referenced 
test  for  main  idea,  sequence  of  events,  fact  versus  opinion,  and  cause/effect  factors.  There  should  be  further  study  to  provide 
evidence  to  support  the  effectiveness  for  the  use  of  generative  test  questions.  This  could  be  accomplished  by  measuring  gains  in 
scores  for  this  sample  with  a comparable  group  who  did  not  receive  the  treatment.  The  answer  to  question  3 is  answered  by 
comparing  the  means  between  both  posttests.  Descriptive  statistics  show  that  students  did  perform  better  when  answering  their 
peers’  questions  than  the  items  from  the  researcher’s  test  (see  Table  3).  At  first,  this  may  appear  of  little  interest,  but  with  the 
qualitative  evaluation  of  students’  test  kerns  for  higher  thinking  processes,  their  performance  on  this  test  may  reveal  some 
positive  effects  from  the  treatment. 

Students  were  also  highly  motivated  to  develop  a test  that  would  be  taken  by  their  peers.  This  was  an  authentic  problem  in 
which  students  were  challenged  to  write  a mock  test  for  Accelerated  Reader.  Since  research  has  shown  that  students  are 
motivated  to  solve  a real-world  problem,  I was  able  to  observe  students  as  they  exhibited  an  effort  to  construct  questions  which 
they  judged  as  fair  and  would  be  important  for  measuring  reading  comprehension.  Because  of  the  earlier  experiences  with  AR 
testing,  students  were  familiar  with  the  multiple-choice  format.  Based  on  this  experience,  they  were  also  able  to  make  some 
Judgments  in  what  kinds  of  questions  measure  reading  ability. 

The  pool  of  questions  written  by  the  students  was  characteristic  of  high  level  of  thinking  processes  for  seeing  relationships 
among  ideas  presented  in  the  story.  Students  also  demonstrated  a high  level  of  evaluative  skill  for  constructing  test  questions  that 
are  suitable  for  measuring  reading  comprehension.  Although  student  performance  showed  marginal  performance  on  the  criterion- 
referenced  posttest,  students  showed  significant  improvement  when  answering  test  questions  developed  by  their  peers.  This  study 
clearly  provides  support  for  the  use  of  generative  strategies  for  higher  level  thinking  when  reading  and  interpreting  textual 
information. 
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Appendix  I 

Pretest  developed  using  framework  from  Dimensions  for  Thinking.  Available  online: 
[http://www.ceap.wcu.edu/Houghton/Leamer/Think94/NCmarzanoThink.html] 

NCDPI  adapted 
from  Marzano’s 
Dimensions  for 
Thinking 

Pre  and  Post  Test  Questions 
Generated  by  Researcher 

Knowledge 
Declarative  and 
procedural  knowledge 

Recall 

1 1 . Who  did  Lucy  meet  first? 

1 3.  What  do  the  children  do  after  Aslan  comes  back  to  life? 

Organizing 
Comparing, 
classifying,  and 
ordering  sequences 
Analyzing 
Identifying  attributes 
and  components; 
Identifying 
relationships  and 
patterns 

Sequence 

9.  What  happened  first? 

a.  Peter  discovered 

b.  Lucy  found.... 

c.  Edmund  found... 

d.  Edmund  and  Lucy  found.... 

10.  Aslan  was  victorious  .... 

a.  after  Edmund... 

b.  after  Aslan  was... 

c.  after  Peter  was.... 

d.  after  all  the  animals.... 

12.  Who  were  the  last  two  people  to  see  Aslan  alive? 

Cause  and  Effect 

14.  Edmund  loved  the  idea  of  becoming  a Prince  because 

15.  The  Professor  told  Susan  and  Peter  that  Lucy  might  be  telling  the  truth  because.... 

16.  The  Battle  between  Aslan  and  the  Witch  would  have  been  lost  except 

8.  Lucy  was  very  worried  about  Mr.  Tumnus  because 

Main  Idea 

1.  Which  of  the  following  best  describes  Aslan? 

2.  Which  of  the  following  best  describes  the  Witch? 

3.  Which  of  the  following  best  describes  Edmond? 

Applying 

Demonstration  of  prior 
knowledge  within  a 
new  situation 
Evaluating 

Main  Idea 

4.  At  the  beginning  of  the  story,  Edmund  pretended  that  he  had  not  really  found  Narnia  and  was  making 
the  story  up  because...? 

7.  After  Edmund  had  been  rescued  by  Aslan  he  felt 

Confirming  or  proving 
the  truth  of  an  idea, 

Fact  versus  Opinion 

17.  The  Witch  was  wicked  and  lonely 

18.  Aslan  was  good  and  brave 

19.  In  the  end,  Edmund  was  a hero 

20.  Turkish  Delight  is  delicious 

Generating 

Inferring,  Predicting  & 
Elaborating 
Integrating 
Summarizing  and 
restructuring; 
identifying  important 
components  to 
generate  a cohesive 
thought. 

Main  Idea 

5.  Why  did  Edmund  have  to  die  at  the  hand  of  the  Witch? 

6.  Why  did  Aslan  die? 

7.  After  Edmund  had  been  rescued  by  Aslan  he  felt 
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Appendix  II 

Comparison  of  researcher-test  with 
student-test  for  match  with  higher  levels 
of  thinking. 

Note:  certain  test  items 
may  match  more  than 
one  category. 

NCDPI  adapted  from 
Marzano’s  Dimensions  for 
Thinking 

Post  Test  Questions 
Generated  by  4^**  Grade  Students 

Dimensions  for 
Thinking  category 

Number  of  items 
from 

researcher’s  test 
that  match 
category  (see 
Table  2) 

Number  of  items 
from  students’  test 
that  match  category 

Knowledge 

1 . How  did  Lucy  discover  Narnia? 

2.  Who  was  Aslan? 

5.  The  White  Witch  told  Edmund  he 
would  become  a when  she  was 

gone.... 

10.  What  did  Father  Christmas  give  Peter? 
16.  What  game  were  the  kids  playing 
when  they  found  the  wardrobe? 

18.  Where  did  the  Professor  live? 

1 9.  What  food  did  Mr.  Tumnus  NOT  give 
Lucy  in  his  home? 

2 1 . How  many  reindeer  did  the  White 
Witch  have  on  her  sled? 

Knowledge 

2 

8 

Organizing 

Analyzing 

3.  When  Lucy  and  Susan  followed  Aslan, 

they  followed  him  to 

6.  The  first  kid  who  knew  about  Narnia 
was.... 

a.  the  youngest  in  the  family 

b.  the  last  to  know  about  Narnia 

c.  the  meanest  kid  in  the  family 

d.  the  oldest  kid  in  the  family 

8.  When  Lucy  went  to  Narnia,  who  did 
she  first  meet? 

1 5.  Did  Edmund  get  to  eat  his  favorite 
food  when  he  got  to  the  Witch’s  house? 

Organizing 

Analyzing 

19 

4 

Applying 

Evaluating 

7.  Why  did  Aslan  die? 

9.  Why  did  Mr.  Tumnus  get  made  into 
stone? 

1 1 . Why  did  Mr.  Tumnus  ask  Lucy  if  she 
was  a daughter  of  Eve? 

12.  Why  does  Edmund  leave  the  Beaver’s 
house? 

13.  What  does  Edmund  want  from  the 
White  Witch? 

26.  Why  did  the  Beavers  say,  “Be  quiet!”? 

Applying 

Evaluating 

6 

6 

• Generating 

• Integrating 

14.  Why  didn’t  Edmund  tell  the  others  he 
had  found  Narnia  just  like  Lucy  did? 

1 7.  How  was  Aslan  brought  back  to  life? 

Generating 

Integrating 

3 

2 
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The  Technology  Teaching  Lab:  Meeting  the  ISTE  Standards 


Terri  Teal  Bucci 
Ohio  State  University 


Abstract 

The  technology  teaching  lab  program  is  a series  of  2-hour  labs  that  runs  concurrently  with  preservice  methods  blocks.  The 
purpose  of  the  lab  is  to  give  the  students  the  experiences  necessary  to  integrate  technology  into  their  classroom.  The  lab 
provides  the  students  with  instruction,  opportunities,  and  equipment  to  take  their  technology-enhanced  lessons  directly  to  the 
field. 

This  research  found  that  given  time,  technology,  assistance,  and  experience,  students  could  create  technology-enhanced 
lessons.  The  implementation  of  the  technology  teaching  lab.  connected  with  the  profile  template,  provides  preservice  teachers 
with  the  structure  and  opportunities  to  meet  the  profile  goals  set  by  ISTE.  Finally,  the  technology  teaching  lab  and  the 
accompanying  template  provides  for  opportunity  for  reflection  and  demonstration  of  best  works,  similar  to  a portfolio.  The 
benefits  from  this  program  are  visible,  follow  from  the  collected  data,  and  provide  for  opportunities  to  infuse  technology  into  the 
preservice  teacher  education  program  and  expectations. 

Problem 

According  to  the  U.S.  Department  of  Education,  only  20%  of  the  2.5  million  teachers  currently  working  in  our  public  schools 
feel  comfortable  using  the  technologies  available  to  them  (NCES,  1999).  This  is  a tragedy,  but  one  that  is  remedied  by  providing 
students  with  a true  working  knowledge  of  current  educational  technologies  and  opportunities  and  experiences  to  integrate  those 
technologies  into  their  classrooms.  Many  colleges  of  education  have  found  ways  to  increase  students'  knowledge  of  technology 
through  independent  courses.  It  is  reported  by  the  Office  of  Technology  Assessment  (1995)  that  much  of  technology  instruction 
is  related  to  the  teaching  of  technology  instead  of  teaching  with  technology  to  enrich  curriculum  (Duhaney,  2001).  Nevertheless, 
institutions  should  teach  teachers  how  to  use  technology  to  support  multiple  content  curriculums  (Ingram,  1994).  Unfortunately, 
there  has  been  little  done  in  the  area  of  experiences  of  technologies  in  the  classrooms.  Because  of  the  importance  of  experiences 
with  technology  in  the  field,  it  is  the  goal  of  many  departments  of  education  to  include  clinical  experiences  (Duffield,  1997). 

Currently,  the  education  department  at  Ohio  State  University  at  Mansfield  is  trying  to  incorporate  technology  into  our  courses 
by  giving  assignments  that  encourage  the  use  of  technology.  Unfortunately,  there  is  no  time  given  to  students  to  experiment  or 
develop  the  technology -enhanced  lessons.  The  methods  courses  are  full  of  methods  content.  Nevertheless,  it  is  vital  to  give 
students  experiences  both  in  teaching  with  technology  and  participating  as  a student  using  technology  (Cuba,  1995).  Because  of 
this,  our  department  makes  every  attempt  to  enhance  our  teaching  with  technology.  This  happens  through  required  presentations, 
including  the  input  of  digital  images,  expectations  of  technology  for  required  lesson  development  and  assessments,  the  use  of 
digital  picture  displays  to  enhance  an  activity,  and  the  use  of  Web  CT.  Through  the  development  of  these  and  other  uses  of 
technology  in  our  methods  courses,  we  are  beginning  to  give  our  students  experiences  from  a student  perspective.  In  addition  to 
this,  though,  there  must  be  a teaching  component  to  technology  experiences.  This  happens  with  the  development  of  the 
technology  teaching  lab. 

Research  Questions 

This  is  a qualitative  ethnographic  study  in  case  format  as  defined  by  Cuba  and  Lincoln  (1994).  It  is  because  of  this 
methodology  that  issues  of  description  and  interpretation  are  of  utmost  concern.  The  questions  informing  the  interpretation  of 
data  in  this  study  have  the  following  foci: 

■ Determine  whether  a student  can  or  cannot  reach  the  expectations  of  the  ISTE  Profile  Standards  for 
Professional  Performance  through  the  implementation  of  the  technology  teaching  lab. 

■ Determine  types  of  technology  infusion  used  to  meet  these  standards. 

■ Determine  the  perceived  ability  of  the  students  following  the  technology  teaching  lab. 

The  Technology  Teaching  Lab 

Before  the  Technology  Teaching  Lab,  if  a student  wished  to  use  technology  in  her  field  placement  Ohio  State  University  at 
Mansfield,  not  only  did  she  have  to  plan,  create,  and  write  the  lesson  in  her  own  time  in  addition  to  the  other  lessons  from  the 
content  areas,  but  she  had  to  depend  on  the  technologies  of  the  school  in  which  she  is  placed.  Often  times,  the  schools  would 
have  little  equipment  or  equipment  that  is  not  compatible  with  the  developed  lesson. 

The  Lab  Course 

The  goal  for  this  program  is  to  increase  the  use  of  technology  in  our  students’  lessons  in  ways  that  will  enhance  their 
teaching.  The  purpose  of  this  lab  is  to  provide  the  preservice  teachers  in  our  elementary  education  program  opportunities  and 
assistance  in  creating  and  using  technology -enhanced  lessons  into  their  field  placements.  The  Technology  Teaching  Lab 
component  is  a 2-credit  course  scheduled  for  two  quarters  to  run  simultaneously  with  our  methods  blocks.  Connections  and 
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constructivist  theoiy  are  two  foundations  to  our  program.  Any  new  addition  to  our  program  needed  to  address  both  core  areas. 
Because  of  our  strong  commitment  to  the  integrated  approach  to  teaching,  our  technology  component  needed  to  connect  to  our 
methods  blocks.  Our  methods  blocks  run  in  two  consecutive  quarters  to  include  a methods  course  from  social  studies,  math, 
language  arts,  and  science.  We  addressed  the  issue  of  connection  by  including  requirements  for  technology  -enhanced  lessons  in 
each  of  our  syllabi.  Within  each  content  area,  there  are  expectations  for  lesson  development  and  integration  within  those  lessons. 
We  expect  our  students  to  write  their  lessons  in  a unit:  integrating  content  as  much  as  possible,  when  there  is  a natural  fit.  By 
connecting  the  technology  teaching  lab  with  the  content  methods  courses,  we  provide  our  students  with  multiple  ways  to 
integrate  technology  into  each  content  area  and  thus  integrate  the  content  areas  through  technology.  Each  methods  course  has  an 
expectation  of  technology  integration  into  the  lessons. 

The  lab  course  meets  once  every  week  during  the  same  quarters  as  our  methods  block  for  a two-hour  period.  There  are  two 
time  slots  available  to  better  meet  the  needs  of  the  students:  one  after  school  on  class  days  and  one  after  school  on  their  field 
placement  days.  In  addition,  the  lab  space  is  available  for  walk-ins  throughout  the  week.  The  students  work  during  this  time  to 
create  technology -enhanced  lessons  that  they  will  take  directly  to  their  field  placement.  There  is  little  direct  instruction;  instead, 
time  is  spent  on  the  uses  of  instructional  technologies,  demonstrations  of  those  uses,  and  play  with  the  equipment.  The  primary 
structure  of  the  lab  course  is  open  and  one  of  discovery  and  experience.  Students  are  to  play  and  create  lessons,  again,  to  take 
directly  to  the  field. 

Students  write  technology  -enhanced  lessons  and  use  educational  technology  in  their  field  placements.  This  important  facet 
(that  of  experience)  of  the  lab  course  gives  students  the  practice  in  developing  and  revising  educationally  sound,  technology - 
enhanced  lessons  for  their  future  classrooms.  The  lab  provides  students  supervised  time  to  experiment  with  the  hardware  and 
software  to  create  technology -enhanced  lessons  that  connect  to  the  requirements  cf  the  methods  courses.  These  technology - 
enhanced  lessons  incorporate  imaging,  Internet  use,  and  presentation  tools.  Then,  the  students  take  the  created  lessons  directly  to 
the  field. 

The  Lab  Instructor 

We  addressed  our  other  area  of  focus,  constructivist-learning  philosophies,  through  the  format  of  the  lab  course.  The  natural 
starting  point  for  instruction  in  a constructivist  classroom  is  not  the  material  to  be  taught,  but  student  interests,  prior  experiences, 
and  current  understandings  (Ravitz,  Becker,  & Wong,  2000).  Because  of  this,  a true  constructivist  form  of  a Technology 
Teaching  Lab  would  have  to  accommodate  for  a variety  of  levels  of  technology  abilities  in  the  students  and  provide  for  their 
varying  interests.  We  designed  the  Technology  Teaching  Lab  course  to  be  one  of  discovery  and  experience.  The  purpose  of  the 
lab  is  to  provide  our  students  with  the  opportunities  to  develop  appropriate  uses  of  the  technologies  in  their  field  placements  and 
to  then  take  those  lessons  directly  to  the  field,  gving  the  students  the  experiences  necessary  to  integrate  technology  into  their 
classroom. 

The  teacher's  role  in  a constructivist  setting  is  to  facilitate  student -designed  efforts.  This  instructor  is  also  available  in  the 
physical  space  of  the  lab  to  assist  students.  The  major  focus  of  the  instructor’s  time  is  on  play  and  on  emergent  needs  of 
particular  lesson  creations.  The  instructor’s  responsibility  is  to  help  the  students  develop  educationally  sound  applications  of 
technology  in  their  field  placement  in  close  connection  to  the  education  department  and  the  needs  and  requirements  of  the 
methods  courses.  The  instructor  is  also  available  in  the  physical  space  of  the  lab  to  assist  students. 

Equipment 

We  purchased  a variety  of  technology  tools  that  tour  students  could  take  directly  to  their  field  placements.  These  tools 
included:  portable  laptop  and  projector  sets,  flex  cams,  digital  microscopes,  computer  calculator  sets,  a variety  of  canned 
software,  digital  cameras,  and  digital  video  cameras.  By  providing  the  students  with  the  equipment,  the  students  can  create  a 
lesson  and  deliver  the  lesson  directly  without  concern  for  lack  of  hardware,  software,  or  hardware  mismatches. 

The  Assessment  tool 

As  a department,  we  have  adopted  the  ISTE  standards  (International  Society  for  Technology  in  Education,  2000)  for 
technology  in  preservice  education.  As  an  evaluation  piece,  I created  a template  out  of  those  competencies  specifically  set  to  the 
professional  preparation  performance  profile  created  for  pre-service  teachers  to  be  completed  before  their  internship  experience 
(ISTE,  2000).  Each  of  our  students  downloaded  this  template,  the  profile  in  table  form,  onto  a zip  disc.  Our  students  wrote  a 
short  narrative  addressing  how  they  hit  each  of  the  competencies  and  included  with  this  narrative  a hyperlink  to  electronic 
evidence  of  their  work.  This  evidence  could  be  in  a variety  of  forms.  For  example,  if  a student  wanted  to  demonstrate  that  she 
wrote  and  taught  a lesson  using  Hyper  Studio,  she  might  use  two  forms  of  evidence.  First,  she  might  hyperlink  the  lesson  portion 
to  her  actual  text  document  write-up  of  the  lesson.  Second,  she  might  hyperlink  her  teaching  evidence  to  an  example  of  a 
student's  presentation.  In  doing  this,  the  student  has  demonstrated  her  capabilities  to  write  and  teach  a technologically  enhanced 
lesson,  and  demonstrated  her  ability  to  use  technology  as  a form  of  self-evaluation.  She  is  documenting  her  technology  use  and  at 
a future  time,  can  reflect  on  that  use  and  revise  and  recreate.  Another  example  might  be  that  of  email  threaded  discussions 
maintained  throughout  a field  assignment.  The  student  teacher  could  simply  retain  a copy  of  the  discussions  and  hyperlink  them 
to  the  template  to  use  as  electronic  documentation.  A sample  portion  of  template  is  provided  in  table  1. 
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Table  L Template  Sample 


Prior  to  the  culminating  student  teaching  or 

internship  experience 

Technology  Operations  and  Concepts 

Task 

Electronic  Evidence 

Examine  technology  tools  used  to  collect, 
analyze,  interpret,  represent,  and 
communicate  student  performance  data. 

“Free  online  grade  book  software.” 

http://www.classbuilder.co 

m 

Planning  and  Designing  Learning  Environments  and  Experiences 

Identify  technology  resources  available  in 
schools  and  analyze  how  accessibility  to 
those  resources  affects  planning  for 
instruction. 

“Taught  an  integrated  lesson  which 
included  an  Excel  graphing  exercise 
in  the  computer  lab.  Twenty-three 
students  in  class.” 

bellvillelesson  1 7.doc 

Design  and  teach  technology -enriched 
learning  activities  that  connect  content 
standards  with  student  technology  standards 
and  meet  the  divers  needs  of  students. 

“An  example  of  a student-produced 
graph  from  an  integrated  technology 
lesson  I taught  in  the  computer  lab 
at  Bellville  Elementary.” 

Cody.xls 

Teaching,  Learning,  and  the  Curriculum 

Apply  on-line  and  other  technology  resources 
to  support  problem  solving  and  related 
decision  making  for  maximum  student 
learning. 

“Incorporate  higher- level  thinking 
problems  and  questions  withNCTM 
math-related  activities.” 

http://standards.nctm.org/do 

cument/eexamples/index.ht 

m 

Productivity  and  Professional  Practice 

Identify  and  engage  in  technology -based 
opportunities  for  professional  education  and 
lifelong  learning,  including  the  use  of 
distance  learning. 

“Opportunities  for  professional 
development  and  resources.” 

http://www.nctm.org 

htto://www.ohioschoolnet.k 

12.oh.us/ 

Social,  Ethical,  Legal,  and  Human  Issues 

Identify  issues  related  to  equitable  access  to 
technology  in  school,  community,  and  home 
environments. 

“Equity  project  WI  ‘01” 

EquityWIOl.ppt 

Methodology 

Purpose  of  Study 

The  purpose  of  this  study  is  multifaceted:  first,  to  determine  whether  or  not  students  can  meet  the  expectations  of  the 
professional  preparation  standards  set  ty  the  ISTE  standards  can  be  met  by  the  implementation  of  the  technology  teaching  lab 
and  second,  to  determine  what  level  of  technology  infusion  students  choose  to  use  in  meeting  these  standards  and  finally  to  guide 
development  of  the  technology  teaching  lab.  The  results  of  this  study  will  be  beneficial  to  other  departments  of  education  in  their 
drive  to  meet  national  standards  and  infuse  technology  into  their  programs. 

Design,  Instrumentation,  and  Data  Analysis 

The  design  of  this  study  utilized  qualitative  research  methods  from  the  interpretivist  paradigm  (Cuba  & Lincoln,  1994).  This 
is  a case-study  to  represent  a molar  unit,  a multiple  of  individuals  (preservice  teachers).  The  use  of  a molar  unit  in  this  case  is  to 
extend  external  validity  of  results  (Huberman  & Miles,  1994).  The  case  format  is  not  one  of  generlizability;  it  instead  has  a focus 
of  transferability.  This  transfer  is  to  similar  sets  of  participants.  Data  were  gathered  from  multiple  sources  during  the  2000-2001 
school  year. 

The  data  was  collected  in  two  formats.  First,  the  ISTE  profile  template  (as  defined  previously  as  the  tool  for  the  course).  The 
data  from  the  template  was  inserted  into  a database  for  ease  of  analysis.  The  database  was  analyzed  in  a variety  of  manners. 
First,  the  database  was  used  to  determine  what  the  students  used  as  an  electronic  example  for  their  meeting  a particular  standard. 
Second,  the  variety  of  evidence  used  to  document  to  student’s  meeting  the  standards.  Finally,  the  database  was  used  to  interpret 
the  level  of  technology  infusion. 

In  addition  to  the  database,  a survey  was  distributed  at  the  end  of  the  second  quarter  in  which  the  students  took  the  lab.  This 
survey  asked  questions  about  the  student’s  perceptions  of  their  technology  abilities,  their  anticipated  use  of  technology  in  the 
classroom,  and  their  impressions  of  the  technology  teaching  lab.  Open-ended  comments  listing  strengths  and  concerns  were 
coded  by  emergent  topics.  A cluster  method  of  data  analysis  was  used  throughout  the  interpretation  of  data  (Huberman  & Miles, 
1994). 


Description  of  site  and  participants 

This  study  took  place  at  a regional  campus  of  a large,  mid-west  University.  The  participants  for  this  study  were  2 1 Masters 
of  Education  students.  Their  ages  ranged  from  23-43,  four  male;  17  female,  20  Caucasian;  1 Asian-American,  and  from  middle- 
class  background.  Students  came  to  the  masters  degree  certification  program  with  a variety  of  undergraduate  degrees:  elementary 
education,  psychology,  law,  the  ministry,  and  private  business.  All  students  were  required  to  take  the  2-credit  hour  technology 
teaching  lab  course  with  each  of  their  methods  block  quarters  (totaling  4 credit  hours).  This  was  the  first  year  of  this 
requirement. 

Results 

Demonstration  of  Technology  Use 

At  the  end  of  the  second  lab  course,  students  completed  their  templates  with  a narrative  stating  how  they  met  each  standard 
and  a hyperlink  connecting  to  a piece  of  electronic  evidence  that  supported  their  statement.  Table  2 shows  the  distribution  of 
electronic  evidence  used  by  the  students  in  their  templates.  The  data  is  heavy  in  the  areas  of  web  sites  and  presentations.  The  use 
of  web  sites  was  intriguing  and  warranted  further  investigation  into  the  manner  by  which  they  used  the  sites  as  documentation  of 
meeting  a particular  standard.  The  manner  in  which  the  sites  were  used  in  denoted  in  the  second  category  of  results,  level  of 
evidence.  The  use  of  presentations  as  a demonstration  of  evidence  stems  from  the  assignments  given  in  the  methods  block 
courses.  In  the  2002-2001  school  year,  the  year  of  this  study,  the  students  were  required  to  produce  a presentation  in  the  math 
and  science  methods  block  courses  in  which  they  took  digital  video  of  an  integrated  lesson,  imported  portions  of  their  video  into 
a power  point  presentation,  and  reflected  upon  the  concerns  and  strengths  of  the  lesson.  This  project  was  called  the  MST  project 
and  each  student  created  two  presentations  in  this  format;  one  in  the  first  methods  block  and  another  in  the  second  methods  block. 

Table  2.  Evidence  Used 


Web  Sites 

Documents 

Excel 

Presentations 

Other 

Technology  Operations  and  Concepts 

19 

1 

1 

1 

18 

Planning  and  Designing  Learning  Environments  and 
Experiences 

40 

22 

2 

71 

3 

Teaching,  Learning,  and  the  Curriculum 

48 

11 

1 

Assessment  and  Evaluation 

48 

20 

10 

Productivity  and  Professional  Practice 

20 

4 

2 

14 

Social,  Ethical,  Legal,  and  Human  Issues 

92 

2 

2 

Total 

267 

38 

5 

109 

33 

Percentage  of  profile  standards  met 

59% 

9% 

1% 

24% 

7% 

Level  of  Technology  Evidence 

In  addition  to  the  types  of  technology  used  by  the  students  as  evidence  of  meeting  the  profile,  the  data  was  analyzed  for  level 
of  technology  integration.  In  this  analysis,  a clustering  and  coding  method  was  implemented.  This  coding  produced  three  major 
categories  of  technology  infusing:  teacher-centered  technology,  child-centered  technology,  and  task-centered  technology. 
Teacher-centered  technology  refers  to  technology  used  by  the  teacher  but  in  the  context  of  a lesson.  In  this  case,  the  teacher  is 
the  worker.  It  would  be  similar  to  the  teacher  using  technology  to  enhance  a lesson  that  would  otherwise  be  teacher  directed.  An 
example  of  teacher-centered  technology  would  be  a teacher-created  presentation  using  power  point  or  HyperStudio  to 
demonstrate  the  development  of  a seed.  Another  example  of  teacher-center  technology  would  be  a class-created  excel  chart  in 
which  the  teacher  input  the  data  on  a centered  machine  that  is  connected  to  a projector  or  large  screen  television.  Another 
example  of  a teacher-center  use  would  be  the  teacher  using  some  sort  of  project  device  to  demonstrate  the  components  of  a seed 
using  a flex  cam.  Still  another  example  of  a teacher-center  infusion  would  be  a class  game  using  a canned  CD  and  a projection 
device. 

The  second  category  for  technology  infusion  determined  by  analyzing  the  data  from  the  templates  is  child-centered 
technology.  This  format  refers  to  technology  used  by  the  children  in  the  class,  either  in  small  groups  or  individually.  In  this 
case,  the  child  is  the  worker.  This  would  be  similar  to  the  teacher  using  technology  to  enhance  a small  group  task  or  individual 
seat  work  assignment.  An  example  of  child-centered  technology  would  be  child  creating  a publishing  document  that  focused  on 
a particular  country  to  be  used  as  an  assessment  in  a class.  Another  example  of  this  format  would  be  a group  of  students  creating 
a presentation  documenting  the  recycling  of  garbage.  Another  form  of  child- centered  technology  would  be  a student  writing  a 
story  and  importing  still  photos  to  illustrate  the  story.  Still  another  form  of  child-centered  technology  would  be  the  use  of 
Internet  interactive  tools  with  each  child  at  a computer  station  (ex.  E-examples  from  NCTM). 

The  third  category  for  technology  infusion  determined  by  the  template  is  task-center  technology.  This  format  refers  to 
technology  used  by  the  teacher  in  the  design  of  a lesson,  a class,  or  profession  growth  or  organization.  In  this  case,  the  task  is  the 
focus  with  the  teacher  as  the  worker.  This  would  be  aligned  to  traditional  preparation  and  paperwork  connected  to  the  profession 
of  teaching.  An  example  of  task-centered  technology  would  be  a web  site  used  to  obtain  information  about  a lesson  on  the 
circulatory  system.  Another  example  would  be  the  creation  of  an  excel  chart  or  word  “chat”  or  email  format  that  demonstrated 


51 


60 


collaboration  on  issues  of  teaching.  Still  another  example  would  the  completion  of  an  assignment  for  a course  in  education  (in 
document,  sheet,  or  presentation  format). 

Table  3 indicates  the  various  forms  of  technology  evidence  used  by  the  preservice  teachers  to  demonstrate  competence  in  the 
performance  indicators.  Special  note  is  made  to  the  performance  indicator  goal,  which  in  many  cases  does  not  require  lesson 
development,  but  rather  technology  use  in  the  profession  of  teaching. 

Table  3.  Narrative  Coding 


Teacher- 

Child- 

Task- 

center 

centered 

centered 

technology 

technology 

technology 

Technology  Operations  and  Concepts 

0 

0 

54(100%) 

Planning  and  Designing  Learning  Environments  and 

Experiences 

12  (7%) 

67  (38%) 

96  (55%) 

Teaching,  Learning,  and  the  Curriculum 

0 

4 (6%) 

60  (94%) 

Assessment  and  Evaluation 

1 (1%) 

12(12%) 

87  (87%) 

Productivity  and  Professional  Practice 

0 

0 

43  (100%) 

Social,  Ethical,  Legal,  and  Human  Issues 

0 

0 

104(100%) 

Total 

13 

83 

444 

Percentage  of  profile 

3% 

15% 

82% 

Student  Perceptions 

Secondary  to  the  template,  was  the  survey.  This  survey  was  administered  for  the  purposes  of  course  development. 
Nevertheless,  the  survey  indicates  a level  of  ability  and  comfort  that  could  inform  other  institutions  where  technology  infusion 
programs  are  being  developed  and  under  consideration.  Table  4 indicates  the  technologies  that  student  would  most  likely  use  in 
their  teaching.  This  reflects  technologies  that  were  emphasized  during  the  technology  teaching  lab  and  those  technologies  that 
were  demonstrated  during  the  technology  teaching  lab. 

Table  4.  Technology  most4ikely  to  use  in  teaching 


Hardware 

Raw 

score* 

Rank 

Raw 

score* 

Software 

Digital  camera 

26 

1 

22 

Word  processing 

Flatbed  scanner 

57 

2 

43 

Internet 

Digital  video  camera 

64 

3 

75 

Power  point 

Flex  cam 

71 

4 

83 

HyperStudio 

Multimedia  projector 

94 

5 

93 

Educational  programs 

Dissecting  microscope 

100 

6 

104 

Spreadsheet 

Digital  balance 

113 

7 

111 

Database 

* Combined  rating  of  20  students  with 

a rating  of  1 

. as  most  likely. 

Student  Perceptions  of  Technology  Teaching  Lab 

An  open-ended  comment  section  of  the  survey  revealed  what  the  students  thought  were  concerns  and  strengths  of  the 
technology  teaching  lab.  These  concerns  could  be  categorized  into  three  subheadings:  Instructor,  time,  technology,  and  ability. 
Interestingly,  the  instructor  and  ability  were  indicated  and  coded  as  major  categories  for  both  the  concerns  and  the  strengths  of 
the  technology  teaching  lab.  The  instructor  was  commended  on  several  surveys  for  being  available,  experienced,  and  flexible. 
An  example  of  this  was,  "Instructor  was  flexible,  helpful,  patient,  personable,  kept  the  class  'real'".  Alternatively,  the  instructor 
was  also  listed  as  a concern  for  a lack  of  expertness,  and  not  conducting  the  class  on  an  individual  level.  Ability  was  an  indicator 
of  strength;  "Class  is  needed,  gained  an  enormous  amount,  a lot  further  in  my  ability  to  use  technology,  I learned  a lot,  I feel 
capable,"  and  a concern;  "...overkill,  quite  comfortable  previous  to  this  class,  lab  times  inconvenient,  class  needs  to  be  'stepped 
up',  should  have  been  much  more  help,  start  classes  in  fall,  need  better  connection  to  methods  courses." 

One  of  the  categories  that  did  not  show  through  in  both  strengths  and  concerns  was  that  of  time.  Time  was  indicated  as  a 
strength  because  students  felt  that  they  were  given  an  extended  period  of  time  to  actually  write  technology -enhanced  lessons; 
"...lots  of  time  to  work  on  our  technology  components  of  projects,  more  time  to  use  technology  to  plan  lessons."  Technology  was 
listed  as  a concern,  as  it  often  is  in  the  cases  of  infusing  technology  into  the  classroom;  "...technology  fail  (ures),  better 
connection  to  printer,  (need)  better  equipment" 


Discussion 

The  technology  teaching  lab,  while  still  in  its  infancy,  shows  great  promise.  It  provided  a venue  for  preservice  teachers  to 
address  issues  of  technology  connected  to  their  field  placements  and  to  their  classroom  assignments.  It  demonstrated  a 
connection  to  the  questions  stated  earlier  in  this  writing. 


The  Technology  Teaching  Lab  and  the  ISTE  Standards 

It  follows  that  students  can  meet  the  profile  set  by  ISTE  Profile  Standards  for  Professional  Performance  through  the 
implementation  of  the  technology  teaching  lab  and  courses  similar  in  focus.  The  students  used  a variety  of  electronic  evidence 
but  the  primary  tool  was  that  of  web  sites.  At  first  glance,  this  might  be  disturbing.  However,  this  indicates  that  students  find  the 
tool  of  the  computer  useful  in  the  research  and  information  gathering  aspect  of  teaching  and  learning.  When  reading  the  data,  it 
was  necessary  to  realize  that  the  ISTE  profile  contains  six  categories:  Technology  Operations  and  Concepts,  Planning  and 
Designing  Learning  Environments,  Teaching,  Learning,  and  the  Curriculum,  Assessment  and  Evaluation,  Productivity  and 
Professional  Practice,  and  Social,  Ethical,  Legal,  and  Human  Issues.  The  majority  of  the  six  categories  focus  on  technology 
issues  and  not  technology  as  a tool  of  instruction.  Under  the  heading  of  Planning  and  Designing  Learning  Environments, 
Teaching,  Learning,  and  the  Curriculum,  and  Assessment  and  Evaluation,  it  would  be  reasonable  to  find  students  using  a variety 
of  electronic  evidence  indicators.  It  may  be  advantageous  for  instructors  to  suggest  types  of  evidence  that  would  meet  particular 
goals,  but  I hesitate  to  say  this  in  light  of  the  focus  of  constructive  methods  for  the  class.  By  suggesting  a specific  piece  of 
evidence,  it  may  limit  the  preservice  teacher’s  creative  role  in  this  venture.  Still,  a wider  variety  of  evidential  components  would 
be  preferable. 


Technology  Infusion:  Types  and  Level 

By  categorizing  the  evidential  narratives  given  by  the  students,  it  was  clear  that  there  are  three  major  roles  of  technology  in 
preservice  teacher  education:  teach-center  technology,  child-centered  technology,  and  task-center  technology.  Again,  looking  at 
the  profile  stated,  the  issues  that  students  needed  to  address  were  various.  Many  students  met  professional  profile  expectations 
using  appropriate  manners.  The  students  who  used  child-center  technology  used  it  under  headings  were  that  was  appropriate: 
Planning  and  Designing  Learning  Environments  and  Experiences  and  Assessment  and  Evaluation.  Again,  at  first  glance,  one 
might  wish  to  see  more  teacher-centered  and  child-center  technologies  indicated  in  the  narrative  section  of  the  template. 
However,  the  fact  that  the  students  used  a variety  of  teacher,  child,  and  task  centered  activities  is  encouraging.  It  is  vital  to  use  a 
variety  of  methods  of  technology  infusion  into  teaching.  Just  as  it  is  vital  to  use  a variety  of  teaching  methods  in  teaching.  The 
fact  that  the  preservice  teachers  used  many  task-centered  narratives  indicates  the 

place  they  are  in  their  program  (using  technology  to  complete  assignment  tasks)  and  their  developmental  stage  in  teaching  (initial 
play  and  discovery). 

Student  Perceptions 

It  is  reasonable  for  the  students  to  wish  to  use  technologies  in  their  teaching  that  they  have  familiarity  with  and  that  they  have 
had  success  with  in  the  past.  This  is  clear  in  their  indication  that  word  processing,  digital  cameras,  and  presentations  would  be  of 
interest  to  them  in  their  teaching.  The  students  in  this  program  were  expected  to  use  these  technologies  throughout  the  methods 
blocks  and  chose  to  use  them  in  their  field  placements.  They  are  very  familiar  with  presentation  software  through  class 
assignment  in  their  methods  blocks.  It  is  a goal  to  get  them  to  feel  more  comfortable  with  the  uses  of  other  technologies:  flex 
cams,  computer  calculators,  digital  imaging,  and  projection  devises.  This  will  happen  through  the  emphasis  of  these  technologies 
in  their  methods  blocks  to  give  them  the  student  perspective.  This  experience  then  will  give  students  a feeling  of  comfort  and 
success  that  they  can  then  transfer  to  their  teaching.  It  is  also  vital  for  preservice  teachers  to  see  how  professors  use  technology 
to  enhance  their  teaching.  Through  this,  the  process  of  teaching  with  technology  is  modeled  and  can  then  be  applied  to  their 
teaching. 

Modifications  for  the  Technology  Teaching  Lab 

There  are  a few  considerations  to  make  to  modify  the  technology  teaching  lab  to  better  meet  the  needs  of  students  and  the 
expectations  of  the  ISTE  standards.  One  issue  that  arose  through  the  data  was  that  of  instructor.  While  the  original  intend  of  the 
technology  teaching  lab  was  to  create  a block  of  time  where  students  could  play  and  invent,  the  instructor  relied  on  a more 
instruction-based  format.  This  created  problems  for  students  who  felt  that  they  did  not  need  the  instructional  time  on  particular 
technologies  and  for  the  students  whom  desperately  need  the  “play”  time  to  accommodate  their  learning  style.  The  instructor  has 
since  modified  his  instructional  method  to  better  meet  the  constructivist,  discovery  format  and  preliminary  lesson  submissions 
from  the  current  cohort  of  students  shows  an  increase  in  technology  infused  lessons.  This  demonstrates  a need  for  a particular 
teacher  and  learner  paradigm  distinction  to  best  accommodate  the  technology  teaching  lab  and  its  intent  to  foster  technology - 
infused  lesson  development  in  preservice  teachers’  lessons. 

Another  aspect  of  modification  is  that  of  classroom  experience.  Our  students  are  benefiting  from  an  increased  role  of 
technology  in  their  methods  block  course.  This  increase  gives  the  students  opportunities  to  see  technology  used  in  instruction 
from  a students’  perspective.  Some  of  the  additional  and  continuing  aspects  of  technology  infusing  in  the  methods  courses 


include:  Computer  Poster  Sessions,  Computer  Calculators,  Flex  Cam  and  GPS  demonstrations,  electronic  backyard  History 
Projects,  Integrated  Assignments  including  the  MST  Projects  (Math/Science),  Equity  Project  (Math/Social  Studies),  and  Drama 
Display  (Math/Language  Arts). 

Finally,  the  modification  connected  to  the  particular  technologies  used  in  the  methods  blocks  and  in  the  technology  teaching 
lab  need  to  be  consistently  revisited.  Technologies  available  to  teachers  in  the  schools  are  ever*changing  and  need  continual 
modifications. 


Consideration  for  future  research 

Currently,  students  in  the  preservice  program  are  required  to  take  an  inventory  of  abilities  and  experiences  connected  with  the 
ISTE  profile.  Students  take  a 2-part  survey  in  which  they  self-report  experiences  with  technologies  and  ability  with  technologies 
connected  to  the  profile  subsections:  Technology  Operations  and  Concepts,  Planning  and  Designing  Learning  Environments, 
Teaching,  Learning,  and  the  Curriculum,  Assessment  and  Evaluation,  Productivity  and  Professional  Practice,  and  Social,  Ethical, 
Legal,  and  Human  Issues.  This  survey  will  demonstrate  growth  incurred  by  the  implementation  of  the  technology  teaching  lab. 
Additional  data  is  also  being  collected  for  future  research  connected  with  the  students’  lesson  plans.  Future  research  will  not 
only  use  data  from  the  template  and  the  survey,  but  also  coding  from  the  submitted  lessons  and  lesson  reflections  that  include 
technology  as  a component. 

It  would  be  a benefit  to  this  research  and  to  research  on  technology  in  teacher  education  in  general  if  follow-up  studies  were 
conducted  to  connect  preservice  teachers’  indications  of  use  with  actual  use,  observed  and  reported  after  the  teachers  are 
employed  full-time.  Such  research  would  add  to  the  field  of  technology  infusion  in  teacher  education. 


Conclusion 

In  conclusion,  this  report  finds  that  given  time,  technology,  assistance,  and  experience,  students  can  and  will  create 
technology -enhanced  lessons.  These  experiences  of  creating  and  field-testing  lessons  with  the  assistance  of  technology  or  with 
the  inclusion  of  technology  will  aid  students  in  the  task  of  meeting  the  ISTE  professional  preparation  performance  profile.  The 
technology  teaching  lab  is  one  option  for  providing  such  qualities  to  a preservice  education  program.  In  addition,  the  template 
used  throughout  this  program  is  a valuable  tool  for  departments  to  use  when  evaluating  program  and  providing  evidence  for 
current  and  future  grants.  Finally,  the  technology  teaching  lab  and  the  accompanying  template  provides  for  opportunity  for 
reflection  and  demonstration  of  best  works,  similar  to  a portfolio.  The  benefits  from  this  program  are  visible,  follow  from  the 
collected  data,  and  provide  for  opportunities  to  infuse  technology  into  the  preservice  teacher  education  program  and  expectations. 
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Abstract 

This  2X2  factorial  experimental  study  is  to  investigate  effects  of  intrinsic  motivation  and  embedded  instructional  strategies 
designed  to  enhance  relevance  on  students'  perceptions  of  motivation  and  understanding  of  the  instructional  material  within  a 
computer-based  interactive  multimedia  lesson  for  college  English  as  a foreign  language  students.  The  findings  indicated  that 
highly  intrinsically  motivated  students  learned  more  and  the  use  of  embedded  relevance  strategies  facilitates  students'  language 
learning  regardless  of  learners ' level  of  intrinsic  motivation. 

Introduction 

Traditionally,  schools  rely  heavily  on  extrinsically  motivated  behavior  (Brown,  1994).  For  example,  standardized  tests, 
exams  which  have  given  high  authority,  are  often  used  to  drive  student  performance.  In  most  countries  that  teach  English  as  a 
foreign  language,  school-level  instruction  does  not  emphasize  the  function  of  English  as  a tool  for  communication  but  instead 
focuses  on  how  to  pass  the  tests.  The  consequence  of  this  is  that  students  work  hard  to  try  to  pass  the  exam  to  please  teachers  and 
parents  rather  than  develop  an  internal  thirst  for  knowledge  and  experience.  It  is  not  a surprise  that  students  often  lose  their 
interest  in  English  learning  as  a result.  EFL  teachers  should  provide  students  with  authentic,  functional,  interactive,  and 
constructive  language  learning  environments  to  reduce  students'  anxiety,  raise  students'  motivation,  and  increase  students' 
confidence.  The  use  of  relevance- enhancing  strategies  in  a CBIM  language-learning  environment  offers  promise.  This  study 
investigated  the  effects  of  intrinsic  motivation  and  embedded  instructional  strategies  designed  to  enhance  relevance  on  students' 
perceptions  of  motivation  as  well  as  students’  understanding  of  the  instructional  mataial  within  a computer-based  interactive 
multimedia  (CBIM)  lesson  for  college  English  as  a foreign  language  students. 

Background 

In  recent  times,  a major  trend  in  pedagogy  related  to  second  language  learning  has  arisen  from  the  realization  that  knowledge 
of  grammatical  forms  and  structures  alone  does  not  adequately  prepare  learners  for  effective  and  appropriate  use  of  the  language 
they  are  learning.  Researchers  have  claimed  that  there  should  be  a shift  in  emphasis  from  the  structure  and  form  of  language  to 
communicative  meaning.  They  have  argued  pointedly  that  the  ultimate  goal  for  language  learning  and  teaching  is  to  help  learners 
develop  communicative  competence  (Bems,  1990;  Savignon,  1997),  Hence,  communicative  language  teaching  (CLT)  researchers 
have  identified  new  pedagogical  orientations  and  suggested  that  communicative  approaches  are  needed  in  language  teaching  and 
learning  (Angelis  & Henderson,  1989;  Bems,  1990;  Savignon,  1983,  1997;  Underwood,  1984).  In  order  to  help  accomplish  the 
ultimate  goal  of  communicative  language  teaching,  Brown  (1993)  suggested  the  utilization  of  technology  such  as  films,  videos, 
and  computers. 

Keller  (1983,  1987)  claimed  that  strategies  embedded  in  instructional  materials  can  enhance  the  learner's  attention  to  the 
materials  and  perceptions  of  relevance,  confidence  and  satisfaction  about  learning,  which  in  turn  can  enhance  cognitive 
performance.  Keller  and  Kopp  (1987)  argued  that  embedding  relevance  strategies  can  improve  motivation  and  performance  and 
should  be  used  to  connect  the  learning  to  the  learner.  Nwagbara  (1993)  reported  positive  findings  about  building  a relevance 
component  in  instructional  material  to  improve  learners’  motivation. 

CBIM  (computer-based  interactive  multimedia)  is  an  instmctional  approach  that  can  create  a language  learning  environment 
that  embraces  constructivism  and  interaction  (Milheim,  1995)  through  its  multiple  roles  and  unique  properties,  SLA  (second 
language  acquisition)  researchers  have  indicated  that  the  audio-visual  and  textual  resources  found  within  computer  applications 
can  aid  in  building  activities  to  help  students  develop  the  various  competencies  mobilized  in  communication  (Chanier,  1996; 
Murray,  Morgenstem,  & Furstenberg,  1991).  There  is  no  doubt  that  CBIM  has  a promising  future  for  foreign  language  teaching 
and  learning.  However,  technology  itself  cannot  help  instruction  without  careful  instructional  design.  Using  the  unique  features 
of  CBIM,  integration  of  appropriate  instructional  strategies  can  foster  student  learning  and  help  learners  learn  how  to  learri 
(Reigeluth,  1983;  Wenden,  1985). 

This  study  was  conducted  to  investigate  the  effect  of  embedding  relevance-enhancing  strategies  in  foreign  language  classes. 
The  relationship  between  learners'  motivation  and  embedded  relevance  strategies  was  also  examined. 
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Methods 

The  2X2  factorial  experimental  posttest -only  design  was  employed  with  two  categorical  independent  variables,  level  of 
intrinsic  motivation  (higher  or  lower)  and  embedded  relevance  enhancement  (with  or  without),  and  two  continuous  dependent 
variables,  achievement  and  motivation  perception.  The  Intrinsic  Motivation  Orientation  Scale  (IMOS),  a scale  developed  by  the 
researcher  (Cronbach  coefficient  alpha  was  .93),  measured  the  first  independent  variable,  learners’  level  of  intrinsic  motivation. 
Totally  313  subjects  were  categorized  as  having  a higher  or  lower  level  of  intrinsic  motivation  based  on  their  score  on  the  IMOS. 
The  second  independent  variable,  embedded  enhanced  relevance  strategies,  was  the  treatment;  strategies  were  based  on  the 
Relevance  Concept  and  Tactics  Checklist  developed  by  Keller  (1990).  Within  each  intrinsic  motivation  group,  learners  were 
randomly  assigned  to  a treatment  consisting  of  a computer-based  instructional  multimedia  program  featuring  English  language 
text,  videos,  and  exercises  on  the  topic  of  criticism  either  with  or  without  enhanced  relevance  components.  The  dependent 
variables,  operationally  defines  as  the  score  on  a comprehension  test  and  the  score  on  the  Modified  Instructional  Material 
Motivation  Survey  (MIMMS),  were  measured  after  students  completed  the  computer-based  interactive  multimedia  (CBIM) 
program.  Two-way  analysis  of  variance  (ANOVA)  was  used  to  analyze  the  data  collected. 


Data  Analysis  and  Results 

The  comprehension  test  means  for  each  of  the  four  treatment  groups  are  shown  in  Table  1.  When  comparing  scores  by  Level 
of  Intrinsic  Motivation,  one  can  see  the  Higher  Level  of  Intrinsic  Motivation  group  mean  of  1 1.97  was  higher  than  the  Lower 
Level  of  Intrinsic  Motivation  group  mean  of  10.47.  The  Higher  Level  of  Intrinsic  Motivation  group  obtained  a higher  score  than 
the  Lower  Level  of  Intrinsic  Motivation  group  in  both  the  control  and  experimental  condition.  When  comparing  means  by 
treatments,  the  experimental  group  mean  of  12.75  was  higher  than  the  control  group  mean  of  9.63.  In  spite  of  different  levels  of 
intrinsic  motivation,  subjects  who  received  embedded  relevance  enhancement  obtained  higher  scores  on  the  comprehension  test 
than  subjects  who  did  not  receive  embedded  relevance  enhancement. 

Table  I Comprehension  Test  Means  (SDs)  for  All  Four  Groups 

Level  of  Intrinsic  Motivation 


Treatments 

Higher 

Lower 

Combined 

With 

Enhanced  (experimental) 

n = 79 

M = 13.51(4.88) 

n = 80 

M=  11.99(4.49) 

n = 159 

M=  12.75(4.73) 

Relevance  Without 

(control) 

n = 76 

M=  1036(3  J9) 

n = 78 

M = 8,92(3,09) 

n = 154 
M = 9.63(3.52) 

Combined 

n = 155 

M=  11.97(4.64) 

n = 158 

10.47(4.14) 

The  results  of  the  MIMMS  for  each  of  the  four  groups  are  shown  in  Table  2.  The  Higher  Level  of  Intrinsic  Motivation  group 
mean  of  125.63  was  greater  than  the  Lower  Level  of  Intrinsic  Motivation  group  mean  of  1 13.85.  The  mean  for  the  Embedded 
Relevance  Enhancement  group  of  123.10  was  greater  than  the  mean  for  the  control  group  of  1 16.21. 

Table  2 The  MIMMS  Means  (SDs)  for  All  Four  Groups 

Level  of  Intrinsic  Motivation 


Treatments 

Higher 

Lower 

Combined 

With 

n^79 

n^79 

n = 158 

Enhanced 

(experimental) 

130.62(15.19) 

M =115.58(16.45) 

M=123. 10(17.49) 

Relevance 

Without 

n = 76 

n = 78 

n = 154 

(control) 

M = 120.43(15.26) 

M=  112.10(14.64) 

M =116.21(15.48) 

Combined 

n = 155 

n = 157 

M=  125.63(16.01) 

M=  113.85(15.63) 

Two-way  analysis  of  variance  (ANOVA)  was  employed  to  test  the  research  hypotheses.  The  SAS  GLM  procedure  was  used 
because  of  unbalanced  group  size.  Overall  statistical  results  (Tables  3)  indicate  that  there  were  significant  differences  among 
groups  on  both  the  Comprehension  Test  [F  (3,  309)  = 1 8.25,  p = .000 1 ] and  the  MIMMS  [F  (3,  308)  = 2 1 .50,  p < .000 1 ]. 
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For  the  comprehension  test,  there  was  a significant  difference  between  the  two  levels  of  Intrinsic  Motivation  (F  = 10.13,  p = 
0.0016)  as  well  as  between  two  levels  of  Embedded  Relevance  Enhancement  (F  = 44.46,  p = 0.0001).  The  interaction  between 
levels  of  Intrinsic  Motivation  and  levels  of  Embedded  Relevance  Enhancement  was  not  significant  (F  = .01,  p = .93).  The  results 
show  that  there  was  a significant  main  effect  of  the  Level  of  Intrinsic  Motivation  and  a significant  main  effect  of  the  Embedded 
Relevance  Enhancement  on  the  comprehension  test.  Students  with  higher  levels  of  intrinsic  motivation  obtained  higher  scores 
than  students  with  lower  levels  of  intrinsic  motivation.  Students  who  learned  from  the  embedded  relevance  enhancement  program 
outperformed  the  students  who  learned  from  the  no  embedded  relevance  enhancement  program.  There  was  no  interaction  effect 
between  Level  of  Intrinsic  Motivation  and  Embedded  Relevance  Enhancement  on  the  comprehension  test. 


Table  3 Two-Way  GLM ANOVA  for  Comprehension  Test  and MIMMS 


Source 

dl 

TvoelllSS 

MS 

F 

U 

Comprehension  Test 

Level  of  Intrinsic  motivation 

I 

172.54 

172.54 

10.13 

.0016 

Level  of  Relevance  Enhancement 

I 

756.97 

756.97 

44.46 

.0001 

LOIM*LORE 

I 

MIMMS 

.13 

.13 

.01 

.93 

Level  of  Intrinsic  Motivation 

I 

10647.05 

10647.05 

44.88 

.0001 

Level  of  Relevance  Enhancement 

I 

3640.77 

3640.77 

15.35 

.0001 

LOIM*LORE 

I 

876.79 

876.79 

3.70 

.0555 

Note,  LOIM*LORE  = Interaction  between  Level  of  Intrinsic  Motivation  and  Embedded  Relevance  Enhancement. 

For  motivation  perception,  the  results  showed  that  there  was  a significant  difference  between  the  two  levels  of  Intrinsic 
Motivation  (F  = 44.88,  p = 0.0001)  as  well  as  between  the  two  levels  of  Embedded  Relevance  Enhancement  (F  = 15.35,  p = 
0.0001)  on  the  MIMMS.  The  interaction  between  Level  of  Intrinsic  Motivation  and  Embedded  Relevance  Enhancement  on  the 
MIMMS  was  not  significant  (F  = 3.70,  p = 0.0555).  Results  indicated  that  there  were  significant  main  effects  of  Level  of  Intrinsic 
Motivation  and  Relevance  Enhancement  on  the  MIMMS.  The  MIMMS  mean  of  the  Higher  Level  of  Intrinsic  Motivation  group 
was  greater  than  that  of  the  Lower  Level  of  Intrinsic  Motivation  group.  Students  who  learned  from  the  embedded  relevance 
enhancement  program  outperformed  students  who  learned  from  the  program  without  embedded  relevance  enhancement  on  the 
MIMMS.  The  interaction  effect  between  the  Level  of  Intrinsic  Motivation  and  the  Level  of  Embedded  Relevance  Enhancement 
on  motivation  perception  was  small  and  not  statistically  significant. 

The  findings  indicated  that  the  use  of  embedded  relevance  strategies  facilitates  students’  language  learning  regardless  of 
learners’  level  of  intrinsic  motivation.  Appropriately  constructed  CBIM  instructional  materials  with  embedded  relevance 
enhancement  can  benefit  EFL  learners  in  English  learning.  The  findings  of  the  study  also  showed  that  students  with  higher  levels 
of  intrinsic  motivation  learned  more.  More  highly  intrinsically  motivated  students  performed  better  regardless  of  the  specific 
treatments  they  received.  Among  all  four  treatment  goups,  the  highly  intrinsically  motivated  students  who  learned  from  the 
program  with  embedded  instructional  strategies  program  performed  the  best  overall.  There  was  no  significant  interaction  between 
these  two  variables.  However,  a possible  tendency  toward  a differential  motivational  response  to  the  instructional  materials  was 
noted;  more  highly  intrinsically  motivated  subjects  tended  to  be  more  motivated  by  enhanced  relevance  strategies  than  less 
intrinsically  motivated  subjects. 

Discussions  and  Implications 

Findings  of  this  study  support  Keller’s  (1983,  1987)  declaration  that  embedded  instructional  strategies  can  enhance  learners’ 
motivation,  which,  in  turn,  can  enhance  cognitive  performance.  This  finding  is  consistent  with  previous  research  showing  that 
incorporating  relevance  strategies  into  instructional  materials  can  increase  students’  motivation  (Means,  Jonassen,  and  Dwyer, 
1997;  Nwagbara,  1993;  Sass,  1989).  It  also  supports  Keller’s  (1983)  theory  of  motivation,  which  argues  that  learners’  motivation 
can  be  influenced,  and  the  ARCS  model,  which  provided  the  set  of  strategies  for  improving  motivation  in  this  study.  The  results 
also  indicate  that  the  effects  of  the  two  variables  are  additive.  The  embedded  instructional  strategies  added  to  the  effect  of  the 
existing  intrinsic  motivation  of  students.  The  results  of  the  study  suggest  that  students  can  learn  better  when  they  are  notified  of 
the  immediate  benefit  of  the  instruction  they  receive;  when  there  are  examples  included  in  the  material  to  stress  the  intrinsic 
satisfactions  of  the  subject  of  instruction  (goal  orientation:  present  worth).  Students  learn  better  when  they  know  what  they  will 
be  able  to  do  after  finishing  the  instructional  materials;  when  there  are  examples  and  exercises  clearly  related  to  the  knowledge 
and  skills  that  they  will  need  in  the  future;  when  they  are  told  that  this  instruction  will  improve  their  general  life  coping  skills 
(goal  orientation:  future  value).  The  results  also  suggest  that  learning  is  facilitated  by  employing  (1)  personal  language  to  make 
learners  feel  that  they  are  being  talking  as  a person,  (2)  exercises  with  feedback,  and  (3)  exercises  that  stimulate  problem  solving 
(motive  matching:  basic  motive  stimulation).  EFL  teachers  may  draw  implications  for  classroom  practice  from  this  research. 
Teachers  should  talk  with  students  as  people;  relate  instruction  to  students’  daily  lives;  and  point  out  how  the  instructional 
materials  can  be  used  in  the  real  world  or  in  students'  lives  in  the  future.  Teachers  should  also  try  different  styles  of  exercise. 
Some  problem  solving  exercises,  in  addition  to  basic  language  practice,  can  help  students’  learning.  Relevance  enhancement 
facilitated  students’  learning  in  this  study. 

The  positive  findings  should  encourage  teachers  and  program  designers  to  use  embedded  instructional  strategies  either  in 
class  or  in  courseware  design.  Instructional  designers  are  encouraged  to  use  the  ARCS  model  as  a model  for  designing  motivating 


57 


66 


instruction.  Based  on  the  learner  assessment,  instructional  designers  can  (1)  integrate  the  whole  ARCS  model  into  instructional 
materials,  (2)  embed  one  of  the  components  of  the  model  into  instructional  material,  or  (3)  apply  strategies  listed  on  the 
motivational  tactics  checklist,  to  enhance  learners  motivation  and  improve  learning.  Future  research  should  further  examine  the 
use  of  relevance  enhancement  strategies.  Also,  other  components  of  the  ARCS  model,  such  as  attention,  confidence,  or 
satisfaction  should  also  be  studied  for  their  impact  on  learning.  This  theoretical  framework  offers  a rich  source  of  instructional 
strategies  for  teachers  and  instructional  designers. 
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Pennsylvania  State  System  of  Higher  Education  School  Web  Site  Review  for 
Usability  And  Information  Availability 

Mark  E.  Chase 
Slippery  Rock  University 


Study  Design 

This  Study  compares  and  contrasts  the  web  sites  of  the  fourteen  Pennsylvania  State  System  of  Higher  Education  schools  to 
identify  those  that  provide  efficient,  accurate,  and  effective  access  to  a standardized  list  of  information  resources.  The  study  takes 
a two-fold  approach  to  achieve  this  goal.  By  examining  the  content  provided  and  further  analyzing  the  presentation  of  that 
content,  an  evaluation  can  be  made,  and  a comparison  can  be  drawn  between  institutional  web  sites.  The  researcher  first  visited 
each  school’s  web  site  in  August  of  1999  and  used  the  same  instrument  to  revisit  the  web  sites  in  May  2000  and  March  2001 
looking  for  twenty  specific  pieces  of  information  common  to  all  state  system  schools. 

Each  of  these  twenty  items  was  evaluated  based  on  five  criteria: 

1 . Does  the  school’s  site  provide  the  requested  information? 

2.  How  difficult  is  the  information  to  locate? 

3.  How  current  is  the  information? 

4.  Does  the  page  use  appropriate  design  skills  and  contain  necessary  navigational  elements  (as  defined  by  Patrick  Lynch’s  Yale 
Web  Style  Manual  [Lynch  & Horton  1999] 

5.  Is  the  page  professional  in  appearance? 

A point  value  from  one  to  five  was  used  for  each  of  the  above  questions  for  a total  of  25  points.  These  25  points  could  be 
accumulated  for  each  of  the  20  information  items  that  were  researched.  A total  of  500  points  were  possible. 

Assessment  criteria  assumptions 

As  the  Internet  continues  to  g'ow,  a school’s  web  site  is  increasing  becoming  marketing,  recruitment,  and  public  relations 
tools  that  educational  institutions  need  to  closely  monitor.  A recent  survey  by  the  Slippery  Rock  University  Admissions  office 
indicated  that  the  school’s  web  site  was  second  only  to  family  and  friends  in  influencing  their  decision  on  applying  to  the 
institution  [Bracco  2000].  The  importance  of  a school  web  site  outlining  accurate  and  detailed  information  for  prospective 
students  is  clear.  A well-designed  web  site  providing  information  that  matches  the  audiences’  needs  will  result  in  higher 
enrollment,  increased  alumni  donations,  and  a greater  level  of  credibility  for  the  institution  [McCollum  1999]. 

This  study  makes  the  following  assumptions  in  the  collection  and  analysis  of  data: 

University  sites  should  all  provide  a baseline  of  information  resources 

University  site  visitors  should  be  able  to  easily  locate  information  without  searching  deep  into  a site  hierarchy 
University  site  pages  should  provide  current  information 

University  site  pages  should  have  a professional  appearance  (good  grammar,  accurate  spelling,  consistent  layout) 

Identification  of  common  information 

University  web  sites  are  designed  to  serve  a variety  of  target  audiences.  These  audiences  include  but  are  not  limited  to: 
prospective  students,  current  students,  alumni,  institutional  employees,  community  members,  researchers,  and  colleagues.  Since 
university  web  pages  are  expected  to  serve  a variety  of  users,  a list  of  information  that  these  groups  might  request  was  generated. 
That  list  was  refined  to  twenty  items  that  would  be  typical  of  the  SSHE  institutions.  Those  twenty  items  include: 

1)  Search  engine 

2)  Site  Map 

3)  Tuition  and  fee  schedule 

4)  Academic  Calendar 

5)  Distance  education  course  offerings 

6)  Library  resources 

7)  Address  of  the  institution 

8)  Registrar’s  name  and  address 

9)  How  to  order  transcripts 

10)  Department  listings 

1 1)  Academic  department  chair  (English) 

12)  Mission  Statement 
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13)  List  of  degree  and  certificate  programs 

14)  Admissions  information 

15)  Name  of  the  President 

16)  Name  of  the  Provost 

17)  Calendar  of  Events 

18)  Link  to  State  System  of  Higher  Education 

19)  Faculty  directory 

20)  Alumni  page 

Data  Collection 

All  data  for  each  of  the  fourteen  schools  was  collected  during  the  first  week  of  August  1999,  the  second  week  of  May  2000, 
and  the  second  week  of  March  2001.  The  researcher  accessed  each  web  site  attempting  to  locate  the  information  from  the  list. 
Search  engines,  directories,  and  browsing  techniques  were  used  to  locate  information.  The  following  point  scale  was  used  for 
evaluating  question  #1 ; “Does  the  school’s  site  provide  the  researched  information  item?” 

5 points  - requested  information  is  provided 

3 points  - some  of  requested  information  is  provided 

1 point  - requested  information  is  not  located 

If  the  information  was  not  located,  the  institution  received  zero  points  for  the  remaining  four  questions.  The  minimum  score 
any  school  could  receive  on  any  of  the  items  was  one  point.  Therefore,  the  lowest  possible  score  was  20,  and  the  highest  possible 
score  was  500. 

Once  the  information  was  located  the  shortest  path  to  accessing  the  information  was  determined.  Significantly  if  the 
information  was  located  via  a search  engine,  the  researcher  needed  to  determine  how  this  information  was  linked  to  the  main 
page  (on  what  level  of  the  hierarchy  did  it  reside).  The  assumption  for  evaluation  is  that  if  the  user  has  to  review  fewer  pages  to 
find  the  requested  item,  it  is  easier  for  them  to  locate  the  information.  The  following  point  scale  was  used  for  question  #2,  “How 
difficult  is  the  information  to  locate?” 

5 points  - required  access  of  three  or  fewer  pages 

4 points  - required  access  of  four  pages 
3 points  - required  access  of  five  pages 

2 points  - required  access  of  six  pages 

1 point  - required  access  of  seven  or  more  pages 

The  web  page  that  contained  the  requested  information  was  evaluated  to  determine  if  the  information  provided  was  current. 
Many  pages  list  a last  date  of  revision.  Other  dates  could  be  determined  by  the  content  on  the  page.  These  dates  were  used  for  the 
evaluation.  If  no  date  of  revision  is  provided,  the  researcher  assumed  that  information  was  current  unless  there  were  overt 
indications  otherwise  (i.e.  calendar  of  events  for  a previous  year).  The  following  point  scale  was  used  for  question  #3,  “How 
current  is  the  information?” 

5 points  - requested  information  is  less  than  6 months  old 

3 points  - requested  information  is  less  than  1 year  old 
1 point- requested  information  is  more  than  1 year 

Question  #4  evaluated  the  web  pages  use  of  good  design  skills.  Patrick  Lynch’^  Yale  Web  Style  Manual  provided  the  standard 
for  evaluation.  The  researcher  looked  critically  at  only  the  most  elementary  of  design  skills.  Did  links  on  the  page  work?  Were 
graphics  used  on  the  page  able  to  load?  Were  consistent  navigational  tools  employed?  Were  text  colors  and  size  appropriate  and 
readable?  The  researcher  limited  the  evaluation  to  three  responses  to  question  #4,  “Does  the  page  use  appropriate  design  skills 
and  contain  necessary  navigational  elements?” 

5 points  - page  uses  excellent  design  skills 
3 points  - page  uses  average  design  skills 
1 point- page  uses  poor  design  skills 

The  final  question  looked  at  the  page’s  professional  appearance.  Does  the  page  use  proper  grammar,  punctuation,  and 
spelling?  Is  the  information  organized  and  presented  in  a logical  format?  The  researcher  will  again  limit  the  responses  to  question 
#5,  “Is  the  page  professional  in  appearance?” 

5 points  - page  is  professional  in  appearance 

3 points  - page  has  one  or  more  spelling,  grammar,  or  punctuation  errors;  or  page  is  poorly  formatted 
1 point -page  has  multiple  errors  and/or  is  poorly  formatted 

Data  Analysis 

Previous  reports  have  discussed  the  finding  of  the  first  two  rounds  of  data  collection.  This  analysis  reviews  the  data  collection 
that  took  place  during  the  second  week  of  March  2001.  The  researcher  visited  each  of  the  fourteen  SSHE  school  websites  looking 
for  the  informational  items.  Internet  Explorer  5.0  was  used  with  a screen  size  set  to  600  x 800.  All  sites  were  accesses  via  a 56k 
modem  via  an  independent  Pennsylvania  Internet  service  provider.  Scores  were  totaled  for  all  of  the  twenty  information  items 
and  a cumulative  score  was  tabulated  for  each  institution.  The  individual  scores  for  each  of  the  twenty  elements  for  each  school 
can  be  found  in  Appendix  A.  The  following  table  is  a summary  of  the  cumulative  scores  for  each  of  the  three  past  years. 
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1999 

2000 

2001 

Total  (Rank) 

Total  (Rank) 

Total  (Rank) 

Lock  Haven 

341(11) 

421  (4) 

486(1) 

California 

368(8) 

409(6) 

468(2) 

Millersville 

398  (4) 

463(1) 

466(3) 

Clarion 

392  (6) 

394(8) 

462(4) 

Mansfield 

319(13) 

363(11) 

460(5) 

Slippery  Rock 

400(2) 

399(7) 

439  (6) 

Bloomsburg 

369(7) 

420(5) 

433(7) 

Shippensburg 

393(5) 

394  (8) 

431(8) 

Indiana 

393(3) 

451  (2) 

425  (9) 

Kutztown 

339(12) 

384(10) 

423(10) 

West  Chester 

432(1) 

439(3) 

414(11) 

East  Stroudsburg 

343(10) 

350(12) 

414(11) 

Edinboro 

360(9) 

319(13) 

346(13) 

Chevnev 

280(14) 

254(14) 

316(14) 

mean 

366 

390 

427 

Table  J Cumulative  scores  and  rankings  for  the  past  three  years  for  each  of  the  fourteen  Pennsylvania  State  System  of  Higher 
Education  schools. 


The  average  total  score  has  increased  each  year,  and  this  has  had  an  impact  on  schools’  rankings.  West  Chester,  who  ranked 
P*  after  the  year  of  data  collection,  is  now  ranked  1 1*  because  their  score  decreased  by  18  points  over  the  three  year  period. 
Shippensburg,  who  has  seen  an  improvement  in  their  score  each  year,  has  fallen  from  4^*^  to  position.  The  lesson  to  be  learned 
is  that  if  you  are  not  actively  working  to  improve  your  site,  you  are  falling  behind  the  competition.  A web  site  is  a constantly 
evolving  product  that  needs  to  be  actively  molded,  revamped,  and  updated.  A stagnant  site  reflects  poorly  on  an  institution  and  is 
not  accomplishing  and  promoting  their  mission. 

These  scores  reflect  a snapshot  of  institutional  web  pages  during  the  collection  period.  They  are  a single  perspective  on 
usability  and  information  availability  to  help  schools  identify  weaknesses  in  their  sites. 

Summary 

Users  are  frequently  relying  on  more  advanced  navigational  aids  to  find  information.  The  need  for  a search  engine  and  a site 
or  index  map  has  become  increasingly  more  important.  Because  of  the  quantity  of  information  that  is  being  delivered  at  most  of 
the  institutional  sites,  the  location  or  organization  of  information  in  the  hierarchy  is  not  always  intuitive.  Some  links  fall  under 
department’s  representative  areas  as  a result  of  institutional  history  rather  than  traditional  form.  The  number  of  schools  that 
provide  search  engines  jumped  from  five  in  2000  to  eleven  in  2001 . The  three  schools  that  do  not  provide  a search  engine  on  their 
site  recorded  the  three  lowest  scores.  Only  six  of  the  schools  currently  provide  a site  map  or  site  index.  The  schools  that  did 
provide  a site  map  provided  an  outline  view  of  the  site  hierarchy.  A few  schools  used  the  site  index  approach,  which  alphabetized 
all  the  major  pages  located  on  the  site.  The  researcher  found  the  alphabetized  approach  easier  to  locate  specific  information. 

Several  of  the  information  items  that  were  searched  for  are  well  represented  at  all  the  institutions.  All  schools  recognize  the 
importance  of  providing  admissions  information,  library  links,  an  academic  calendar,  and  the  address  of  the  institution.  All  but 
one  of  the  schools  provided  tuition  and  fees  information,  registrar’s  name,  transcript  information,  department  listings,  the  chair  of 
the  English  department,  a link  to  the  State  System  web  page,  faculty  directory  and  an  alumni  page.  There  is  confusion  at  some 
sites  concerning  the  distinction  between  department  listings  and  degree  programs.  One  site  provided  identical  links  to  both 
listings.  The  listing  of  degree  programs  offered  by  an  institution  is  a critical  element  to  be  linked  on  the  admission  or  prospective 
student  areas.  By  forcing  students  to  search  through  the  department  listings,  schools  risk  alienating  the  student  because  the 
information  is  difficult  to  locate  or  may  not  be  available  at  the  department  level. 

The  required  element  that  generated  the  highest  mean  score  was  the  academic  calendar.  In  March  2000,  the  department 
listings  provided  the  highest  mean  score.  The  academic  calendar  provides  critical  information  to  those  on  campus,  but  also  to 
prospective  students,  off-campus  organization,  and  those  at  sister  institutions.  It  is  no  surprise  that  the  administration  sees  the 
academic  calendar  as  an  imperative  in  constructing  a web  site. 

There  is  little  argument  that  the  Internet,  web  design,  and  institutional  sites  are  in  the  early  stage  of  their  evolutionary 
process.  The  playing  field  continues  to  change,  and  standards  are  modified  as  hardware,  software,  and  user  behaviors  change. 
Designers  must  attempt  to  create  pages  that  reflect  current  usage  statistics  for  their  audience. 
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A recent  study  by  Mycomputer.com  revealed  that  49.5%  of  users  are  reviewing  web  pages  with  a screen  resolution  of  600  x 800 
and  that  less  than  10%  of  users  still  use  640  x 480  as  their  preferred  resolution  size  [Lake  2001].  While  these  types  of  hardware 
and  software  profiles  change  over  time,  current  designers  need  to  create  screens  that  meet  the  profiles  of  the  majority  of  their 
visitors.  Therefore  as  pages  were  analyzed  for  this  study,  they  were  reviewed  using  a screen  resolution  setting  of  600  x 800.  If  the 
schools’  pages  did  not  format  correctly  to  this  screen  size,  points  were  deducted  related  to  graphic  design.  A few  schools  had 
home  pages  that  were  designed  for  1024  x 768  screen  sizes.  This  caused  the  content  to  be  cropped  and  forced  the  user  to  scroll 
down  and  to  the  right  to  review  all  the  information. 

For  the  first  time  this  study  collected  data  on  how  long  it  takes  for  an  institutions  web  page  to  load.  A recent  Jupiter  Research 
survey  showed  that  84%  of  users  indicated  that  they  left  sites  because  of  slow  or  broken  links  [Lake  2001].  Optimizations  of 
images  and  being  cognizant  of  the  total  file  size  for  the  university’s  main  home  page  is  an  imperative  factor  in  web  site  design. 
The  researcher  used  a 56k  modem  on  a 550mhz  PC  connected  to  a Western  Pennsylvania  Internet  service  provider.  The  data  was 
collected  between  1 1 a.m.  and  12  p.m.  on  a Friday.  This  time  was  selected  to  test  the  schools’  sites  during  a period  when  it  would 
normally  be  receiving  heavy  traffic.  Given  that  88%  of  households  connect  to  the  Internet  at  56kps  or  less,  it  is  easy  for  designers 
at  institutions  with  T3  line  access  to  forget  that  download  times  can  be  critical  [Lake  2001]. 

The  data  collected  on  download  times  was  not  factored  into  the  final  score  for  each  school.  Future  studies  may  include  a 
factor  related  to  download  times.  The  collection  of  this  information  came  from  the  researcher’s  frustration  of  slow  download 
times  while  reviewing  the  individual  sites.  One  of  the  SSHE  school’s  home  page  took  over  six  minutes  to  load,  while  the  average 
for  the  schools  was  over  three  minutes.  Perhaps  some  of  the  slow  response  time  can  be  attributed  to  SSHEnet,  the  State  System  of 
Higher  Education  Network.  If  the  bottleneck  for  delivery  is  the  network,  SSHE  schools  still  need  to  take  this  into  account  as  they 
design  pages,  particularly  pages  for  prospective  students,  alumni,  or  other  outside  groups.  Many  of  the  schools  that  reported  low 
times  had  extensive  graphics  files  (some  with  over  60  files)  or  rotating  color  slide  shows  on  their  home  page.  Edward  Guttman, 
lead  designer  at  Viant,  a web  development  firm,  notes,  “Only  if  you  can  prove  that  users  will  get  added  value  through  enhanced 
site  features  are  they  worthwhile”  [Lake,  2001].  While  these  slideshows  may  provide  wonderful  images  from  the  admissions 
view  book,  their  slow  load  time  outweighs  any  aesthetic  value  they  may  add  to  the  page. 

Shippensburg  and  a few  other  schools  have  begun  to  address  the  multi-audience  design  considerations  through  the  use  of  an 
Intranet,  a portion  of  their  web  site  designed  primarily  for  on-campus  personnel  and  students.  While  some  off-campus  access 
takes  place  on  this  Intranet,  the  design  considerations  can  reflect  that  most  of  the  users  will  have  high  speed  access.  This  strategy 
may  be  appropriate  for  some  of  the  schools. 

One  school  still  uses  a cover  page  on  their  site.  Cover  pages  are  graphical  pages  that  load  as  a preamble  to  a site.  They 
provide  minimal  content  and  usually  only  one  or  two  links  to  proceed  into  a site.  Their  effectiveness  verses  their  annoyance  has 
been  debated  [Lynch  & Horton  1999].  The  majority  of  site  critics  feel  they  simply  create  an  additional  barrier  between  the  user 
and  the  content.  However,  they  can  provide  some  functionality  by  preloading  images  for  graphically  intensive  sites. 

After  the  completion  of  the  data  collection,  a follow-up  question  was'sent  to  each  of  the  schools  asking,  “Do  you  employ  a 
full-time  webmaster?”  and  “What  division  in  the  institution  is  responsible  for  maintaining  the  school’s  web  site?”  The  following 
table  provides  the  results  of  these  questions  for  each  of  the  schools. 

Ten  of  the  fourteen  schools  indicate  that  they  have  full-time  webmasters,  although  Shippensburg  splits  this  responsibility 
between  two  people.  The  top  five  schools  in  the  study  analysis  all  have  full  time  webmasters. 

The  responsibility  of  the  web  site  broke  into  two  distinct  areas,  although  sometimes  this  was  a shared  effort.  Seven  schools 
indicated  that  the  Advancement  or  Public  Relations  (normally  a division  of  Advancement)  areas  were  primarily  responsible  for 
the  content  and  updating  the  schools’  pages.  Four  schools  indicated  that  an  Information  Technology  area  held  that  responsibility. 
Three  schools  have  joint  responsibility  between  the  IT  and  Advancement/Public  Relations  areas.  The  top  four  schools  in  the 
instrument  analysis  either  had  Advancement/Public  Relations  or  joint  responsibility. 
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Webmaster 

Diyision 

Lock  Haven 

Yes 

Advancement 

California 

Yes 

PR 

Millersville 

Yes 

IT/PR 

Clarion 

Yes 

Advancement 

Mansfield 

Yes 

IT 

Slippery  Rock 

No 

IT 

Bloomsburg 

No 

Advancement 

Shippensbure 

Yes* 

IT 

Indiana 

Yes 

Advancement 

Kutztown 

Yes 

IT/PR 

Westchester 

Yes 

IT 

East  Stroudsburg 

No 

Acad  Comp/PR 

Edinboro 

No 

PR 

Cheyney 

Yes 

Advancement 

* Two  1/2  time 

Table  2.  ‘'Does  the  institution  employ  a full-time  webmaster?  " and  "What  division  at  the  institution  is  responsible  for 
maintaining  the  school ‘s  web  site?  " 

A few  years  ago  it  was  easy  to  determine  the  area  of  responsibility  of  a school’s  web  page  by  looking  at  the  images  and 
information  organization.  Schools  that  attempted  to  replicate  their  view  book  in  the  first  few  pages  of  the  hierarchy  were 
normally  created  by  public  relations  areas.  School’s  web  pages  that  pushed  items  like  e-mail  instructions  or  ftp  links  in  the  upper 
levels  of  the  site  were  probably  developed  by  the  IT  divisions.  While  these  distinctions  are  not  as  clear  today,  there  are  design 
issues  that  are  addressed  differently  by  both  areas.  The  IT  area  may  sometimes  choose  solutions  that  are  technologically  efficient 
and  require  less  maintenance.  For  instance,  the  choice  of  searchable  databases  for  faculty  and  staff  directories  may  be  determined 
by  software  availability  and  efficiency  rather  than  usability.  In  contrast,  sites  developed  by  advancement  areas  may  become  too 
graphically  intensive  and  slow  to  load  in  an  effort  to  accomplish  the  “branding”  requirement. 

In  the  brief  period  of  time  that  this  study  has  been  conducted,  a maturation  of  many  of  the  SSHE  school  web  sites  has  taken 
place.  Early  sites  relied  heavily  on  organizational  structure  to  dictate  the  organization  of  information  on  institution’s  web  site 
hierarchy.  Next  came  the  revelation  of  designing  for  the  audience  and  sites  grouped  all  their  content  around  audience 
expectations.  Many  sites  have  now  moved  to  a hybrid  model  melding  the  two  with  a heavy  focus  on  the  audience  but  some 
additional  organizational  structure  from  the  institution  included.  The  past  three  years  has  seen  a consistent  improvement  of  most 
SSHE  school  web  sites,  however  the  work  is  never  complete.  Schools  need  to  be  constantly,  reevaluating  their  audience, 
reviewing  their  content  organization,  and  revising  their  pages  to  reflect  the  changng  dynamics. 


Suggestions  for  Further  Research 

Several  follow-up  or  study  reviews  may  be  incorporated  in  future  reviews.  The  researcher  felt  that  the  top  rated  site  was  not 
necessary  the  most  aesthetic  of  the  sites.  The  instrument  does  a much  better  job  of  evaluating  the  information  architecture  of  a 
site  rather  than  the  visual  design.  Perhaps  a second  phase  of  this  study  could  review  Just  the  graphic  design. 

One  way  to  provide  credibility  to  this  study  would  be  to  identify  a correlation  between  these  ranking  and  inquires  for 
applications  and  enrollment.  In  theory,  schools  with  better  web  sites  should  see  an  increase  in  inquiries,  particularly  electronic 
inquires.  After  three  years  of  using  the  same  information  items,  it  is  probably  time  to  generate  a new  list  for  review.  A review  of 
the  SSHE  imperatives  document  may  provide  ideas  for  new  items  to  include.  In  addition,  some  schools  may  become  savvy  of  the 
study  and  specifically  improve  their  pages  to  score  higher. 


References 

Bracco,  Amy.  “Electronic  Application  Survey  Report.”  Survey  completed  by  Slippery  Rock  University  Admissions  Office, 
Slippery  Rock,  PA.  April  2000. 

Gardner,  Elizabeth.  “Demanding  Clients  Are  Driving  Motivation.”  Internet  World.  November  1,  1999,  pp  54-56. 

Lake,  David.  “Quick  and  Easy”  The  Industry  Standard  Magazine. 
http://www.thestandard.eom/article/display/0, 1 151, 22342,00. html.  March  05,  2001. 

Leibowitz,  Wendy  R.  “Colleges’  Web  Sites  Should  Heed  Users’  Needs,  EDUCAUSE  Speaker  Says.”  Chronicle  of  Higher 
Education.  October  27,  1999,  p A45. 

Lynch,  Patrick  J.  & Horton,  Sarah.  Web  Style  Guide  (1999).  Yale  University  Press,  New  Haven  CT. 


o 

ERIC 


63 


72 


McCollum,  Kelly.  “Colleges  Revamp  Web  Pages  with  Professional  Help,”  The  Chronicle  of  Higher  Education^  July  1 6, 
1999,  pp.  A25-A26. 

Nielson,  Jakob.  Useit.com:  Usable  Information  Technology,  www.useit.com  (April  3,  2001). 

Nielson,  Jakob.  Useit.com:  Flash:  99%  Bad.  www.useit.eom/2000l029.  (October,  2000). 

Ruffini,  Michael  “Blueprint  to  Develop  a Great  Web  Site,”  The  Journal,  March  2001,  Vol.28  Issue  8,  pp.  64-73. 


ERIC 


Selecting  Computer  Mindtools:  A Tool  for  Constructivist  Learning 


Hui-Hui  Chen 
Steven  M.  Crooks 
Nancy  J.  Maushak 
Texas  Tech  University 


Introduction 

A number  of  media  selection  models  have  been  introduced  over  the  last  few  decades  (Anderson,  1976;  Briggs  and  Wager, 
1981;  Gagne  and  Briggs,  1979;  Cropper,  1976;  Kemp,  1980;  and  Reiser  and  Gagne,  1983).  These  models  were  designed  to  assist 
educators  and  trainers  in  selecting  the  most  appropriate  media  for  an  instructional  situation.  A common  assumption  undergirding 
these  models  is  that  instruction  is  based  on  a transmission  model  of  learning  in  which  an  instructional  medium  (e.g.,  computer, 
teacher,  television)  delivers  or  transmits  an  instructional  message  to  a learner.  This  assumption  is  often  inadequate  for  educators 
interested  in  selecting  media  for  constructivist  learning  environments. 

From  a constructivist  learning  perspective,  students  do  not  learn  "from"  technology,  but  rather  "with"  technology  (Jonassen, 
2000).  The  concept  of  learning  with  technology  focuses  on  the  intellectual  partnership  between  the  learner  and  the  computer. 
Jonassen  refers  to  computer  technologies  applied  in  this  manner  as  Mindtools.  These  tools  partner  with  learners  to  engage  and 
facilitate  higher  order  learning  and  critical  thinking.  Using  Mindtools  in  schools  is  often  more  feasible  than  other  computer 
applications  because  Mindtools  are  relatively  inexpensive  and  they  are  often  already  available. 

This  tool  was  developed  to  help  educators  make  informed  decisions  about  the  most  appropriate  Mindtool  for  students  to 
partner  with  in  specific  instructional  situations.  Instead  of  focusing  on  transmission  attributes  of  media  (e.g.,  motion  and/or  sound 
capabilities),  the  tool  interacts  with  and  asks  educators  to  specify  the  critical,  creative,  or  complex  thinking  skills  they  want 
students  to  develop  within  a given  content  domain.  Once  this  information  is  entered,  the  educator  is  given  a prioritized  list  of 
Mindtools  and  activities  that  could  effectively  be  used  to  help  students  develop  the  higher-order  thinking  skills  specified.  It  will 
be  useful  to  all  educators  who  are  interested  in  resources  that  can  support  their  efforts  in  designing  constructivist  learning 
environments. 


Constructivist  Learning  Environments 

Constructivists  believe  that  we  construct  our  own  knowledge  based  on  our  own  experiences  with  reality,  through  reflection 
on  our  interactions  with  objects  and  ideas  (Brooks  & Brooks,  1993).  From  this  perspective,  learning  occurs  through  the  continual 
creation  and  revisions  of  rules  and  hypotheses  to  explicate  what  is  observed  (Brooks,  1990).  Learners  act  more  like  active 
participants  rather  than  passive  recipients  during  the  learning  process.  In  the  learning  process,  each  person  is  continuously 
examining  new  information  with  respect  to  old  rules.  Rules  are  revised  when  discrepancies  appear  until  reaching  new 
understandings  or  constructions  of  reality. 

Based  on  the  belief  that  we  learn  by  constructing  our  own  understandings  of  reality,  contructivist  learning  environments 
should  be  student  centered,  allowing  students  to  take  an  active  role  exploring,  inquiring,  and  solving  problems.  In  addition, 
educators  should  encourage  students  to  experience  an  abundance  of  objects  and  ideas  while  empowering  them  to  ask  their  own 
questions  and  seek  their  own  answers  to  the  complexities  of  reality.  This  pedagogical  perspective  often  leads  to  assignments  and 
materials  that  are  interdisciplinary  and  integrated  and  to  instructional  methods  that  focus  on  student-to-student  interaction.  In 
such  learning  environments,  students  are  more  likely  to  take  risks  and  approach  assignments  with  a willingness  to  accept 
challenges  to  their  current  understandings  (Brooks  & Brooks,  1993).  Such  environmental  conditions  honor  students  as  emerging 
thinkers  and  engage  students'  active  understanding  with  reference  to  past  experiences  and  personal  purposes. 

Brooks  (1990)  stated  that  models  of  learning  based  on  constructivist  principles  suggest  a sequence  of  lessons  in  which 
exploration  comes  first.  In  contrast,  traditional  instruction  often  focuses  on  information  dissemination  for  the  majority  of 
instruction  and  may  only  introduce  exploration  after  all  the  material  has  been  "taught".  Brooks  (1990)  suggested  a three-stage 
model  from  constructivist-based  science  education  as  a means  of  applying  constructivism  across  the  curriculum:  the  exploration 
stage,  the  invention  stage,  and  the  discovery  stage.  Using  this  model,  teachers  engage  students  with  lesson  concepts  by  allowing 
them  to  explore  materials  or  information  (exploration  stage),  then  by  formally  introducing  the  concepts  (invention  stage)  and 
finally  by  providing  students  with  further  activities  (discovery  stage).  This  sequence  enables  students  to  develop  a robust  usable 
knowledge  base  within  a domain. 

Constructivist  learning  is  a complex  interaction  between  the  students'  disposition,  personal  purposes,  prior  knowledge,  and 
the  requirements  for  specific  subject-matter  inquiry  (Henderson,  1996).  In  constructivist  learning  environments,  instead  of 
asking  the  learners  to  memorize  and  practice  rote  skills,  educators  inspire  and  facilitate  meaningful  inquiry  learning. 


Mindtools  for  Constructivist  Learning 

Mindtools  can  be  an  especially  effective  way  to  augment  contructivist  learning  experiences  in  the  classroom.  Mindtools 
depart  from  the  conventional  use  of  instructional  technology  because  of  the  intellectual  partnership  that  is  created  between  the 
learner  and  the  computer.  In  other  words,  when  the  learner  works  with  the  computer,  the  learner  enhances  the  capabilities  of  the 
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computer,  and  the  computer  in  turn  enhances  the  thinking  of  the  learner  (Jonassen,  2000).  In  this  way,  the  computer  becomes  an 
intellectual  tool  that  learners  can  use  to  engage  and  facilitate  critical  thinking  and  higher  order  learning.  Jonassen  explains  that 
the  properties  of  Mindtools  enable  learners  to  make  more  effective  use  of  their  mental  efforts  and  create  knowledge  that  reflects 
their  comprehension  of  the  information  rather  than  replicating  the  teacher’s  presentation  of  information  (p.  10).  When  students 
use  Mindtools,  knowledge  is  built  by  the  student,  not  provided  by  the  teacher. 

When  teachers  use  Mindtools  to  promote  meaningful  thinking,  they  facilitate  learning  by  incorporating  appropriate 
Mindtools  to  help  students  think  critically  about  the  subject  matter.  Lessons  are  effectively  implemented  by  using  exploration, 
invention,  and  discovery  as  the  framework  for  classroom  learning  experiences.  Using  Mindtools  in  schools  is  often  more  feasible 
than  other  computer  applications  because  they  are  relatively  inexpensive  and  they  are  already  available  in  most  schools. 
However,  as  Jonassen  suggests,  there  is  no  reason  to  limit  the  concept  of  Mindtools  to  computer-based  resources  alone.  Any 
medium  that  supports  knowledge  construction,  manipulation,  representation,  exploration,  communication,  and  reflection  could  be 
considered  a Mindtool. 

The  Computer-based  Tool  for  Selecting  Computer  Mindtools 

Although  computers  have  been  used  for  teaching  and  learning  for  a long  period  of  time,  the  idea  of  making  use  of  computer 
Mindtools  is  still  pretty  new  to  educators.  A search  of  the  WWW  and  the  literature  yielded  few  resources  or  research  that 
addressed  the  integration  of  Mindtools  in  the  practical  teaching  and  learning  environment,  the  school.  There  were  numerous 
broken  links,  which  were  meant  to  link  to  the  pages  relevant  to  Mindtools,  all  over  the  Internet.  In  addition,  there  were 
misconceptions  or  misunderstandings  about  Mindtools  in  current  Internet  resources.  Teachers  have  access  to  few  resources  to 
assist  them  in  designing  lesson  that  incorporate  Mindtools  and  have  little  opportunity  to  get  familiar  with  using  Mindtools  in  their 
classrooms.  It  is  confusing  and  frustrating.  In  order  to  help  educators  to  implement  Mindtools,  a computer-based  tool  for 
selecting  computer  Mindtools  is  being  designed.  This  tool  would  utilize  a database  web  server  to  provide  educators  examples  or 
suggestions  according  to  the  educators'  specific  needs  and  inquiries.  This  easy  to  use  tool  will  assist  educators  to  include  the 
utilization  of  Mindtools  in  the  learning  environment  which  should  ultimately  increase  the  critical  thinking  skills  of  the  students. 
This  session  will  discuss  and  explore  the  development  of  this  selecting  tool,  how  it  is  evolved. 

Selecting  media  for  instruction  (Reiser  & Gagne,  1983) 

A number  of  media  selection  models  have  been  introduced  over  the  last  few  decades  (Anderson,  1976;  Briggs  and  Wager, 
1981;  Gagne  and  Briggs,  1979;  Cropper,  1976;  Kemp,  1980;  and  Reiser  and  Gagne,  1983).  These  models  were  designed  to  assist 
educators  and  trainers  in  selecting  the  most  appropriate  medium  for  an  instructional  situation.  A common  assumption 
undergirding  these  models  was  that  instruction  is  based  on  a transmission  model  of  learning  in  which  an  instructional  medium 
(e.g.,  computer,  teacher,  and  television)  delivers  or  transmits  an  instructional  message  to  a learner.  This  assumption  was  perhaps 
best  articulated  by  Reiser  and  Gagne  (1983)  when  introducing  their  media  selection  model  by  defining  media  as  " . . . the 
physical  means  by  which  an  instructional  message  is  communicated"  (p.  5).  They  further  stated  that  educators  and  trainers  are 
faced  with  the  " . . .problem  of  choosing  the  appropriate  media  to  deliver  an  instructional  message"  (p.  3).  The  focus  on  media  as 
the  communicator  or  deliverer  of  an  instructional  message  is  consistent  with  many  traditional  models  of  instruction  (Salomon, 
Perkins,  & Globerson,  1991).  However,  this  assumption  is  often  inadequate  for  educators  interested  in  selecting  media  for 
constructivist  learning  environments.  This  assumption  is  what  Jonassen  would  call  "learn  from  computers."  The  leaner  under 
this  assumption  merely  receives  instructional  massages  from  the  media  and  learns  with  a passive  role.  Neither  the  interaction  nor 
the  partnership  with  the  media  is  encouraged  or  promoted. 

Discrepancies  of  Reiser  & Gagnes'  assumption  for  constructivist  learning  environment 

Based  on  the  assumption  of  media  as  " . . . the  physical  means  by  which  an  instructional  message  is  communicated”  (p.  5), 
Reiser  & Gagnes’  media  selection  model  implies  that  media  are  merely  delivery  tools  not  cognitive  tools.  There  are  several 
discrepancies  in  applying  Reiser  & Gagnes'  media  selection  model  to  a constructivist  learning  environment.  First  of  all.  Reiser  & 
Gagnes'  media  selection  model  the  model  focuses  on  selecting  media  based  on  the  transmission  attributes  of  media  (e.g.,  motion 
and/or  sound  capabilities).  Selection  of  media  as  cognitive  tools  to  engage  students’  thinking  is  not  addressed  in  this  model. 

Second,  Reiser  & Gagnes’  media  selection  model  focuses  on  providing  sound  guidelines  to  instructional  designers.  In  the 
introduction  part  of  Reiser  & Gagnes'  media  selection  model,  it  indicated  that  it  is  "aimed  primarily  at  instructional 
designers... who  design  and  develop  instructional  materials"  (p.  7).  In  other  words,  it  is  not  primarily  tailored  for  classroom 
teachers  or  educators  who  implement  their  teaching  or  instructional  materials  on  daily  or  weekly  basis  in  a practical  learning 
environment,  nor  does  it  meet  the  needs  of  a constructivist  learning  environment.  Although  claimed  to  be  of  value  to  instructors 
and  classroom  teachers,  their  model  is  only  for  selecting  "media  for  supplementing  or  supplanting  some  of  their  in-class  teaching 
activities"  (p.  7).  In  a constructivist  learning  environment,  media  should  not  be  just  for  supplementing  or  supplanting  teaching 
activities  but  engaging  students  to  think  critically. 

Third,  users  need  to  cross  off  media  which  are  not  feasible  in  the  situation  for  which  instruction  is  being  designed  after  they 
have  used  Reiser  & Gagnes'  media  selection  model  to  identify  the  appropriate  media  for  them.  In  other  words,  users  have  to 
identify  the  appropriate  media  regardless  to  the  feasibility  or  availability  at  the  first  place.  At  the  end,  users  might  not  be  able  to 
use  any  of  the  selected  media  that  Reiser  & Gagnes'  media  selection  model  suggests.  Moreover,  users  have  to  consider  the 
necessity  and  the  comparative  costs  to  make  final  choices.  Consequently,  using  Reiser  & Gagnes'  media  selection  model  is  still  a 
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complicated  decision  making  process.  It  is  time-consuming  for  classroom  teachers  and  educators  to  implement  Reiser  & Gagnes' 
media  selection  model.  It  is  another  extra  workload  to  their  already  tight  and  busy  schedules. 

Descriptions/constructions  of  our  selecting  tool 

Mindtools  are  cognitive  tools.  Students  work  with  Mindtools  in  terms  of  partnerships.  Consequently,  students  and 
Mindtools  empower  each  other’s  capabilities,  especially  the  thinking  and  learning  of  students  and  the  potentials  of  Mindtools. 
Mindtools  are  readily  available  and  can  be  used  in  all  kinds  of  content  areas.  As  a result,  Mindtools  are  the  best  “media”  to  the 
constructivist  learning  environment.  In  order  to  promote  the  uses  of  Mindtools  and  help  educators  who  are  interested  in  applying 
Mindtools  to  their  teaching,  the  idea  to  construct  an  extendable  and  applicable  Mindtool  database  resource  was  developed.  In 
consideration  of  the  availability  and  the  ease  to  use  of  the  WWW  to  every  educator,  it  was  decided  to  build  a database  web  server 
for  selecting  computer-based  Mindtools.  This  tool  is  intended  to  help  educators  make  informed  decisions  about  the  most 
appropriate  Mindtool  for  students  to  partner  with  in  specific  instructional  situations.  The  tool  interacts  with  and  asks  educators  to 
specify  the  critical,  creative,  or  complex  thinking  skills  they  want  students  to  develop  within  a given  content  domain.  Once  this 
information  is  entered,  the  educator  is  given  a prioritized  list  of  Mindtools  and  activities  that  could  effectively  be  used  to  help 
students  develop  the  higher-order  thinking  skills  specified.  The  selection  tool  employs  five  steps  to  guide  users  in  the 
identification  of  appropriate  Mindtools.  Users  will  first  be  asked  to  specify  content  area  and  the  grade  level.  Next,  users  need  to 
select  concepts,  the  desired  critical  cognitive  thinking  skills  of  students  to  engage,  and  check  the  availability  of  their  computer 
applications,  all  from  pre-decided  menus.  Then,  based  on  the  input  of  users,  a prioritized  list  of  Mindtools  with  short  descriptions 
of  example  activities  and  guidelines  to  construct  new  activities  is  provided.  Items  on  the  prioritized  list  are  links  connecting  to 
corresponding  pages  of  examples  stored  in  the  database  web  server  for  selecting  Mindtools. 


Discussion  and  Co  nclusion 

The  tool  is  still  under  development.  It  is  not  a sound  product  yet  at  this  point.  It  is  the  idea  that  counts  for  the  purpose  of 
helping  educators  to  select  appropriate  Mindtools  for  their  constructivist  learning  environments.  However,  this  presentation  will 
still  show  how  this  tool  works  practically.  Sample  pages  will  be  provided  to  illustrate  the  constructions  of  the  idea  of  this 
Mindtool  database  web  server. 

The  functions  to  identify  inputs  are  embedded  on  web  pages.  Queries  are  sent  to  the  database  server  in  terms  of  submitting 
web  forms.  Data  are  retrieved  from  the  collection  of  the  database.  Then,  results  are  returned  and  shown  on  users’  web  pages 
using  server  side  scripting  language.  In  other  words,  in  order  to  develop  this  database  web  server  for  selecting  Mindtools,  a 
database  server,  a server  side  scripting  language,  and  a Web  server  are  required.  In  this  case,  we  are  using  MySQL  as  the 
database  server  that  serves  databases,  PHP  as  the  server  side  scripting  language  that  connects  a database  server  to  fetch 
information  and  sends  the  information  to  users’  web  browsers  as  designed  web  pages,  and  the  Unix  Web  server  of  Texas  Tech 
University.  MySQL  and  PHP  are  chosen  because  of  their  free  distributions  and  their  availabilities.  Moreover,  MySQL  and  PHP 
are  compliant  with  the  Unix  Web  server  of  Texas  Tech  University. 

The  Mindtool  database  is  built  based  on  Jonassen’s  book  “Computers  as  Mindtools  for  Schools.”  We  used  Jonassen’s  terms 
to  specify  the  types  of  Mindtools  as  well  as  the  computer  application  software.  The  terms  for  the  desired  critical  cognitive  skills 
are  put  together  based  on  various  resources  regarding  to  critical  thinking.  The  lists  of  content  and  subject  areas  are  relevant  to 
what  is  taught  at  secondary  schools.  However,  building  the  database  is  much  of  work.  While  working  on  sample  activities,  we 
confront  a serious  problem  regarding  to  content  and  subject  areas.  It  will  be  very  difficult  and  unreasonable  to  build  the  content 
and  subject  specified  activities  without  consulting  with  real  content  experts.  It  needs  content  experts  to  design  and  create  lesson 
plans  and  activities. 

Moreover,  if  the  database  we  are  building  is  a closed  collection,  the  resources  for  educators  who  are  interested  in  Mindtools 
will  be  unexpandable  and  remain  limited.  In  order  to  have  the  database  possess  the  capability  of  ongoing  expansion,  we  have 
decided  to  add  a function,  other  than  guidelines  to  help  users  to  create  lesson  plans  and  activities,  which  allows  any  potential  user 
to  contribute  their  creative  activities  to  the  collection  of  the  database.  We  believe  that  once  the  user  is  more  familiar  with  the 
application  of  Mindtools,  they  will  be  more  ready  to  create  their  own  or  to  contribute  their  ideas  to  the  database.  The 
contribution  function  is  tailored  for  them.  Nevertheless,  not  every  contribution  is  published  or  added  to  the  database 
automatically.  To  prevent  false  example  activities  or  misconceptions  of  applying  Mindtools,  every  contribution  is  sent  to  a 
reviewer  first  prior  to  be  a part  of  the  database.  Full  considerations  are  necessary  and  also  take  much  time  and  efforts.  In  other 
words,  the  development  of  this  Mindtool  database  web  server  requires  all  kinds  of  participations  on  technical  solutions,  content 
specified  expertise,  ongoing  reviewing,  continual  revision,  and  user  engagements.  This  selection  tool  for  Mindtools  is  meant  to 
be  a collaborative  work.  By  working  with  the  collaboration  of  all  aspects,  this  tool  will  work  more  effectively. 

It  is  not  possible  to  construct  this  tool  soundly  without  involving  intended  users.  That  is  one  of  the  reason  that  the  tool  is  to 
be  on  the  Internet.  This  tool  is  made  to  be  a share  resource  of  Mindtools. 
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Abstract 

The  purpose  of  this  study  is  to  investigate  learning  effects  of  on-line  peer -support  for  generating  critical  questions  and 
counter  arguments  in  small  group  on-line  discussion.  The  on-line  peer-support  includes  descriptions  of  what  to  do  and  generic 
and  domain  specific  examples  of  questions.  About  forty  five  students  were  recruited  from  an  on-line  introductory  class  on 
turfgrass  management  offered  by  a land-grant  university.  A field  experimental  time  series  design  with  control-group  was 
employed.  Data  were  collected  from  five  discussion  sessions,  ten  open-ended  essay  exams,  and  three  multiple-choice  exams 
through  a semester.  The  results  indicated  that  the  peer -challenge  guidance  helped  learners  to  generate  more  challenges 
(F=2.465,  p < .05).  But,  it  did  not  improve  the  quality  of  challenges.  The  increased  quantity  of  challenges  alone  might  be  not 
sufficient  to  activate  learners'  reflection  and  critical  thinking  or  to  improve  meaningful  interactions.  Consequently,  it  did  not 
influence  learning,  performance  in  multiple-choice  questions  on  memory  and  comprehension  and  open-ended  essay  questions. 

Background  to  the  Problem 

On-line  discussion  has  become  one  of  the  most  popular  Srategies  used  in  on-line  distance  education  as  well  as  on-site 
education  at  the  college  level.  For  example,  most  on-line  education  support  systems  (e.g.  WebCT)  include  various  forms  of 
computer  supported  communication  functions,  such  as  list-serve  and  bulletin  board  systems  (Abowd,  da  Graca  Pimentel, 
Ishiguro,  Kerimbaev,  & Guzdial,  1999;  Hsi  & Hoadley,  1997).  In  fact,  many  on-line  instructors  use  these  functions  to  encourage 
learners  to  be  involved  in  discussions  about  target  topics  (Berge,  1997;  2000).  Meaningful  discussion  helps  learners  to  construct 
their  own  knowledge  by  providing  several  cognitive  benefits  such  as  articulation,  where  learners  articulate  their  understanding, 
perspectives,  or  opinions;  cognitive  conflicts,  where  learners  reflect  on  new  knowledge  to  justify  or  defend  conflicting  positions; 
and  co-construction,  where  learners  share  and  refine  meaning  with  peers  in  a social  context  (Crook,  1994;  Jonassen,  Peck,  & 
Wilson,  1999;  Koschmann  et  al.,  1996;  Tao  & Gunstone,  1999b).  Although  on-line  discussion  has  been  used  largely  with  these 
expectations  for  learning  benefits,  the  actual  effects  are  unclear.  In  cases  where  simple  question-answer  cycles  are  employed, 
students  do  not  become  actively  involved  in  critical  thinking  processes.  These  non-thoughtful  interactions  are  not  sufficient  to 
promote  active,  knowledge  construction. 

One  possible  reason  for  the  lack  of  reflection  during  on-line  discussions  is  that  students  do  not  know  what  to  ask  or  howto 
ask  questions  (Miyake  & Norman,  1979;  van  der  Meij,  1990).  Peer  interactions  can  be  initiated  when  learners  raise  thoughtful 
questions  or  provide  critical  feedback;  however,  in  order  to  propose  important  questions  or  thoughtful  feedback,  question-askers 
need  to  have  a certain  level  of  domain  knowledge  and  to  activate  metacognitive  skills  such  as  reflection,  monitoring  and 
evaluation  (Dillon,  1986;  Miyake  & Norman,  1979;  Palincsar  & Brown,  1984;  van  der  Meij,  1990;  Wong  1985).  Unfortunately, 
novice  learners  who  begin  to  explore  a new  domain  are  often  limited  in  those  metacognitive  skills,  so  they  can  neither  ask  the 
right  questions  nor  generate  productive  feedback.  It  is  a "metacognitive  knowledge  dilemma"  (Land,  2000),  whereby  effective 
monitoring  and  reflection  is  linked  to  having  some  prior  domain  knowledge  (Gamer  & Alexander,  1989). 

This  dilemma  provides  an  essential  need  for  developing  on-line  instructional  strategies  that  can  guide  meaningful  on-line 
discussion  between  or  among  peers  (e.g.  Abowd  et  al.,  1999;  Scardamalia  & Bereiter,  1996a).  Specifically,  learners'  generation 
of  questions  or  feedback  needs  to  be  supported  to  lead  meaningful  discussions  (Brown,  1989;  King,  1994;  King  & Rosenshine, 
1993;  Palincsar  & Brown,  1984;  Scardamalia  & Bereiter,  1991;  van  der  Meij,  1998). 

Purpose  of  the  Study 

This  Study  develops  a framework  intended  to  overcome  the  metacognitive  dilemma  and  to  facilitate  effective  peer 
interactions  in  on-line  discussion.  This  framework  assumes  that  novice  students  who  lack  domain  knowledge  and  cognitive  skills 
can  be  supported  in  generating  meaningful  interactions  at  an  early  stage  of  learning  (King  & Rosenshine,  1993;  Palincsar  & 
Brown,  1984).  The  resulting  questions  and  feedback  in  turn  can  enhance  peers'  metacognition,  such  as  reflecting  and  monitoring, 
which  allows  them  to  refine  and  restructure  their  domain  knowledge  (Piaget,  1985;  Webb  & Palincsar,  1996).  This  framework  for 
peer-challenge  support  has  three  assumptions.  First,  on-line  support  for  students  to  generate  meaningful  challenges  can  increase 
the  quality  of  students'  questions  and  feedback  by  providing  externalized  support  for  metacognition  (Palincsar  & Brown,  1984). 
Second,  in  order  to  receive  learning  benefits  from  on-line  discussion,  such  as  articulation,  cognitive  conflicts,  and  co-construction 
of  knowledge  (Crook,  1994),  this  peer-challenging  strategy  should  guide  specific  types  of  challenges  that  facilitate  these  learning 


activities  (Forman  & Cazden,  1985).  Thus,  effective  types  of  challenges  should  be  questions  that  seek  missing  information  from 
learners'  explanations,  counter  arguments  that  contradict  learner's  opinions,  and  more  systemic  questions  such  as  hypothetical 
questions  that  force  learners  to  consider  complex  contexts  and  various  perspectives.  Third,  once  the  quality  of  peer-generated 
challenges  is  increased,  meaningful  cycles  of  verbal  interactions  should  be  initiated.  When  learners  receive  critical,  valuable, 
reasonable,  and  sophisticated  questions  or  challenges  from  their  peers,  those  challenges  and  interactions  should  magnify  learner's 
cognitive  dissonance  and  trigger  a conscious  cognitive  process  to  re-construct  and  enhance  existing  understanding.  Thus,  the 
purpose  of  this  study  is  to  test  this  peer-challenge  support  framework  by  investigating  the  effects  of  on-line  support  for  peer 
challenges  on  discussion  activities  and  learning  of  college  students. 

Research  Questions 

Question  1:  Does  the  use  of  on-line  guidance  for  generating  effective  peer-challenges  affect  students'  on-line  challenging 
behaviors  such  as  types  of  challenges  generated,  clarity  of  challenges,  and  clarity  of  rationale  in  challenges  during  small  group 
on-line  discussion? 

Question  2;  Does  the  use  of  on-line  guidance  for  generating  effective  peer-challenges  affect  students'  on-line  discussion 
activities  (frequency  of  interactions,  threaded  responses,  and  off-task  interactions)  during  small  group  on-line  discussion? 

Question  3:  Does  the  use  of  on-line  guidance  for  generating  effective  peer-challenges  during  small  group  on-line  discussion 
affect  students'  performance  in  memory  and  comprehension? 

Question  4:  Does  the  use  of  on-line  guidance  for  generating  effective  peer-challenges  affect  changes  in  students'  performance 
of  open-ended  essay  questions  during  small  group  on-line  discussion? 

Methods 

Participants 

About  forty  five  students  were  recruited  from  an  on-line  introductory  class  on  turfgrass  management  during  the  2001  spring 
semester,  which  is  regularly  offered  from  a land-grant  university  in  the  northeastern  United  States.  The  audience  for  this  on-line 
course  has  no  restriction  in  their  location  and  time.  They  can  be  full-  or  part-  time  students  working  toward  either  a degree  or  a 
certificate.  In  the  2001  spring  semester,  the  majority  of  the  participants  were  part-time  male  students  who  pursued  a certificate  for 
turfgrass  management.  The  participants  were  randomly  assigned  to  a small  group  of  five  to  six  members.  Then  those  small 
groups  were  randomly  assigned  to  either  experimental  or  control  group. 

Discussion  tool  and  intervention 

An  on-line  discussion  tool  used  for  this  class  is  called  Collaboration  and  Negotiation  Tool  for  Case-Based  Learning  (Conet-C 
version  1)  and  was  designed  by  the  authors. 

Guidelines  for  effective  peer-challenges  were  embedded  into  the  discussion  tool  to  facilitate  learners'  generation  of  three 
different  types  of  challenges  to  their  peers'  initial  answers:  clarification  questions,  counter  arguments,  and  context-  or 
perspective-oriented  questions. 

Clarification  questions  are  peer-generated  questions  seeking  additional  information  from  learners'  initial  answers  for 
clarifying  or  elaborating  the  learners'  ideas.  These  questions  identify  missing  information,  indicate  unclear  parts,  and  detect 
errors  in  learners'  initial  explanations  on  given  topics.  This  type  of  peer-challenge  could  facilitate  learners  to  elaborate/articulate 
their  idea,  explain  these  idea  clearly,  and  correct  their  partial  misunderstanding  (Koschmann  et  al.,  1996;  Tao  & Gunstone, 
1999a). 

Counter  arguments  are  peer-generated  opinions  expressing  disagreements  with  learners'  initial  ideas.  Unlike  clarification 
questions,  these  opinions  identify  major  differences  between  peers'  and  users’  understanding  on  given  topics.  This  type  of  peer- 
challenge  generates  explicit  (Dgnitive  conflicts  which  could  encourage  learners  to  justify  their  positions,  reconstruct  their 
misconceptions,  and  negotiate  their  understandings  (Tao  & Gunstone,  1999a). 

Context- /perspective-oriented  questions  are  hypothetical  questions  changing  critical  factors  in  given  problem  situations  or 
considering  different  perspectives  on  the  problems.  Unlike  clarification  questions  or  counter  arguments,  these  challenges  do  not 
indicate  any  specific  problem  with  learners'  responses.  Instead,  these  challenges  could  stimulate  learners  to  systemically  think 
about  dynamic  aspects  of  the  problems  beyond  the  levels  of  the  assigned  questions.  This  type  of  peer-challenge  could  facilitate 
learners  to  generate  predications  and  explanations. 

Procedure 

Small  groups  of  students  were  asked  to  answer  the  same  set  of  five  or  six  open-ended  essay  questions  in  five  sessions  of  on- 
line discussion  throughout  the  semester.  At  each  session  of  discussion,  students  in  each  group  were  asked  to  answer  their 
assigned  question  and  post  their  answer  within  a week,  so  the  initial  answers  were  available  for  group  members  to  review.  Then, 
each  student  was  required  to  ask  questions  or  to  provide  different  opinions  at  least  two  times  to  group  members  in  each 
discussion  session.  At  the  same  time,  each  student  was  asked  to  answer  peers'  questions  or  counter  opinions  about  his/her  initial 
answer.  After  completing  each  on-line  discussion  (approx.  1 week),  each  student  revised  his/her  initial  answer  and  submitted  the 
final  answer  to  the  instructor. 

During  the  first  two  discussion  sessions  (pre-observation),  there  was  no  treatment  given  to  the  groups.  After  finishing  the 
second  discussion  session,  all  students  took  a multiple-choice  exam  (pretest  for  memory  and  comprehension)  in  their  local  area 
administered  by  local  librarians.  In  the  third  and  forth  discussion  sessions  (treatment  observation),  the  guidance  for  effective 
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peer-challenges  was  presented  to  only  the  experimental  group  through  the  on-line  discussion  tool.  After  finishing  the  forth 
discussion  session,  another  multiple-choice  exam  (post-test)  was  administered.  In  the  fifth  discussion  (post-observation)  no 
treatment  was  given  to  the  groups.  After  this  last  discussion,  the  last  multiple-choice  exam  (delayed  and  transfer  test)  was 
administered. 

During  each  discussion  session,  students'  on-line  verbal  interactions  and  their  initial  and  final  answers  for  the  given  open- 
ended  essay  questions  were  recorded  in  a computer  database  for  later  analysis. 

Independent  variables  measured 

The  followings  are  a list  of  independent  variables  measured. 

• Challenging  activities 

• Challenge  types:  frequency  of  peer-generated  clarification  questions,  counter  arguments,  and  context -/perspective 
oriented  questions. 

• Challenge  clarity:  how  clearly  peer-generated  questions  or  disagreement  points  are  described. 

• Challenge-rationale  clarity:  how  clearly  rationales  for  challenges  are  justified. 

• Discussion  activities 

• Interactions:  frequency  of  all  postings 

• Threaded  responses:  the  number  of  postings  under  one  issue 

• Off-task  discussions:  frequency  of  off-task  postings 

• Learning  outcomes 

• Multiple-choice  tests  of  memory  and  comprehension 

• Open-ended  essay  questions 

Results 

Effects  on  on-line  challenging  behaviors 

The  frequency  of  three  types  of  peer-generated  challenges  (clarification  questions,  counter  arguments,  and  context - 
/perspective-oriented  questions)  were  counted  from  the  five  discussion  sessions  respectively.  According  to  the  results  of  ANOVA 
repeated  measures  on  the  total  frequency  of  three  types  of  challen^  generated  during  discussion  sessions  (see  Table  1),  there  was 
a statistically  significant  interaction  effect  for  time  and  groups  (F  = 2.465,  p < .05).  The  result  indicates  that  the  treatment  group 
generated  more  challenges  than  the  control  group  during  and  after  treatment  sessions  (see  figure  1).  In  the  post-hoc  analysis, 
however,  it  failed  to  find  specific  time  points  and  types  of  challenges  that  were  attributable  to  the  significant  interaction  effect. 


Table  1.  Results  of  ANOVA  on  total  frequency  of  three  types  of  challenges 


Source 

Sum  of 
Squares 

df 

Mean 

Square 

F 

Sig. 

Time* 

3.995 

4 

.999 

.466 

.761 

Time  X Group 

21.137 

4 

5.284 

2.465 

.048 

Error  (Time) 

317.229 

148 

2.143 

Figure  1.  Total  frequency  of  three  types  of  challenges  between  control  and  treatment  groups  across  five  discussion  sessions. 
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In  order  to  obtain  the  scores  for  challenge  clarity  and  challenge-rationale  clarity,  the  challenges  posted  during  discussion 
sessions  were  also  evaluated  by  two  judges  according  to  rubrics.  The  results  of  ANOVA  with  repeated  measures  did  not  show 
any  significant  differences  between  two  groups  in  both  challenge  clarity  (F  = .608,  p.  < .658)  and  challenge -rational  clarity  (F  = 
.356,  p.  < .839)  across  discussion  sessions. 

Effects  on  on-line  discussion  activities 

The  frequency  of  all  postings  and  off-task  postings  were  counted  throughout  the  five  discussion  sessions.  Also,  the  average 
number  of  postings  under  each  issues  (threaded  discussions)  was  calculated  by  dividing  the  number  of  issues  into  the  number  of 
all  postings.  The  results  of  ANOVA  with  repeated  measures  of  the  on-line  discussion  activities  did  not  show  any  significant 
differences  between  the  two  groups  across  all  discussion  sessions  in  the  frequency  of  postings  (F  = .832,  p.  = .507)  or  off-task 
postings  (F  = 1.227,  p < .302).  In  addition,  the  results  of  threaded  discussions  calculated  from  each  small  group  did  not  show  any 
consistent  patterns  of  the  curve  in  the  average  postings  under  each  issue.  This  indicates  that  there  are  no  clear  differences 
between  the  two  groups  in  threaded  discussions. 

Effects  on  memory  and  comprehension  tests 

The  multiple-choice  scores  on  memory  and  comprehension  from  pre-,  post-,  delayed-,  and  transfer  tests  were  collected  and 
analyzed  by  ANOVA  repeated  measures.  The  ANOVA  results  did  not  show  any  significant  difference  between  the  two  groups  in 
the  scores  across  all  tests  (F  = .060,  p = .981). 

Effects  on  changes  in  open-ended  essay  questions 

The  initial  and  final  answers  of  students  on  the  open-ended  essay  questions  in  each  discussion  session  were  collected  and 
evaluated  by  two  judges  according  to  given  rubrics.  The  ANOVA  results  did  not  show  any  significant  difference  between  the  two 
groups  in  the  gain  scores  made  from  initial  to  final  answers  across  all  discussion  sessions  (F  = 1 .101,  p = .358). 

Discussions 

In  summary,  the  results  indicated  that  the  peer-challenge  guidance  helped  learners  to  generate  more  challenges.  But,  this  did 
not  improve  the  quality  of  challenges.  The  increased  quantity  of  challenges  alone  mi^t  not  be  sufficient  to  activate  learners’ 
reflection  and  critical  thinking  or  to  improve  meaningfiil  interactions.  Consequently,  this  did  not  influence  learning  or 
performance  on  multiple-choice  questions  of  memory  and  comprehension  and  open-ended  essay  questions  on  higher  order 
thinking. 

One  likely  reason  for  the  failure  of  the  treatment  to  improve  the  quality  of  challenges  may  be  that  students  did  not  frequently 
refer  to  the  on-line  guidance;  it  was  not  strongly  emphasized  by  either  the  instructor  or  the  interface  of  the  on-line  discussion  tool. 

Instead,  it  was  simply  recommended  by  the  instructor  only  twice  during  the  treatment  discussion  session  through  the 
instructor’s  announcement  board.  Thus,  students  may  not  have  paid  attention  to  the  guidance  during  discussions.  In  addition,  the 
interface  design  of  the  discussion  tool  may  not  have  strongly  encouraged  students  to  look  at  and  use  the  guidance;  students  in  the 
treatment  group  were  required  to  voluntarily  seek  this  guidance  by  clicking  on  the  guidance  icon.  Most  students  reported  on  a 
survey  collected  at  the  end  of  the  semester  that  they  referred  to  the  guidance  only  one  or  two  times  during  the  treatment 
discussion  sessions. 

Possible  solutions  to  these  limitations  involve  changes  to  the  interface  design  and  class  administration,  although  these  might 
generate  additional  problems.  The  guidance,  for  example,  could  be  re-designed  to  open  automatically  whenever  students  open  the 
discussion  window.  But,  students  might  want  more  control  without  being  “forced”  to  view  the  guidance  for  every  posting. 
Another  possible  solution  might  be  to  provide  a template  for  generating  effective  challenges  that  required  them  to  go  through  all 
steps  in  order  to  post  their  challenges.  If,  however,  we  use  more  sophisticated  interfaces  for  on-line  guidance,  then  we  may  lose 
the  feasibility  of  large-scale  use  of  on-line  strategies  because  instructors  might  find  it  difficult  to  incorporate  such  interface- 
dependent  strategies  into  the  generic  discussion  tools  they  use  currently.  Lastly,  instructors  might  strongly  emphasize  the  use  of 
on-line  guidance  to  students  by  sending  messages  to  them  more  frequently. 

Although  the  current  study  did  not  find  significant  effects  of  using  on-line  guidance  on  discussion  activities  and  learning,  it 
showed  very  reasonable  results  indicating  that  the  quality  of  challenges  could  be  essential  for  meaningful  discussion  and  learning. 
Further  studies  need  to  be  focused  on  finding  ways  to  improve  the  quality  of  challenges  and  need  to  be  tested  again  to  find 
possible  learning  effects  of  the  on-line  guidance. 

References 

Abowd,  G.  D.,  da  Graca  Pimentel,  M.,  Ishiguro,  Y.,  Kerimbaev,  B.,  & Guzdial,  M.  (1999).  Anchoring  discussion  in  lecture: 

An  approach  to  collaborativelv  extending  classroom  digital  media.  Paper  presented  at  the  Computer  Supportive  Collaborative 
Learning  '99. 

Brown,  A.  L.  (1989).  Guided,  cooperative  learning  and  individual  knowledge  acquisition.  In  L.  B.  Resnick  (Ed.),  Knowing, 
learning,  and  instruction:  Essays  in  honor  of  Robert  Glaser  (pp.  393-451).  Hillsdale,  NJ:  Lawrence  Erlbaum  Associates. 

Chong,  S.-M.  (1998).  Models  of  asynchronous  computer  conferencing  for  collaborative  learning  in  large  college  classes.  In 
C.  J.  Bonk  & K.  S.  King  (Eds.)  (pp.  157-182).  Mahwah,  NJ:  Lawrence  Erlbaum  Associates. 

Crook,  C.  (1994).  Computers  and  the  collaborative  experiences  of  learning  London  ; New  York:  Routledge. 


’2  81 


Dillon,  J.  T.  (1986).  Student  questions  and  individual  learning.  Educational  Theory.  36.  333-341. 

Forman,  E.  A.,  & Cazden,  C.  B.  (1985).  Exploring  Vygotskian  perspectives  in  education:  The  cognitive  value  of  peer 
interaction.  In  J.  V.  Wertsch  (Ed.),  Culture,  communication,  and  cognition:  Vygotskian  perspectives  (pp.  323-347).  New  York: 
Cambridge  University  Press. 

Gamer,  R.,  & Alexander,  P.  A.  (1989).  Metacognition:  Answered  and  unanswered  questions.  Educational  Psychologist.  24. 
143-158. 

Jonassen,  D.  H.,  Peck,  K.  L.,  & Wilson,  B.  G.  (1999).  Learning  with  technology:  A constructivist  perspective.  Uper  Saddle 
River,  NJ:  Prentice-Hall,  Inc. 

King,  A.  (1994).  Guiding  knowledge  construction  in  the  classroom:  Effects  of  teaching  children  how  to  question  and  how  to 
explain.  American  Educational  Research  Journal.  3K21  338-368. 

King,  A.,  & Rosenshine,  B.  (1993).  Effects  of  guided  cooperative  questioning  on  children's  knowledge  construction.  Journal 
of  Experimental  Education.  6 1 . 127-148. 

Koschmann,  T.,  Kelson,  A.  C.,  Feltovich,  P.  J.,  & Barrows,  H.  S.  (1996).  Computer-supported  problem-based  learning:  A 
principled  approach  to  the  use  of  computers  in  collaborative  learning.  In  T.  Koschmann  (Ed.),  CSCL:  Theory  and  practice  of  an 
ememrging  paradigm  (pp.  83-124).  Mahwah:  New  Jersey:  Lawrence  Erlbaum  Associates. 

Land,  S.  M.  (2000).  Cognitive  requirements  for  learning  with  open-ended  learning  environments.  Educational  Technology: 
Research  and  Development.  48(3).  61-78. 

Miyake,  N.,  & Norman,  S.  A.  (1979).  To  ask  a question,  one  must  know  enough  to  know  what  is  not  known.  Journal  of 
Verbal  Learning  and  Verbal  Behavior.  1 8.  357-364. 

Palincsar,  A.  S.,  & Brown,  A.  L.  (1984).  Reciprocal  teaching  of  comprehension-fostering  and  monitoring  activities. 

Cognition  and  Instruction.  1.  117-175. 

Piaget,  J.  (1985).  The  equilibrium  of  cognitive  structures:  The  central  problem  of  intellectual  development  (T.  Brown  & K.  L. 
Thampy,  Trans.).  Chicago:  Universityh  of  Chicago  Press. 

Scardamalia,  M.,  & Bereiter,  C.  (1991).  Higher  levels  of  agency  for  children  in  knowledge  building:  A challenge  for  the 
design  of  new  knowledge  media.  Journal  of  the  Learning  Science.  K 1).  37-68. 

Scardamalia,  M.,  & Bereiter,  C.  (1996).  Adaptation  and  understanding:  A case  for  new  cultures  of  schooling.  In  S.  Vosniadou 
& E.  De  Corte  & R.  Glaser  & H.  Mandl  (Eds.),  International  perspectives  on  the  design  of  technology -supported  learning 
environments.  Hillsdale,  NJ:  Lawrence  Erlbaum  Associate. 

Tao,  P.  K.,  & Gunstone,  R.  F.  (1999a).  Conceptual  change  in  science  through  collaborative  learning  at  the  computer. 
International  Journal  of  Science  Education.  21('n.  39-57. 

Tao,  P.  K.,  & Gunstone,  R.  F.  (1999b).  The  process  of  conceptual  change  in  force  and  motion  during  computer-supported 
physics  instruction.  Journal  of  Research  in  Science  Teaching.  36(7).  859-882. 

van  der  Meij,  H.  (1990).  Question  asking:  To  know  that  you  do  not  know  is  not  enough.  Journal  of  Educational  Psychology. 
M3),  505-512. 

van  der  Meij,  H.  (1998).  The  great  divide  between  teacher  and  student  questioning.  In  S.  A.  Karabenick  (Ed.),  Strategic  help 
seeking:  Implications  for  learning  and  teaching  (pp.  195-218).  Mahwah,  NJ:  Lawrence  Erlbaum  Associates. 

Webb,  N.  M.,  & Palincsar,  A.  S.  (1996).  Group  processes  in  the  classroom.  In  D.  C.  Berliner  & R.  C.  Cafree  (Eds.), 

Handbook  of  Educational  Psychology  (pp.  841-873).  New  York:  Simon  & Schuster  Macmillan. 

Wong,  B.  Y.  L.  (1985).  Sblf-questioning  instructional  research:  A review.  Review  of  Educational  Research.  55.  227-268. 


82 


best  copy  available 


Model  of  Learner-Centered  Computer-Mediated  Interaction  for 
Collaborative  Distance  Learning 

C.  Candace  Chou 
University  of  Minnesota 


Abstract 

Interaction  research  in  distance  education  has  focused  mostly  on  learner -teacher  interaction  in  a learning  environment 
based  on  a behavior  is  t curriculum.  This  presentation  focuses  on  factors  contributing  to  learner -learner  interaction  in  a distance 
learning  course  based  on  learner -centered  and  collaborative  instructional  design.  The  proposed  model,  which  resulted  from 
research  on  patterns  of  learner  interaction  in  both  synchronous  and  asynchronous  computer -mediated  communication  modes, 
examines  factors  contributing  to  interaction  in  the  areas  of  learner  characteristics,  technology  attributes,  and  learning  activities. 


Introduction 

Interaction  research  provides  important  information  on  student  behaviors  in  distance  learning  environments  to  educators, 
researchers,  and  instructional  designers.  The  current  state  of  interaction  research  has  focused  mostly  on  the  quantitative  results  of 
inter-connected  messages  in  computer-mediated  communication  (CMC)  conferences.  Contributing  factors  to  interaction  such  as 
theoretical  principles  of  course  design  and  learning  contexts  are  largely  ignored.  While  various  virtual  learning  environments  and 
course  management  systems  are  being  introduced  to  the  distance  learning  community,  it  is  easy  to  loose  sight  on  the  pedagogical 
application  of  the  learning  systems.  Teachers  are  rushed  to  learn  various  state-of-art  instructional  technologies  but  not  given 
instructional  examples  or  time  to  develop  well-designed  instructional  materials  for  conducting  distance  learning  courses.  The 
issues  faced  by  the  educators  are  similar  to  that  of  a novice  pilot  being  rushed  to  drive  a commercial  airplane  without  going 
through  appropriate  training  via  flight  simulation.  More  research  on  how  these  systems  can  enhance  student  learning  and 
examples  of  best  practices  on  instructional  design  in  various  disciplines  are  needed  for  the  success  of  distance  education.  As 
interaction  has  been  identified  as  the  key  to  the  success  of  online  learning  by  researchers  (Gunawardena  et  al.,  1997),  this  study 
examines  patterns  of  online  interaction  and  the  types  of  instructional  design  that  would  enhance  online  interaction  via  both 
synchronous  and  asynchronous  communication. 

Learner- centered  computer-mediated  interaction  in  this  study  is  defined  as  reciprocal  communication  among  participants  of 
computer-mediated  learning  environments  that  emphasizes  learner  developments  in  cognition,  motivation,  and  social 
advancement  for  the  purpose  of  knowledge  construction  and  community  building.  Two  theoretical  principles  that  are  highly 
relevant  to  such  interaction  are  constructivism  and  learner-centered  principles  (LCPs).  A constructivist  distance  learning 
environment  places  emphases  on  knowledge  construction  through  interaction  with  the  physical  environment  and  through  the 
appropriation  of  culturally  relevant  activities.  In  other  words,  knowledge  is  co-constructed  with  peers  or  experts  and  through  the 
immersion  in  a social  context  (Bonk  & Cunningham,  1998).  The  Learner-Centered  Principles  were  developed  by  the  American 
Psychological  Association  (APA,  1997)  as  a framework  for  the  new  educational  approaches  that  stress  the  integration  of  the 
needs,  skills,  interests,  and  backgrounds  of  the  students  into  the  curriculum  planing.  The  following  section  on  literature  review 
examines  the  connection  between  these  principles  and  interaction. 

Literature  Review 

Interaction  is  often  emphasized  in  different  contexts  for  different  purposes,  such  as  construction  of  knowledge  (Gunawardena 
et  al.,  1997),  and  student  satisfaction  (Hackman  and  Walker,  1990).  Moore  (1989)  contributed  to  the  discussion  of  interaction  by 
providing  an  important  framework  of  three  types  of  interaction:  learner- content,  learner-instructor,  and  learner-learner 
interaction.  Moore  pointed  out  that  learner-content  interaction  is  a “defining  characteristics  of  education.”  As  a result  of  learner- 
content  interaction,  learners  achieve  intellectual  growth  or  changes  in  perspectives.  The  second  type,  learner- instructor 
interaction,  highlights  the  important  role  of  instructors.  In  addition  to  defining  the  learning  objectives,  activities,  and  materials, 
distance  instructors  are  also  responsible  for  revising  teaching  methods  and  providing  evaluation  as  their  students  progress  in  the 
process  of  learning.  The  third  type,  learner-learner  interaction,  takes  place  between  learner  and  other  learners  in  real-time  or 
delayed  time  and  is  not  restricted  to  the  presence  of  the  instructor.  This  “inter-learner  interaction”  can  foster  learning  through 
student  collaboration  and  knowledge  sharing.  Although  the  strategies  used  to  increase  learner-learner  interaction  vary  according 
to  the  characteristics  and  backgrounds  of  the  learners,  learner-learner  interaction  can  significantly  encourage  the  development  of 
student  expertise  in  different  subject  areas  and  promote  community  building. 

Hillman  et  al.  (1994)  added  a fourth  component  on  learner-interface  interaction  to  the  literature  discussion.  They  defined 
learner- interface  interaction  as  “a  process  of  manipulating  tools  to  accomplish  a task”  (p.  34).  They  stressed  the  importance  of 
learner- interface  interaction  because  the  “learner  must  interact  with  the  technological  medium  in  order  to  interact  with  the 
content,  instructor,  or  other  learners”  (p.  33).  The  learner  must  be  empowered  to  profess  the  necessary  skills  to  use  the 
communication  tools  and  feel  comfortable  with  the  learning  environment.  Good  interface  design  can  enhance  interactivity  and 
minimize  technological  barriers  to  online  learning. 
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These  definitions  also  highlight  the  importance  of  the  interrelationship  among  learners,  content,  and  technology.  In  the  design 
of  a learner-centered  distance  learning  course,  it  is  important  to  include  the  four  types  of  interaction  in  the  design.  Furthermore, 
learner- centered  principles  also  provide  “an  essential  framework  to  be  incorporated  in  new  designs  for  curriculum  and 
instruction,  and  assessment  systems  for  evaluating  educational  goal  attainments’"  (American  Psychological  Association,  1997,  p. 
1).  LCPs  consists  of  the  following  areas  of  learning:  cognitive  and  metacognitive  factors,  motivational  and  affective  factors, 
developmental  and  social  factors,  and  individual  differences. 

As  pointed  out  by  Wagner  and  McCombs  (1995),  these  principles  emerged  from  the  following  considerations: 

• Learners  operate  holistically  as  a function  of  intellectual,  emotional,  social  and  physical  characteristics. 

• The  learner’s  behavior  is  based  on  his  or  her  perceptions  and  evaluations  of  situations  and  events  from  a self- 
orientation that  interprets  meaning  and  value  relevant  to  personal  goals  and  interests. 

• The  learner’s  development  across  all  domains  of  functioning  is  never  static  and  unchanging,  but  is  a dynamic  growth 
process  that  serves  inherent  needs  for  mastery,  control  and  belonging,  (p,  34) 

In  the  context  of  distance  education,  the  infusion  of  LCPs  into  the  design  of  learning  systems  and  instructional  activities  has 
provided  enhanced  opportunities  for  educators  to  improve  teaching/leaming  activities.  Traditionally,  teachers  decide  what  the 
learners  need  to  know  by  devising  the  objectives,  instruction,  procedures,  curriculum,  materials,  and  evaluation.  Recently,  the 
increased  discussions  on  learner-centered  education  have  led  more  educators  to  recognize  the  values  of  empowering  the  students 
to  take  control  of  their  learning.  The  design  of  the  curriculum  takes  into  consideration  students’  background  and  prior  knowledge 
in  the  subject  matter. 

The  LCPs  provide  a solid  framework  for  the  new  educational  approaches.  However,  the  actual  implementation  is  subject  to 
individual  interpretation  and  still  requires  much  effort  for  educators  to  come  up  with  feasible  strategies.  Fortunately, 
constructivism  that  originates  from  philosophical  and  educational  theories  has  provided  viable  strategies  for  teaching  and 
learning.  Jonassen  et  al.  (1995),  long-time  advocates  of  constructivism  for  CMC  systems  in  distance  education,  argued: 

Constructivist  principles  provide  a set  of  guiding  principles  to  help  designers  and  teachers  create  learner- centered, 
collaborative  environments  that  support  reflective  and  experiential  processes.  Students  and  instructors  can  then  build 
meaning,  understanding,  and  relevant  practice  together  and  go  far  beyond  the  mere  movement  of  information  from 
instmctors’  minds  to  students’  notebooks,  (p.  8) 

According  to  Jonassen  et  al.  (1995),  the  four  constructivist  attributes  for  building  learning  systems  are  context,  construction, 
collaboration,  and  conversation.  Co«/e;t/ refers  to  the  “real  world”  scenario  in  which  learners  can  carry  out  learning  tasks  as  close 
to  the  real  world  as  possible.  Learning  tasks  should  have  real-world  implications  so  that  learners  can  connect  what  they  learn  in 
the  classroom  with  the  real  world.  Construction  concerns  knowledge  that  is  built  on  the  “active  process  of  articulation  and 
reflection  within  a context”  (Jonassen,  1995,  p.  8).  Learners  acquire  knowledge  better  when  they  can  link  their  own  experience 
with  the  learning  materials  and  make  sense  of  them.  Learners  master  a subject  better  in  the  process  of  constructing  knowledge. 
Collaboration  helps  learners  to  develop,  test,  and  evaluate  their  ideas  with  peers.  Learners  are  exposed  to  multiple  perspectives  in 
a problem-solving  case  and  then  come  to  a self-selected  conclusion  on  a particular  issue.  This  is  an  important  part  of  the  learning 
process.  Conversation  is  engaged  by  group  members  for  purposes  such  as  planning,  collaboration,  and  meaning  making.  It  is 
especially  important  for  distance  learning  because  most  communication  is  done  through  online  exchanges.  A successful 
conversation  will  lead  to  good  preparations  for  and  completion  of  online  tasks  (Jonassen  et  al.,  1995). 

Distance  learning  courses  that  are  based  on  the  LCPs  and  constructivism  have  demonstrated  enhanced  interaction  and 
academic  achievements.  The  next  section  reviews  the  process  of  implementing  the  theoretical  principles  into  the  instructional 
design  of  a distance  learning  course. 

Background 

The  course  for  this  research  is  an  upper  level  undergraduate  course  titled  “Theories  and  Applications  of  Computer-Mediated 
Communication  Systems”  offered  at  the  University  of  Hawaii.  The  main  objective  of  the  course  is  to  enrich  the  understanding  of 
CMC  systems  through  discussions  and  effective  use  of  various  CMC  systems.  The  course  design  is  based  on  the  following 
theoretical  principles: 

Principle  1:  Learner-centered  instructional  design:  The  course  design  considers  student  development,  especially  in  the 
following  areas:  cognitive,  meta-cognitive,  motivational,  affective,  social,  and  individual  differences.  Students  learn  to  monitor 
their  own  progress,  manage  the  course  content,  and  develop  expertise  in  a sub-domain  of  CMC  study.  Specific  examples  of 
learner- centered  instructional  activities  include  the  use  of  student  reflection  journals  for  the  purpose  of  metacognition  and 
student -centered  discussion  for  motivating  them  to  take  control  of  the  subject  matter. 

Principle  2:  Constructivist  activities:  The  emphasis  is  placed  on  student  acquisition  of  knowledge  via  active  involvement  with 
the  curriculum  rather  than  via  imitation  or  memorization  of  facts  or  course  content.  Specific  instructional  activities  based  on  the 
constructivist  principles  include  synchronous  and  asynchronous  discussions  for  co-construction  of  knowledge  and  project -based 
learning  for  real-world  application. 

Principle  3:  Small  group  cooperative  learning:  Students  collaborate  on  tasks  in  small  groups  to  accomplish  a set  of  predefined 
learning  objectives  and  to  advance  their  knowledge  in  a domain.  Emphases  are  placed  on  community  building  and  knowledge 
sharing.  They  equally  share  the  responsibilities  of  the  assigned  tasks  and  semester  projects.  At  the  end  of  each  term,  they 
demonstrate  the  ability  to  accomplish  the  task  on  an  individual  base. 

The  course  for  this  study  was  conducted  through  a number  of  text  -based  (WebCT  chat  and  ICQ),  audio-video  conferencing 
(CU-Seeme  & Netscape  Cooltalk),  and  enhanced  virtual  systems  (The  Palace  & Active  World).  Students  took  turns  to  moderate 
seminars  in  three-member  small  groups  each  week.  They  followed  the  guidelines  of  Student-Centered  Discussions  (SCD)  (Chou, 
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1999;  Shooop,  1997)  to  participate  in  the  online  seminars.  In  general,  students  participated  each  online  seminar  by  following  the 
SCD  principles  such  as  respecting  each  other,  generating  ideas,  listening  tentatively,  and  referencing  each  other  during 
conversation.  Whereas,  student  moderators  kept  the  discussion  alive  by  observing  rules  such  as  greeting  participants,  devising 
warm-up  activities,  making  an  opening  statement,  using  a iep -by-step  discussion  process,  asking  questions,  scripting  the 
discussion,  and  preparing  concluding  remarks  (Chou,  1999),  Detailed  description  of  instructional  design,  course  syllabus,  and  the 
CMC  systems  employed  are  described  in  the  research  by  Chou  (2001a,  2001b) 

Research  Design  and  Methods 

This  study  examines  interaction  patterns  at  both  interpersonal  and  system  levels  in  a learner-centered  distance  collaborative 
learning  environment.  The  research  focuses  on  factors  that  affect  interaction  from  three  areas:  learning  activities,  technology 
attributes,  and  learner  characteristics.  At  the  system  level,  student  perceptions  of  both  synchronous  and  asynchronous  CMC 
systems  and  the  relationship  with  interaction  are  investigated.  At  the  interpersonal  level,  patterns  of  learner-learner  interaction 
over  both  communication  modes  are  compared  and  contrasted.  Furthermore,  the  overall  effects  of  various  theoretical-based 
instructional  activities  on  learner  interaction  are  also  scrutinized.  The  research  methods  include  content  analysis,  formative  and 
summative  evaluation  of  the  instructional  activities,  and  technologies  employed  in  a distance- learning  course.  The  data  for 
content  analysis  are  based  on  conference  transcripts  from  both  synchronous  and  asynchronous  communication.  Formative  data 
are  based  on  student  reflection  journals,  instructor’s  log,  and  observers’  logs.  Four  observers  were  invited  to  observe  the  class  on 
a weekly  basis.  They  submitted  a weekly  log  to  the  instructor  to  suggest  improvement  on  the  instructional  design  and  activities 
for  this  class.  The  summative  data  are  collected  from  the  following  student  surveys:  student  background,  course  evaluation, 
CMC-skill  assessment,  student  perceptions  of  communication  characteristics  of  technology,  group  cohesiveness  and 
performance.  Detailed  descriptions  of  these  surveys  can  be  found  in  Chou’s  dissertation  work  (2001b). 

Both  synchronous  and  asynchronous  seminars  were  conducted  on  a weekly  basis.  In  the  synchronous  seminar,  students  were 
responsible  to  take  turns  in  moderating  small  group  discussions.  In  the  asynchronous  seminar,  students  collaborated  in  building  a 
knowledge  base  by  sharing  and  exchanging  constructive  views  on  a topic  related  to  CMC  systems.  In  addition  to  the  weekly 
discussion,  the  semester-long  projects  also  required  the  students  to  collaborate  in  small  groups  via  various  CMC  systems. 

Bale’s  (1950)  Interaction  Process  Analysis  (IP A)  was  adapted  as  the  basis  for  content  analysis  to  describe  the  patterns  of 
student  interaction  in  small  groups  via  both  synchronous  and  asynchronous  networks.  IPA  was  originally  developed  to  study 
small  group  interaction  in  two  main  areas:  socioemotional  -oriented  and  task-oriented  interaction. 

Research  questions  are  divided  into  the  following  three  main  categories: 

A.  Technology  Attributes 

QAl : What  are  the  technological  factors  that  affect  student  interaction? 

B.  Learning  Activities 

Synchronous  vs.  Asynchronous  discussions 

QBl:  Is  there  a significant  difference  in  the  social-emotional  contents  and  task-oriented  contents  between  synchronous 

and  asynchronous  communication? 

Conference  Moderation 

QB2:  Moderator  vs.  Participants:  How  can  a conference  moderator  help  to  facilitate  online  discussions? 

Small  Group  Collaboration 

QB3:  What  are  the  student  perceptions  of  small  group  collaborative  activities? 

C.  Learner  Characteristics 

Prior  Computer  experience 

QCl:  Is  learner’s  experience  with  computer  correlated  with  total  number  of  messages  submitted? 

Gender 

QC2:  Is  there  a significant  difference  in  the  SE-oriented  vs.  task-oriented  contents  between  female  and  male  participants? 


Analysis  and  Discussion 

Technology  Attributes 

Students  rated  each  CMC  system  on  the  following  communication  characteristics:  social  presence,  communication 
effectiveness,  and  communication  interface.  Of  all  systems  tested,  WebCT  chat  received  the  highest  rating  and  the  Palace  came 
in  second  place.  This  is  an  indication  that  students  valued  reliable  and  smooth  connection  for  communication.  WebCT  chat 
turned  out  to  be  the  most  reliable  and  straightforward  CMC  systems  used.  In  addition,  the  add-on  affective  components 
(wearable- avatars,  voice-activation,  bubble  messages,  etc.)  in  the  Palace  actually  made  the  conversation  more  realistic.  Students 
demonstrated  enjoyment  in  using  the  avatars  to  express  themselves  during  the  online  conversations. 

Positive  technological  attributes  can  enhance  interaction  and  negative  technological  attributes  can  hinder  interaction.  Table  1 
is  a summary  of  various  technological  factors  that  affect  interaction. 
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Table  1.  Positive  and  Negative  Technological  Attributes  Affecting  the  Degree  of  Interaction 


Positive  Features  I 

Negative  Features 

A.  System  performance 

Fast  loading,  low  bandwidth 

Bandwidth  intensive 

Transcript  recording 

Non-recordable  conversation 

Good  audio/video  quality 

Poor  audio/video  quality 

Cross-platform  compatibility 

Platform-specific 

B.  Interface  design 

User-friendly  navigation  tools 

Nontransparent  or  no  navigation  tools 

Learner  centered  (e.g.,  customizable,  flexible,  and  scalable 

Program  controlled  (e.g.,  fixed  and  un- customizable 

interface) 

interface) 

Wearable  avatars  with  a variety  of  selections 

Fixed-type  avatars  with  stereotypical  selections 

Low  levels  of  distraction  (e.g.,  good  visualization  of  screen 

High-levels  of  distraction  (e.g.,  lack  of  or^nization  of 

icons) 

screen  icons) 

Status  indication  (e.g.,  occupied,  off-line,  online,  etc.) 

Lack  of  status  indication 

Accessible  to  users  with  disabilities  (e.g.,  Bobby -approved, 
text-to-speech  option) 

Not  accessible  to  users  with  disabilities 

C.  Communication  characteristics 

High  degree  of  social  presence 

Low  degree  of  social  presence 

Effective  for  communication  at  interpersonal  level 

Ineffective  for  communication  at  interpersonal  level 

Effective  for  communication  at  system  level  (e.g.,  fast 

Ineffective  for  communication  at  system  level  (e.g.,  delayed 

message  exchan  ges) 

message  exchanges) 

High  degree  of  expressiveness  (e.g.,  mood  indicators) 

Low  degree  of  expressiveness 

Affective  communication  components  (e.g.,  optional 
toolbox  for  emoticons,  props  for  avatars,  etc) 

Impersonal  communication  components  (e.g.,  command- 
line oriented  communication) 

Division  of  public  vs.  private  space  (e.g.,  breakout  sessions 
for  small  groups) 

Lack  of  division  of  meeting  rooms 

Based  on  the  observation  by  the  instructor  and  evaluators,  student  adaptation  to  technology  can  be  summarized  in  four  stages: 

• The  WOW  stage:  At  the  initial  phase,  students  were  fascinated  with  the  potential  of  technology  and  amazed  at  what 
CMC  systems  could  have  accomplished. 

• The  FUN  stage:  At  the  second  phase,  students  actually  used  the  systems  for  simple  tasks  and  derived  a great  deal  of 
pleasure  in  the  hands-on  experience. 

• The  OH-OH  stage:  This  was  the  frustrating  stage  when  more  complex  tasks  were  given  and  their  skills  had  not 
developed  enough  to  handle  these  tasks. 

• The  “Back-to-Normal”  stage:  Students  either  became  more  competent  in  the  use  of  technology  or  became  comfortable 
with  dealing  with  technical  difficulties.  They  internalized  their  anxiety  and  accepted  that  technical  glitches  were 
inevitable  in  the  learning  process. 

Learning  Activities 

a.  Synchronous  vs.  asynchronous  discussion 

The  two  main  categories  in  Bale’s  IPA  are  social- emotional  oriented  interaction  and  task-oriented  interaction.  The  multiple 
regression  analysis  shows  that  both  variables  significantly  predict  the  interaction  patterns  in  both  communication  modes,  F(2, 
1 16)  = 85.7,  p < .0001  (Table  2).  The  mean  sentence  per  person  in  synchronous  mode  is  26.31  sentences  and  51  sentences  in 
asynchronous  mode.  Because  R - J1  and  - 0.6,  60%  of  the  variance  is  accounted  for  by  these  independent  variables.  The 
analysis  shows  that  there  is  a significantly  higher  amount  of  SE-oriented  interaction  in  synchronous  discussions  and  a 
significantly  higher  volume  of  task-oriented  interaction  in  asynchronous  discussions. 

Table  2:  Multiple  Regression  Analysis  Predicting  Interaction  in  Synchronous  Versus  Asynchronous  Discussions 


Syn.  Mean 

Syn.  SD 

Asyn.Mean 

Asyn.  SD 

F 

SE 

8.66 

8.12 

4.26 

4.72 

-7.46 

TASK 

17.65 

15.53 

46.74 

19.11 

12.21 

totals 

26.31 

22.01 

51.00 

20.76 

5.85 

♦ ♦♦/><  0001 


In  the  synchronous  communication  mode,  there  was  more  spontaneous  communication  going  back  and  forth.  The 
communication  processes  between  asking  and  answering  questions  are  more  equally  distributed  in  synchronous  communication, 
whereas  in  asynchronous  communication,  students  tended  to  volunteer  to  give  more  information  than  to  ask  questions. 
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The  synchronous  communication  mode  also  made  it  easier  to  provide  immediate  feedback  to  information  seekers.  Some 
students  were  actively  engaged  in  discussions  while  other  students  waited  until  they  were  asked  to  say  something.  The  researcher 
observed  that  there  was  more  equal  participation  in  the  discussions  in  three-member  small  groups  than  in  large  groups.  In 
addition,  in  synchronous  mode,  participants  asked  more  personal  questions  and  revealed  more  about  their  frustration  or  need  for 
help  with  less  hesitation.  Personal  questions  such  as  one’s  occupation,  schooling  history,  and  background  of  technical  training 
were  included  more  often  in  synchronous  discussions, 
b.  Conference  moderation 

Students  took  turns  moderating  small  group  discussions  in  the  weekly  synchronous  seminars.  Every  group  was  responsible 
for  hosting  one  online  seminar  in  the  semester.  Because  there  were  thee  members  in  each  group,  the  seminar  was  usually  divided 
into  three  small  groups  so  that  each  member  of  the  host  group  could  moderate  one  group  in  the  online  seminar.  The  moderator’s 
action  is  highly  correlated  with  the  performance  of  the  conference  participants.  According  to  Table  3,  when  a moderator  sent  out 
more  task-oriented  content,  the  participants  also  responded  with  more  task-oriented  messages,  F(l,  163)  = 36.58,  p < .0001. 
Likewise,  when  a moderator  sent  out  more  SE-oriented  content,  the  participants  responded  with  messages  of  the  same  nature, 
F(l,  163)  = 1 1.91,  p < .001.  In  addition,  the  total  number  of  messages  sent  by  the  moderators  also  contributed  positively  to  the 
total  number  of  messages  sent  by  the  participants,  fl(l,  163)  = 28.85,  p < .0001.  Overall,  the  moderator’s  functions  are  vital  to  the 
information  exchanges  in  a small  group  discussion.  The  comparison  of  the  mean  sentences  between  moderator  and  participant 
indicates  that  in  order  to  encourage  active  discussion,  the  moderator  usually  sent  out  two  or  three  times  more  sentences  than  the 
participant. 

Table  3:  One-Way  AN  OVA  Between  the  Mean  Sentences  Sent  by  Moderators  and  Participants 


Moderators 

Participants 

F 

SE  Mean 

9.05 

3.5 

11.91 

mm 

Task  Mean 

24.15 

6.29 

36.58 

mmm 

Total  Mean 

33.19 

9.78 

28.85 

mmm 

***p<  . 0001,  **p<  001 


c.  Small  group  collaboration 

Two  forms  of  collaboration  took  place  in  small  groups:  synchronous  seminar  and  project  preparations.  In  addition  to  working 
together  to  host  a successful  synchronous  seminar,  members  of  a small  group  also  met  several  times  in  private  throughout  the 
semester  to  prepare  for  seminar  moderation  and  case  study.  Students  were  asked  to  complete  the  questionnaires  on  group 
cohesiveness,  individual  commitment,  and  individual  performance  at  the  end  of  the  term.  The  highest  score  one  member  of  a 
group  could  get  was  40  points.  In  Table  4,  the  mean  score  of  each  group  is  listed. 

Table  4 Group  Performance  Evaluation  and  Group  Cohesiveness  Score 


Groups  (n  = 3) 

Cohesiveness 

Perceived  quality 

of  group  performance 

Mean 

SD 

Mean 

SD 

GROUP  1 

39.67 

0.82 

5.98 

0.86 

GROUP  2 

37.5 

3.89 

6.02 

1.16 

GROUP  3 

38.5 

2.07 

5.94 

0.97 

GROUP  4 

32.5 

10.61 

3.76 

1.63 

GROUP  5 

37.0 

4.08 

5.53 

1.18 

The  correlation  between  the  perceived  quality  of  group  performance  and  group  cohesiveness  is  significantly  high,  r = .95,  p = 
.01.  Group  members  who  rated  their  actions  highly  cohesive  also  deemed  their  performance  high.  However,  the  correlation 
between  perceived  quality  of  group  performance  and  individual  commitment  is  low,  r=  -.12,  /?  = .29.  Individuals  who  were 
committed  to  their  work  did  not  necessarily  consider  group  performance  quality  high  (see  Table  5).  In  some  instances,  members 
of  a group  might  work  harder  when  they  foresaw  that  the  quality  of  group  performance  would  not  be  up  to  standards.  Putting 
students  in  small  groups  and  assigning  collaborative  tasks  to  each  group  does  not  always  guarantee  a successful  learning 
experience.  The  summary  section  concludes  a number  of  factors  affecting  online  interaction.. 

Table  5 Mean  Score  of  Individual  Commitment  and  Perceived  Quality  of  Group  Performance 


Groups  (n  = 3) 

Perceived  quality  of  group 
performance 

Individual  Commitment 

GROUP  1 

5.98 

5.83 

GROUP  2 

6.02 

6.64 

GROUP  3 

5.94 

5.50 

GROUP  4 

3.76 

6.13 

GROUP  5 

5.53 

6.00 
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Learner  Characteristics 

Due  to  insufficient  data  and  small  sample  selection,  no  significant  correlation  was  found  between  total  messages  sent  by  each 
participant  and  their  previous  computer  experience.  In  terms  of  gender  differences,  significant  differences  were  found  in 
synchronous  mode  in  both  SE-oriented  and  task-oriented  interaction.  In  general,  female  participants  sent  out  more  sentences  than 
the  male  participants  in  both  synchronous  and  asynchronous  communication  modes;  the  female  mean  sentences  are  hi^er. 
Nevertheless,  female  participants  sent  out  significantly  higher  number  of  messages  in  both  SE-oriented  and  task-oriented  areas. 
Overall,  female  participants  consistently  sent  more  SE-oriented  messages  in  both  communication  modes. 

Summary 

Interaction  Factors 

The  main  conclusion  drawn  from  this  study  is  that  the  design  of  learner- centered  online  activities  and  the  selections  of 
appropriate  technologies  do  contribute  to  different  patterns  of  interaction.  The  research  findings  are  summarized  as  follows: 

A.  Learning  activities:  Constructivist-based  instructional  activities  such  as  student -moderated  discussion  and  small  group 
cooperative  learning  are  conducive  to  interaction  and  learning.  Specific  findings  are  listed  as  below: 

a.  The  appropriate  use  of  synchronous  online  seminar  can  enhance  interpersonal  relationship.  In  general,  students  submit  a 
higher  percentage  of  task-oriented  messages  than  social-emotional  oriented  messages  in  both  asynchronous  and  synchronous 
communication  modes.  Nevertheless,  there  is  a higher  percentage  of  social-emotional  interactions  in  synchronous  mode  than  in 
asynchronous  mode. 

b.  Asynchronous  peer  review  provides  the  opportunity  for  collaboration  on  building  knowledge  bases  and  information 
sharing. 

c.  Interestingly,  there  was  more  one-way  communication  in  asynchronous  mode.  In  asynchronous  mode,  students  seemed  to 
be  more  interested  in  expressing  opinions  than  challenging  each  others’  views;  whereas  in  synchronous  mode,  there  were  more 
questions  and  answers.  Students  were  more  engaging  in  the  synchronous  discussions.  There  was  a stronger  sense  of  immediacy  to 
respond  to  peers’  questions  in  synchronous  mode  than  in  asynchronous  mode. 

d.  Student-moderated  conference  based  on  the  SCD  Model  allows  learners  to  take  initiatives  in  their  learning  and  be  efficient 
in  communication  via  various  CMC  systems. 

e.  Forming  small  groups  for  online  seminars  or  group  projects  helped  to  reduce  the  initial  disorientation  and  confusions  of 
online  learners. 

B.  Technology  attributes:  Discussion  on  technology  attributes  focuses  on  the  mode  of  communication  systems  and  the 
communication  characteristics  such  as  social  presence,  communication  effectiveness,  and  communication  effectiveness. 

a.  Communication  systems:  the  selections  of  synchronous  or  asynchronous  technologies  contributed  to  the  different 
interaction  patterns.  Students  tended  to  spend  much  more  time  in  task-oriented  discussions  in  asynchronous  mode.  When  online 
tasks  were  clearly  defined  and  students  passed  the  initial  “get-to-know-each-other”  stage,  students  were  inclined  to  spend  less 
time  in  SE-oriented  interactions  in  both  communication  modes.  Nevertheless,  learners  consistently  spent  more  time  in  SE- 
oriented  interaction  in  synchronous  mode  than  in  asynchronous  mode. 

b.  Communication  characteristics:  Student  ratings  of  a CMC  system  increased  as  the  frequency  of  uses  increased.  Student 
perceptions  of  the  communication  characteristics  of  technologies  might  affect  their  initial  interaction  online.  Time  played  an 
important  role  in  student  adoption  of  new  technology.  Usually  after  the  first  two  or  three  weeks,  students  were  able  to  ignore 
some  of  the  “obstacles”  of  a system  and  concentrated  on  the  task  at  hand. 

c.  Learner  characteristics:  Gender  difference  affects  how  students  interact  online.  Female  students  contributed  more  to  SE- 
oriented  interaction  than  the  male  students  in  both  communication  modes.  In  addition,  prior  knowledge  in  a subject  matter  and 
computer  experience  contributed  to  the  different  interaction  patterns  in  the  individuals  at  the  beginning  stage.  After  the  first  two 
weeks,  the  difference  was  hardly  noticeable. 

As  the  result  of  the  research,  a model  of  Learner-Centered  Computer-Mediated  Interaction  for  Collaborative  Distance 
Learning  is  proposed  to  explain  factors  that  could  affect  interaction  as  shown  in  Figure  1. 

Conclusions 

Research  in  distance  education  covers  a wide  spectrum  of  issues.  Although  interaction  is  not  the  only  key  to  successful 
distance  education,  this  factor  is  vital  to  the  progress  of  learners,  teachers,  and  the  school  as  a whole.  As  Gunawardena  et  al. 
(1997)  has  boldly  put  it:  “No  interaction,  no  education.”  This  research  emphasizes  the  importance  of  interaction  research  by 
providing  supporting  evidence  in  activity  design,  technology  employed,  and  learner  differaices.  This  study  advocates  the 
integration  of  learner-centered  instructional  design  and  constructivism  into  the  curriculum.  The  researcher  hopes  to  break  the 
myth  that  synchronous  communication  is  impossible  to  manage.  On  the  contrary,  as  shown  in  this  study,  the  appropriate 
incorporation  of  synchronous  activities  can  enhance  learning  interests  and  interpersonal  relationship.  Although  there  is  no  lack  of 
research  in  distance  education  since  the  1980s,  there  is  a need  for  more  research  on  emerging  technology  employed  in  distance 
education  because  the  implications  and  applications  also  affect  educational  policy  and  management.  This  study  is  a small 
contribution  to  the  understanding  of  the  ever-changing  technological  ecology  of  distance  education. 
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Figure  1:  Model  of  Learner -Centered  Computer -Mediated  Interaction  for  Collaborative  Learning 
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Web  Enhanced  Learning  and  Student  Awareness  of  Strategy  Use 


Jane  Crozier 


Abstract 

This  qualitative  study  examined  the  awareness  of  strategy  use  and  justification  of  that  use  of  gifted  undergraduate  students  as 
they  researched,  utilized  resources,  and  evaluated  finalist  for  a global  understanding  and  peace  award.  The  participants  in  the 
informal  learning  environment  utilized  a web-based  learning  environment.  Their  homepage  was  developed  in  WebCT  and 
allowed  them  to  access  to  informational  resources,  conduct  online  chats,  and  write  reflective  journals.  The  findings  focused  on 
the  student  awareness  of  their  strategy  use  and  their  online  learning  community  experience. 

Introduction 

An  increasing  number  of  universities  and  K-12  school  systems  are  using  some  type  of  web-based  support  for  learning 
(Mandiach  & Cline,  2000).  Mandiach  and  Cline  (2000)  argue  that  despite  the  enthusiasm  of  some  educational  institutions, 
practical  and  curricular  problems  related  to  the  integration  of  web-based  learning  aivironments  persist.  These  environments 
range  from  low-level  integration  where  a homepage  contains  links  to  related  information;  to  mid-level  integration,  where  a 
homepage  contains  informational  links,  asynchronous  and/or  synchronous  communication  connections  in  the  form  of  chat 
sessions  or  a bulletin  board,  and  course  handouts;  to  a high-level  integration,  where  the  entire  course  is  delivered  via  the 
homepage  including  lectures,  homework,  all  communications,  testing,  and  student  grades  (Miller  & Miller.  2000). 

Researchers  have  just  begun  to  scratch  the  surface  of  the  usefulness  of  the  range  of  different  levels  of  web-based  learning 
environments  (Gunawardena  & Zittle,  1997;  Miller  & Miller,  2000;  Palloff  & Pratt,  1999).  The  research  highlighted  the  online 
learning  communities  involved  in  various  levels  of  web-based  environments  and  learner  characteristics  that  enable  students  to 
successfully  interact  and  learn  using  these  environments.  These  elements  were  identified  through  exploration  of  either  reflection 
or  collaboration,  and  from  the  perspective  of  the  teacher  and/or  student. 

Purpose 

This  study  continues  the  research  into  web-based  learning  by  examining  a mid-level  web -based  learning  environment  as  a 
support  for  an  informal  learning  experience.  The  informal  learning  situation  was  a group  of  undergraduate  students,  that  were 
Fellows  in  the  Honors  program,  who  served  as  the  selection  committee  for  the  finalists  of  a global  awareness  and  peace  award. 
The  committee  members  needed  to  determine  the  award  criteria,  research  and  learn  about  the  backgrounds  and  attributes  of  each 
nominee,  and  evaluate  each  nominee  based  on  the  criteria  for  the  award.  There  were  a variety  of  strategies  the  students  would 
utilize  in  order  to  develop  criteria,  research  nominees,  and  evaluate  nominee  attributes.  The  learning  aspect  explored  was  the  role 
the  web-based  environment  had  in  supporting  student  awareness  of  strategy  use. 

Online  reflection  and  collaboration  activities  to  identify  strategy  were  used  and  the  reasoning  behind  that  use,  from  the 
student  perspective.  In  addition,  perceptions  of  the  student’s  online  experience  were  sought.  The  original  research  question 
addressed  in  this  study  was,  “How  does  a web  enhanced  learning  environment  effect  the  nomination  process?” 


Research  Literature 

Awareness  of  strategy  use 

Researchers  in  the  field  of  psychology  consider  “awareness  of  strategy  use”  as  part  of  an  individual’s  metacognitive  abilities. 
BJorklund  (1995)  and  Sternberg  (1990)  identify  these  abilities  as  an  individual’s  knowledge  and  regulation  of  cognitive 
processes.  The  knowledge  of  an  individual’s  cognitive  processes  or  strategies  involves  knowing  the  proper  time  to  use  strategies 
and  the  reasoning  behind  their  use.  The  regulation  of  cognitive  processes  or  strategy  use  involves  planning  strategy  use  and 
evaluation  of  the  result  of  the  strategy  usage,  although  mentioned  here  as  a definition,  regulation  is  not  a focus  in  this  study. 

The  research  on  strategy  use  and  the  reasoning  behind  that  usage,  the  focus  of  exploration  for  this  paper;  involves  strategy 
use  in  problem  solving  (Royer,  et  al,  1993)  and  among  gifted  students  (Carr,  et  al,  1995).  Strategy  usage  has  been  easy  for 
students  to  identify,  however  the  reasoning  behind  strategy  use  is  more  difficult  for  individuals  to  identify.  The  findings  dealing 
with  knowing  when  to  use  a strategy  have  indicated  that  it  is  fairly  easy  for  adults  to  identify  strategies  they  have  used;  however 
younger  learners  cannot  always  identify  a strategy  by  name  (1995).  In  both  cases,  discussing  their  reasons  for  strategy  use  was 
much  more  difficult,  if  not  impossible. 

Online  student  reflection 

The  research  conducted  involves  student  reflection  activities,  and  usually  revolves  around  student  impressions  of  learning 
and  working  in  an  online  learning  community  (Sherry,  et  al,  1998;  Gunawardena  & Zittle,  1997).  However,  one  study  of 
particular  interest  was  that  of  Guzdial  and  Turns  (2000);  where  they  examined  student  reflective  practice  in  an  online  community 
as  the  students  collaborated  on  projects  in  an  online  engineering  course  at  the  University  of  Michigan.  According  to  the  findings, 
in  order  for  the  students  to  successfully  contribute  to  their  class,  they  had  to  reflect  on  the  bulletin  board  discussion  threads  and 
determine  whether  a new  or  alternative  idea  should  be  contributed  to  the  concept  thread  and  how  their  response  would  be 
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perceived.  These  practices  assisted  in  student  contribution  to  course  discussion  threads  and,  the  researchers  noted,  in  turn, 
enhanced  student  learning. 

Online  student  collaboration 

The  research  involving  web-based  learning  support  of  student  collaboration  focused  on  providing  students  with  immediate 
access  to  their  peers  and  to  informational  resources  on  the  Internet  (Palloff  & Pratt,  1999;  Sherry,  et  al,  1998).  Through 
collaboration,  many  students  form  online  learning  communities  within  the  bounds  of  the  course  or  project.  Palloff  and  Pratt 
(1999)  emphasize  that  these  communities  have  a great  influence  on  the  success  of  the  class;  in  fact,  these  researchers  suggest  that 
if  the  community  is  unsuccessful  or  does  not  exist  then  no  learning  occurs. 

The  strength  of  the  collaboration  is  key  to  the  success  of  the  community  and  is  dependent  upon  the  student’s  comfort  level  as 
a member  of  the  community.  Sherry,  et  al  (1998)  discussed  that  participants  in  their  study  had  stronger  communities  when  the 
members  had  a high  comfort  level  while  working  in  the  online  environment.  An  element  that  enhances  a member’s  feeling  of 
comfort  is  what  Gunawardena  and  Zittle  (1997)  called  a “sense  of  presence.”  According  to  these  researchers,  this  sense  of 
presence  indicates  that  the  member  feels  a strong  connection  to  the  online  community.  In  addition,  it  has  been  identified  as  a key 
factor  in  successful  online  learning  experiences. 

Method 

Participants 

The  participants  involved  in  this  study  were  nine  Honors  Fellowship  undergraduate  students  at  a public  university.  Students 
who  received  a fellowship  are  viewed  as  gifted.  These  Honors  Fellows  served  as  members  of  a selection  committee  responsible 
for  selecting  the  finalists  for  an  annual  global  understanding  and  peace  award.  One  third  of  the  committee  had  participated  the 
previous  year.  The  names  used  in  the  study  were  pseudonyms  selected  by  the  participants.  There  were  two  teams.  The  advisor 
to  the  this  student  committee  was  also  the  researcher. 

Study  Design 

This  study  took  place  in  the  informal  educational  setting  of  the  award  selection  committee  for  approximately  two  and  a half 
months.  The  participants  worked  individually  to  research  their  nominee  (included  individuals  and  nonprofit  organizations 
involved  with  issues  related  to  global  understanding  and  peace  initiatives)  and  in  teams  to  support  each  other’s  efforts.  The  first 
two  committee  meetings  were  design  ed  to  orient  the  students  to  research  resources,  establish  evaluation  criteria  for  the  nominees, 
and  allow  the  students  to  select  their  nominee. 

In  the  following  month,  the  teams  met  weekly  to  discuss  member  research  progress  and  evaluation  issues.  These  weekly 
meetings  were  held  in  the  team  chat  rooms.  The  final  team  meeting  was  held  face-to-face.  Each  team  member  introduced  his/her 
nominee  and  answered  questions  posed  by  other  team  members.  At  the  end  of  the  meeting,  the  team  members  selected  one  or 
two  nominees  to  submit  to  the  full  committee  as  finalists.  The  team  members  who  nominated  the  team  finalists  incorporated  the 
feedback  from  other  team  members  into  a one  page  composite  listing  of  the  nominee’s  background  and  attributes  for  submission 
to  the  full  committee.  The  full  committee  reconvened  after  the  final  team  meetings  and  nominated  the  finalists. 

During  the  final  two  meetings,  in  the  first  full  committee  meeting,  each  team  introduced  their  nominee  to  the  group,  in  a 
similar  manner  as  they  did  during  the  final  weekly  team  meeting.  The  full  committee  then  voted  on  the  nominees  from  the  teams 
and  selected  all  the  nominees  submitted,  six  in  all.  The  finalist  composites  submitted  to  the  full  committee  were  expanded  into 
nomination  documents.  These  documents  detailed  the  nominee’s  background,  affiliated  organization,  and  criteria  based 
attributes;  and  were  then  sent  to  the  governing  board. 

Homepage  Design 

The  committee  web  page  was  developed  in  WebCT  (an  Internet  based  course  support  system  which  supports  asynchronous 
and  synchronous  communication  such  as  Chats  and  bulletin  boards,  as  well  as  student  records,  faculty  lectures  and  notes  via 
audio  and/or  video).  The  homepage  provided  the  group  with  secure  access  to:  1)  Weekly  meeting  schedules  and  updates,  2)  team 
chat  rooms,  3)  reflective  journals  via  bulletin  board,  4)  forms  to  support  research,  and  5)  links  to  research  databases.  These 
resources  provided  a convenient  and  effective  means  for  feedback  among  the  team  members  and  the  advisor.  After  the  weekly 
chat  meetings,  students  were  asked  to  reflect  on  the  process  they  used  and  their  online  experience,  in  their  individual  online 
journals  (shared  only  with  the  advisor,  not  the  other  students).  The  students  used  forms  on  the  Homepage  that  served  as 
guidelines  for  recording  sources.  The  students  provided  research  database  links  that  grew  in  number  as  they  located  more 
sources  of  information.  The  links  were  made  available  to  all  committee  members  in  order  to  help  those  having  problems  locating 
resources. 

Data  Collection 

The  study’s  data  collection  resources  were  triangulated  to  trace  student  strategy  use,  depict  student  awareness  of  strategy  use 
and  its  justification,  as  well  as  provide  confirmation  of  findings.  This  triangulation  of  data  resources  was  utilized  to  provide 
greater  validation  to  the  study  (Patton,  1990). 
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> Weekly  Team  Chat  Session  Discourse:  The  chat  session  discourse  was  recorded  and  reviewed  in  search  of  evidence  of 
criteria  and  evaluation  strategy  discussions.  The  reflective  journals  were  included  as  a main  source  of  data  related  to 
student  awareness  of  evaluative  strategy  use. 

> Student  Reflective  Journals:  Each  week,  the  students  were  asked  to  respond  to  a few  brief  questions  and  incorporate 
their  own  thoughts  and  ideas,  about  the  process,  and  their  research,  into  their  own  reflective  journal. 

> Semi-formal  Interviews:  The  interviews  of  team  leaders/facilitators  were  conducted  half  way  through  the  project. 
These  individuals  were  selected  because  they  facilitated  the  chat  discussions  and  their  role  focused  on  review  of  team 
progress.  They  presented  opportunities  for  obtaining  data  related  to  student  awareness  (Patton,  1990).  Most  of  the 
student  questions  focused  on  the  criteria  selection  process,  research,  and  the  online  experience. 

> Participant  Observer  Fieldnotes:  Since  the  researcher  (myself)  was  also  the  advisor  to  the  group,  participant  observer 
fieldnotes  were  used.  These  notes  were  summaries  of  events  during  the  face-to-face  meetings. 

> Student  Artifacts:  The  student  artifacts  or  final  nominee  documents  were  used  to  review  the  synthesis  of  the  team’s 
work  and  the  progression  of  evaluative  strategy  use. 

Data  Analysis 

The  qualitative  case  study  method  was  used  (Glaser,  1995;  Merriam,  1997)  to  analyze  the  data  collected  in  this  study.  The 
case  study  method  incorporates  constant  comparison  for  the  analysis.  During  the  analysis,  the  researcher  reviewed  the  data, 
identified  patterns,  determined  categories,  and  identified  overriding  themes.  The  student  reflections,  chat  discourse,  and  final 
documents  were  compared  to  identify  repetitive  themes  related  to  awareness  of  strategy  use  and  online  learning.  The  field  notes 
and  artifacts  were  used  to  verify  findings.  The  credibility  of  the  study  analysis  was  enhanced  through  an  exhaustive  search  for 
negative  cases  found  in  the  themes.  According  to  Merriam  (1998),  a negative  case  is  one  that  is  the  extreme  opposite  of  the 
patterns  or  trends  discovered  during  analysis.  These  cases  were  sought  to  extend  the  definition  of  the  rule  of  the  pattern,  as  in 
participant  awareness  of  strategy  use  or  participant  experience  as  an  online  learner  (Patton,  1990;  Merriam,  1998). 

Key  Findings 

The  students  began  their  discussion  to  establish  the  criteria  used  in  evaluating  nominee  goals  and  accomplishments;  the  group 
discussion  required  an  additional  group  meeting  in  order  to  determine  the  final  criteria.  After  this  first  meeting,  the  participants 
selected  their  nominee  (there  were  fifteen  nominees  and  nine  students,  so  six  students  selected  two)  and  began  their  research.  At 
the  end  of  the  second  meeting,  the  criteria  was  established.  The  students  developed  a list  of  five  criteria:  Awareness, 

Benevolence,  Commitment,  Diplomacy,  and  Influence.  The  presence  of  the  criteria  served  as  a foundation  for  the  students  and 
was  mentioned  periodically  during  team  meetings  as  reference  points  from  which  to  gauge  the  information  they  were  reviewing. 
The  criteria  guided  their  research,  discussions,  and  decision-making.  The  responses  to  the  interview  questions,  guided  reflection, 
and  chat  discussions  provided  a comprehensive  picture  of  the  student  strategy  use  and  their  online  experience.  Two  key 
questions  asked  were,  “What  steps  did  you  take  to  determine  the  criteria?”  and  “Why  did  you  make  your  selection?” 

Awareness  of  Strategy  Use 

The  students  were  able  to  identify  the  strategies  they  used  to  establish  the  criteria,  such  as  brainstorming,  adaptation,  compare 
and  contrast,  and  simplification.  However,  most  did  not  adequately  discuss  their  reasons  justifying  their  choices.  One  student, 
Madi,  from  Team  A,  demonstrated  this  when  she  discussed  her  strategy  selection  during  her  participation  in  the  first  round  of 
criteria  development: 

I just  basically  brainstormed  ...  And  then  in  my  group,  Don  and  Jan,  they  had  really  specific  things,  like  “have  an 
extended  period  of  service”  ...  - mine  are  a little  vague  so  I adapted  mine  to  the  more  specific  things  that  they  had. 

Madi’s  awareness  of  her  use  of  the  brainstorming  and  adaptation  strategies  were  present,  however  her  reasoning  for 
discarding  her  original  criteria  list  and  adopting  the  list  of  her  teammates  seemed  weak.  Madi  adopted  the  new  list,  citing  that  it 
was  more  specific  than  her  list,  but  did  not  expand  on  why  she  found  this  attribute  preferable.  This  awareness  of  strategy  use  and 
lack  of  reasoning  was  apparent  in  the  reflections  of  most  of  the  participants.  The  ability  has  been  noted  in  gifted  students  by 
Carr,  et  al  (1995);  their  findings  indicated  that  gifted  students  more  so  than  average  students  possess  specific  strategy  knowledge 
or  the  metacognitive  knowledge  about  when  and  where  to  use  a strategy  with  fewer  of  the  students  knowing  why  they  used  a 
strategy. 

The  students  in  this  study,  indicated  that  they  knew  when  and  where  to  use  a strategy,  however,  only  one  expressed  exactly 
why  she  used  a specific  strategy.  Another  student,  able  to  identify  the  strategies  she  used,  was  also  able  to  discuss  her  reason  for 
using  the  strategy.  This  unusual  illumination  occurred  with  one  participant,  a case  which  further  defines  the  strategy  usage  theme 
to  allow  for  some  rare  occasions  when  gifted  students  are  able  to  justify  their  use  of  a strategy.  Radison,  from  Team  B, 
commented  on  her  own  strategy  use  and  reasoning,  when  she  discussed  the  hour  of  debate  that  ensued  prior  to  the  acceptance  of 
the  final  criteria  list.  This  debate  was  a war  of  words  rather  than  meaning;  the  paragraph  length  descriptions  of  each  attribute  was 
eventually  reduced  to  the  five  general  terms  representing  global  understanding  and  peace  at  Radison ’s  Team’s  prompting  (The 
five  criteria  for  global  understanding  and  peace  were:  Awareness,  Benevolence,  Commitment,  Diplomacy,  and  Influence). 
Radison  wrote  in  her  journal: 


We  came  up  with  five  words  that  seemed  to  cover  the  overall  theme  we  had  been  trying  to  get  at  in  all  the  previous  work. 
I think  simplifying  is  often  the  most  overlooked  step  in  processes  such  as  these.  Sometimes  it’s  hard  to  take  ideas  off  the 
list  because  it  feels  like  we’re  moving  backwards. 

Radison  recognized  that  the  strategy  the  group  was  using  to  finalize  the  criteria  (refining  the  phrases,  word  by  word) 
was  not  working  and  suggested  another  way  of  accomplishing  the  task.  This  way  was  accepted  almost  immediately. 

Online  Learners 

Online  community: 

The  chat  sessions,  participant  observer  field  notes  of  face-to-face  meetings,  and  interview  responses  from  the  two  team 
leaders  relayed  information  regarding  student  roles  and  impressions  of  the  online  chat  meetings.  The  patterns  that  emerged 
depicted  themes  of  the  nature  of  online  collaboration  that  focused  on  member  support  and  interaction  in  the  online  community. 
Each  member  had  a role,  i.e.  facilitator,  support  member,  or  advisor;  most  of  these  roles  were  identified  in  Palloffs  and  Pratt’s 
work  (1999).  In  a supportive  function,  team  members  voluntarily  offered  information  resources,  research  tactics,  and  emotional 
support  for  those  frustrated  with  either  an  over  abundance  or  lack  of  informational  resources.  These  aforementioned  practices 
demonstrate  a camaraderie  among  team  members  that  appears  to  have  strengthened  throughout  the  committee’s  tenure.  The 
teammates  joked  with  and  teased  each  other. 

However,  the  establishment  of  the  online  community  could  not  be  solely  attributed  to  the  participants’  involvement  in  the 
chat  sessions.  Since  many  of  the  students  knew  each  other  prior  to  their  involvement  in  the  committee,  due  to  their  honors 
fellowship  affiliation;  the  camaraderie  among  Team  B community  members,  while  strengthened  by  the  online  interaction,  was 
also  a product  of  the  prior  affiliation  and  the  face-to-face  meetings  (Palloff  & Pratt.  1999:  Gunawardena  & Zittle,  1997). 

The  students  preferred  interaction  among  their  peers,  indicating  a greater  satisfaction  and  comfort  level,  than  from  their 
online  interactions.  While  the  facilitated  discussions  and  online  chats  provided  team  members  with  a convenient  opportunity  to 
give  and  obtain  the  additional  skills  they  needed  to  access  additional  information  about  their  nominees  and  support  better 
decision  making;  the  chat  environment  didn’t  offer  the  personal  interaction  the  team  members  were  accustomed  to  and  preferred. 
The  students  stated  that  they  felt  the  online  meetings  were  convenient  and  productive;  however,  since  they  were  on  the  same 
campus,  their  preference  was  for  face-to-face  meetings  (Sherry,  Fulford,  and  Zhang  (1998).  The  team  mentioned  that  they  would 
work  in  the  online  environment  again,  but  only  out  of  convenience  (1998).  Overall,  the  team  members  impressions  of  their 
online  experiences  were  productive  and  facilitated  their  research,  but  less  satisfying  than  their  face-to-face  interactions. 

Lack  of  social  presence: 

The  students  indicated  that  the  online  experience  Just  wasn’t  the  same  as  meeting  in  person.  They  exhibited  some  resistance 
to  the  use  of  online  means  of  communication  by  requesting  that  they  have  another  face-to-face  meeting  instead  of  a final  chat 
session.  This  resistance  could  have  hindered  their  online  experience,  but  was  not  the  only  deterrent  to  their  online  experience. 
The  students  mentioned  that  technical  problems,  such  as  delays  in  response  time  and  disconnections,  made  for  a less  than  optimal 
experience;  and  due  to  typing  delays,  the  chats  were  sometimes  difficult  to  contribute  to  and  follow. 

Online  behavior : 

As  mentioned  in  the  Online  Community  section,  much  of  the  online  behavior  observed  supported  past  research  that  identified 
members  of  the  community  assuming  and  maintaining  their  roles  in  the  community  (Palloff  & Pratt,  1999).  One  role,  of 
particular  interest  during  the  team  chat  sessions,  was  the  that  of  the  chat  facilitator.  The  facilitator  was  voted  in  by  the  team  to  be 
the  team  leader  and  facilitate  the  chat  meetings.  The  unexpected  online  strategies  used  by  the  chat  facilitator  were  monitoring 
and  troubleshooting  the  chat  session.  These  were  used  to  maintain  continuity  during  the  discussion  and  ease  any  discomfort 
among  chat  participants,.  This  practice  moved  team  members  through  awkward  moments  and  is  illustrated  by  TLeader,  from 
Team  B,  after  a few  minutes  of  silence  and  an  interruption  in  the  chat: 

TLeader:  this  may  be  an  awkward  silence,  so  i'll  move  on. 

Nimbus,  how's  your  research  going? 

Nimbus:  Do  they  focus  on  promoting  peace  between 

certain  religions  or  do  they  have  broader  g)als  of 
general  tolerance/acceptance? 

During  later  reflection  she  indicated  that  it  was  difficult  to  determine  why  chat  participants  stopped  interacting  (typing). 

Somebody  could  have  been  typing,  somebody  could  have  been  thinking,  somebody  could  have  been  preparing  a 
response,  but  nobody  really  knows  so  you  are  Just  sort  of  sitting  there  wondering  well  should  I break  the  awkward 
silence. 

TLeader  monitored  and  effectively  smoothed  over  the  rough  spots  during  the  online  chat.  When  she  encountered  a difficulty 
or  interruption  in  the  communication  line,  she  mentioned  it  and  moved  on.  This  practice  seemed  to  assuage  some  team  members 
discomfort  in  the  online  environment.  Madi,  chat  facilitator  for  Team  A,  did  not  specifically  address  the  silences;  she  simply 
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moved  on  quickly  through  the  session.  This  practice  could  have  affected  her  team  interaction;  they  did  not  have  a great  deal  of 
camaraderie. 

Recommendations  for  Practice 

> Online  monitoring  and  troubleshooting  strategies  of  chat  facilitators  can  improve  comfort  level  and  promote  community 
during  chat  sessions.  Acknowledging  awkward  moments,  such  as  silences,  inactivity  or  technical  difficulties  and 
quickly  move  forward  may  improve  comfort  level  and  promote  community  among  group  members. 

> In  the  online  learning  community,  to  maintain  student  interest  and  increase  student  comfort,  create  a sense  of  presence 
for  the  students,  by  the  use  of  humorous  emoticons  ©,  self-selected  avatars,  or  other  methods  to  improve  a student’s 
online  experience. 

> Guided  reflection  may  encourage  student  awareness  of  strategy  use  for  evaluation  purposes  and  online  community 
facilitation. 

Conclusion 

This  Study  has  laid  the  groundwork  for  future  research  in  online  learning  communities  through  the  examination  of  online 
strategy  use  of  chat  facilitators.  The  findings  here  encourage  further  analysis  of  successful  facilitative  strategies  for  chat  sessions 
in  order  to  provide  continuity  in  communication,  enhance  member  comfort,  ensure  the  productivity  of  the  group,  and  facilitate 
the  learning  process. 

In  addition,  the  findings  of  this  study  support  those  of  other  researchers  in  the  area  of  metacognition  or  awareness  of  strategy 
use  among  gifted  students.  Specifically,  the  participants  in  the  study  were  able  to  identify  their  strategy  use  during  criteria 
development  and  all  but  one  experienced  greater  difficulty  identifying  their  reasons  for  using  a particular  strategy. 

Furthermore,  the  findings  support  previous  research  on  successful  online  learning  communities,  especially  in  the  importance 
placed  on  student  comfort  level  and  having  a sense  of  presence  in  the  online  environment. 

Finally,  this  study  offers  additional  insights  on  web-based  learning,  such  as  facilitation  of  online  communities  and  reflective 
practices,  leaving  the  essence  of  successful  web-based  learning  less  of  a mystery. 
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Abstract 

Authoring  tools  have  evolved  over  the  last  decade  based  on  technological  and  pedagogical  innovations,  from  authoring 
bounded,  program-controlled  learning  systems  such  as  Computer-Based  Instruction  (CBI)  to  authoring  unbounded,  learner - 
centered  environments  such  as  Web-Based  Instruction  (WBI),  This  paper  discusses  the  current  and  future  state  of  authoring  tools 
and  their  pedagogical  ejfect  on  the  development  of  learning  systems.  It  provides  a taxonomy  of  authoring  tools  and  their 
underlying  paradigms;  a detailed  table  that  compares  and  contrasts  pedagogical  attributes  of  CBI  and  WBI;  and  it  discusses  two 
innovative  approaches  on  how  future  authoring  tools  can  preserve  the  level  of  usability  and  the  instructional  methods  that 
instructional  designers  have  become  familiar  with  while  allowing  more  powerful  and  flexible  learning  systems  to  be  built. 

What  are  authoring  tools? 

Authoring  tools  are  software  tools  that  enable  instructional  designers,  educators,  teachers  and  learners  to  design  interactive 
multimedia  and  hypermedia  learning  environments  without  the  knowledge  of  programming  languages.  “The  premise  behind 
authoring  tools  is  the  absence  of  a programmer  or  the  ability  of  designers  with  little  or  no  programming  experience  to  develop 
and  design  instructional  applications”  (Hedberg  & Harper,  1998).  For  example,  multimedia  authoring  tools  facilitate  the 
development  of  Computer-Based  Instruction  (CBI)  by  masking  the  programming  layer,  and  Web-based  authoring  tools  facilitate 
the  development  of  Web-Based  Instruction  (WBI)  by  masking  the  HTML  scripting  layer  (Craney,  1996).  In  essence  authoring 
tools  are  an  accelerated  application  or  simplified  form  of  programming  by  virtue  of  their  inclusion  of  pre-programmed  elements 
for  the  development  of  interactive  multimedia  and  the  deployment  of  a point  and  click  user  interface  to  activate  these  elements. 
Authoring  tools  however  accomplish  their  tasks  using  a certain  methodology  or  paradigm  that  requires  a type  of  heuristic  or 
algorithmic  thinking  similar  to  that  of  programming  languages  (Siglar,  1999).  Some  of  these  paradigms  include  the  scripting 
metaphor,  the  card-scripting  metaphor,  iconic/flow  control,  hypermedia  linkage,  the  frame  metaphor,  the  cast-score  metaphor,  the 
hierarchical  object  metaphor,  and  tagging  (Kozel,  1997;  Siglar,  1999).  Table  1 provides  a brief  explanation  of  these  heuristic 
paradigms  and  examples  of  authoring  tools  that  utilize  these  paradigms. 

Table  I — Authoring  Paradigms 


Authoring  Paradigm 

Explanation  of  paradigm 

Examples  of  Authoring  Tools 

Scripting  metaphor 

Resembles  a programming  language 
in  that  it  involves  specifying  all 
media  elements  by  filename  and 
interactions  by  coding 

TenCORE  Language  Authoring 
System 

Card/scripting  metaphor 

Uses  an  index-card  structure  or  a 
book  metaphor  to  link  elements 

HyperCard,  Supercard,  HyperStudio, 
TenCore,  Toolbook  II 

Iconic/flow  control 

Uses  icons  to  represent  interactions 
and  links  them  sequentially  in  a flow 
line  that  depicts  the  actual  result 

CourseBuilder,  Authorware, 
IconAuthor,  Authorware  Attain 

Frame  metaphor 

Uses  icons  to  specify  interactions  and 
links  them  conceptually  providing  a 
structural  flow 

StorySpace,  Digital  Chisel,  Astound, 
Quest,  Multimedia  Fusion 

Hierarchical  object  metaphor 

Uses  an  object  metaphor  like  Object 
Oriented  Programming  which  is 
visually  represented  by  embedded 
objects  and  iconic  properties 

Dazzler  Deluxe,  Docent,  Metropolis, 
MediaSweets,  Toolbook  II  Instructor, 
Quest  Net  +,  Oracle’s  Media  Objects 

Hypermedia  linkage 

Uses  a hypermedia  navigation 
metaphor  to  link  elements 

Frontpage,  Dreamweaver,  Homesite, 
Claris  HomePage 

Tagging 

Uses  tags  in  text  files  to  link  pages, 
provide  interactivity  and  integrate 
multimedia  elements 

SGML,  HTML,  VRML,  3DML 

Cast-score  metaphor 

Uses  horizontal  tracks  and  vertical 
columns  to  synchronize  media  events 
in  a time-based  fashion 

Director,  Flash,  Javascript,  Java 
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Authoring  paradigms  can  be  thought  of  as  organizational  structures  that  facilitate  the  design  of  instructional  materials  and 
learning  activities.  Depending  on  the  paradigm  used  by  a specific  authoring  tool,  the  design  approach,  development  time, 
instructional  capabilities,  and  learning  curve  (ease  of  use),  could  vary  widely  from  one  authoring  tool  to  the  next  Hedberg  and 
Harper  (1998)  emphasize  this  point  by  stating  that:  “The  organizing  metaphor  of  the  authoring  system  has  become  critical  to  the 
effective  design  of  the  final  learning  environment”  (p.  1).  Kasowitz  (1998)  however  insists  that  the  value  of  an  authoring  tool  is 
measured  by  how  well  it  can  support  a particular  designer's  task  regardless  of  its  strength  or  approach.  In  order  to  understand  how 
authoring  tools  impact  a designer’s  task,  it  is  important  to  look  at  the  evolution  of  authoring  tools  from  a technological  and 
pedagogical  perspective. 

Evolution  of  Authoring  Tools 

Authoring  tools  have  evolved  over  the  last  decade  based  on  technological  and  pedagogical  innovations  from  authoring 
bounded,  program- controlled  learning  systems  such  as  Computer-Based  Instruction  (CBI),  to  authoring  unbounded,  learner- 
centered  environments  such  as  Web-Based  Instruction  (WBI).  From  a technological  perspective,  the  Internet  has  revolutionized 
teacher-to- learner  and  leamer-to-leamer  communication  by  making  these  interactions  time  and  place  independent  through  the  use 
of  email,  discussion  boards,  and  other  Internet -based  technologies  that  facilitate  asynchronous  learning  and  information  delivery. 
Web-based  course  management  tools  now  include  such  features  and  components  under  an  integrated  structure.  The  World  Wide 
Web  (WWW)  has  also  dramatically  altered  the  concept  of  hypermedia,  which  is  a crucial  attribute  of  an  authoring  tool’s 
interface.  Hypermedia  has  evolved  from  a predetermined  finite  internal  linking  structure  contained  within  the  boundaries  of  a 
learning  system  to  an  infinite  external  linking  structure  that  knows  no  boundaries.  The  WWW  has  also  changed  the  nature  of 
instructional  content  and  resources  from  a well-defined  and  stable  knowledge  base  to  an  unfiltered  and  dynamic  information 
base.  CDROM -based  authoring  tools  for  example  have  commonly  relied  on  stable  content  to  organize  and  structure  instruction, 
which  is  why  the  resulting  learning  system  is  bounded  and  program-centered.  Alternatively,  Web-based  course  management  tools 
now  include  features  and  components  that  allow  instructors  and  learners  to  modify  content  and  contribute  resources  resulting  in 
flexible  and  active  information  structures. 

From  a pedagogical  perspective,  this  means  more  flexibility  in  the  design  of  WBI.  Depending  on  how  the  tools’  features  are 
used  in  a course  by  the  instructor  and  the  learners,  the  “pedagogical  philosophy”  underlying  the  teaching  and  learning  process  can 
range  from  a strict  instructivist  approach  to  a radical  constructivist  approach  (Reeves  & Reeves,  1997),  A strict  instructivist 
approach  typically  results  in  a Web-based  course  that  has  a tutorial  structure  in  which  the  content  is  organized  by  the  instructor 
and  delivered  or  imparted  to  the  students;  and  a radical  constructivist  approach  typically  results  in  a more  learner-centered 
pedagogy  where  students  use  Web  features  as  tools  to  construct  their  own  knowledge  representations  by  restructuring  content  and 
creating  and  contributing  their  own  resources  to  the  course  structure  (Bannan  & Milheim,  1997;  Reeves  & Reeves,  1997),  It  is 
more  likely  therefore  that  courses  initially  designed  for  traditional  learning  environments  and  later  transformed  to  a Web-based 
format  using  a Web-based  course  management  tool  will  undergo  a pedagogical  reengineering  that  is  more  constructivist  in 
nature  (Dabbagh  & Schmitt,  1998),  The  presence  of  Internet -based  communication  tools,  collaborative  tools,  and  Web  publishing 
tools  in  Web-based  course  management  authoring  systems  make  such  pedagogical  implications  possible. 

Instructional  Products  of  Authoring  Tools 

The  nature  of  instructional  products  has  also  evolved  with  advances  in  authoring  tools.  Interactivity  as  an  instructional 
variable  can  no  longer  be  “trivialized  to  simple  menu  selection,  clickable  objects,  or  linear  sequencing"  as  is  the  case  with  most 
program -controlled  CBI  (Sims,  1995),  Ambron  & Hooper  (1988)  describe  interactivity  as  “a  state  in  which  users  are  able  to 
browse,  annotate,  link  and  elaborate  within  a rich  non-linear  database”  (cited  in  Sims,  1995,  p,  1),  Web-based  course 
management  tools  include  note-taking  tools,  Web  development  tools,  self-assessment  tools,  communication  tools  and 
collaborative  tools  for  learners,  encouraging  a continuous  dialogue  between  the  user  and  the  courseware  such  that  the  learner  is 
productively  and  continuously  active  (Jonassen,  1988).  This  dialogical  view  of  interactivity  seems  to  align  with  a learning 
strategies  perspective  where  learners  are  using  technological  tools  as  cognitive  tools  to  generate  their  own  learning  (Sims,  1995), 
For  example,  with  the  inclusion  of  learner  tools  in  authoring  systems,  learning  environments  are  becoming  increasingly  student - 
centered.  Learners  can  create  and  organize  information  in  a meaningful  way  and  take  responsibility  for  their  own  learning. 

Another  variable  that  has  greatly  influenced  instructional  products  developed  with  authoring  tools  is  the  ease  with  which 
collaborative  activities  can  be  facilitated  with  Internet -based  technologies  embedded  in  Web-based  course  management  tools. 
The  focus  shifts  from  interaction  with  an  instructional  program  to  human  interaction  in  the  context  of  group  activities.  With  user- 
specific  tools,  connectivity,  and  greater  ease  of  use,  opportunities  for  goal-oriented  projects  by  teaming  students  to  work  on 
creating  Web  based  projects  can  be  truly  maximized, 

CBI  and  WBI:  Instructional  Attributes 

In  order  to  better  understand  the  evolution  of  authoring  tools  from  a pedagogical  perspective  it  is  important  to  compare  the 
instructional  attributes  of  Computer-Based  Instruction  (CBI)  and  Web-Based  Instruction  (WBI)  since  these  are  the  two  primary 
instructional  or  ‘courseware’  products  generated  through  the  use  of  authoring  tools,  with  the  understanding  that  CBI  utilizes 
CDROM  (or  non-Web-based)  technology  to  deliver  its  courseware,  and  WBI  utilizes  Internet  (or  Web-based)  technologies.  The 
type  of  delivery  medium  has  played  an  important  role  in  determining  what  instructional  designs  are  possible.  As  Clark  and  Lyons 
(1999)  state:  "The  lesson  that  we  have  learned  over  decades  of  technological  evolution  is  that  each  new  medium  provides 
instructional  capabilities  that  are  unique.  And  each  medium  demands  a new  approach  to  exploit  its  capabilities  for  promoting 
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learning.”  Table  2 compares  and  contrasts  pedagogical  attributes  of  CBI  and  WBI  on  instructional  approaches,  content  features, 
instructional  activities,  scope  of  interaction,  feedback,  and  evaluation. 

Table  2 - Instructional  Attributes  of  CBI  and  WBI 


CBI  - Instructional  attributes 


WBI  - Instructional  attributes 


Lends  itself  to  a program-centered  or  instructivist 
approach  due  to  the  closed-system  nature  of  the 
courseware,  hence  the  need  to  predetermine  the 
instructional  content  and  instructional  interventions 
(automated  delivery) 

Content  is  fixed,  has  an  inherent  structure  and  remains 
generally  stable  no  matter  when  it  is  accessed  by  a user 

Instructional  sequence  and  learning  contexts  are 
externally  driven  by  objectives  and  tasks 

Restricted  to  references  and  resources  embedded  in 
product  (browsing  is  limited  to  the  particular  CDROM) 

Focus  is  generally  on  creating  sequential  media  such  as 
print,  audio  and  video 

Instant  feedback  is  available  through  programmed 
interactions  but  less  potential  for  personal  or  meaningful 
feedback 

Limited  interaction  with  other  learners  and  instructor 

Instructional  activities  typicaly  consist  of  drill  and 
practice  exercises,  trial  and  error  learning  or  simulations 
with  accelerated  rounds  of  skill  practice 

Testing  of  learner  outcomes  generally  involves  pre-tests, 
posttests,  and  multiple-choice  questions 


Lends  itself  to  criterion-referenced  assessment 


Lends  itself  to  a student-centered  or  constructivist 
approach  due  to  the  open-system  nature  of  the 
courseware,  hence  the  potential  of  dynamically  altering 
the  instructional  content  and  instructional  interventions 

Content  is  dynamic,  instructors  and  learners  can 
contribute  new  knowledge  and  add  new  resources  to  the 
course  content 

Instructional  sequence  and  learning  contexts  can  be 
internally  driven  by  learners 

Links  to  a multitude  of  Web  sites  can  be  readily 
embedded  (browsing  is  [un]  limited  to  the  WWW) 

Focus  is  shifting  from  media  delivery  tools  to 
communication  tools 

Leamer-to-leamer  and  instructor-to-leamer  interaction 
options  providing  meaningful  peer  and  instructor 
feedback 

Unlimited  interaction  with  other  learners  and  instructor 

Instructional  activities  generally  consist  of  browsing 
links,  searching  online  databases,  posting  using  threaded- 
discussions  and  email,  Web  publishing 

Testing  of  learner  outcomes  generally  involves  assessing 
communication  skills,  Web-based  projects,  organization 
of  information  and  synthesis  of  content 

Lends  itself  to  authentic  assessment  (peer  evaluations, 
multiple  assessors,  and  multiple  forms  of  assessment) 


A noticable  shift  from  directed  to  open-ended  hypermedia  learning  environments  can  be  detected  in  the  instructional 
attributes  listed  above.  According  to  Hannafm,  Hill  & Land  (1997),  directed  learning  environments  use  “structured  algorithmic 
approaches  to  convey  a discrete  identifiable  body  of  knowledge”  and  “learning  is  externally  driven  via  ecplicit  activities  and 
practice.”  Directed  learning  environments  can  also  be  described  as  bounded  (well-defined),  happening  in  real  time,  instructor  (or 
program)  controlled,  and  relying  on  stable  information  resources  (Chambers,  1997).  The  instructional  attributes  for  CBI  listed 
above  certainly  fit  these  criteria.  Open-ended  learning  environments  (OELE)  on  the  other  hand  emphasize  generative  learning, 
authentic  contexts,  and  guided  discovery  approaches  where  learners  take  responsibility  of  learning  and  evaluate  their  own  needs 
(Hannafm  et  al.,  1997).  Additionally,  in  OELEs  metacognitive  abilities  take  precedence  over  mastering  content  and  asynchronous 
communication  is  paramount  in  supporting  learning  tasks.  WBI  however  can  still  result  in  a directed  approach  if  the  inherent 
features  of  the  Web  are  not  effectively  utilized  by  all  participants  in  the  learning  environment.  For  example,  it  is  possible  to 
design  a Web-based  course  that  is  self-contained  and  requiring  minimal  instructor  intervention  and  interaction  with  other 
learners.  Practice  and  feedback  activities  can  be  embedded  in  a Web-based  course  much  like  they  would  be  in  a CBI  course  and 
learners  can  proceed  through  linearly -sequenced  tutorial-like  content  presentations  at  their  own  pace,  resulting  in  a program- 
centered  learning  environment.  Caution  must  be  exercised  to  insure  that  WBI  is  not  just  CBI  delivered  over  the  Web. 

Classes  of  Authoring  Tools 

Authoring  tools  can  be  grouped  using  several  variables  e.g.  type  of  author/adopter  (e.g.  corporate  developer  versus  teacher 
educator),  type  of  delivery  medium  (e.g.  CDROM  versus  Internet),  type  of  operating  system  (Windows  versus  Macintosh),  type 
of  scripting  metaphor,  cost,  ease  of  use,  range  of  user  base  (e.g.  learners,  instructors,  developers),  level  of  technical  support,  type 
of  interface,  market  share,  media  capabilities,  instructional  design  capabilities,  etc.  In  this  paper,  authoring  tools  are  grouped  by 
the  type  of  delivery  medium  (CDROM  versus  Web-based),  and  the  type  of  instruction  produced  relative  to  the  specific  features 
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of  the  delivery  medium  (CBI  versus  WBI).  The  reason  for  this  grouping  is  based  on  two  principles.  First,  that  the  effectiveness  of 
an  authoring  tool  can  best  be  measured  by  examining  the  types  of  instructional  and  learning  strategies  it  supports  (Dabbagh, 
Bannan-Ritland,  & Silc,  2001);  and  second,  that  to  date,  authoring  tools  have  been  primarily  used  to  develop  two  types  of 
instruction:  Computer  Based  Instruction  (CBI)  and  Web-Based  Instruction  (WBI). 

Although  most  authoring  tools  designed  to  deliver  instruction  on  a CDROM  have  Web  delivery  capabilities  (Internet  “play” 
capabilities  through  the  use  of  plug-ins),  those  tools  were  not  originally  designed  to  take  advantage  of  the  inherent  and  unique 
features  of  the  Web  such  as  connectivity,  asynchronous  communication,  global  accessibility,  and  ubiquitous  use.  Based  on  this 
fiindamental  distinction  we  classify  authoring  tools  into  two  main  categories:  CDROM-based  and  Web-based.  Examples  of 
CDROM -based  authoring  tools  are  HyperCard,  Authorware,  and  Toolbook.  Examples  of  Web-based  authoring  tools  are 
Macromedia’s  Dreamweaver,  Claris  HomePage,  and  Microsoft's  FrontPage.  Figure  1 provides  a visual  of  these  two  classes  of 
authoring  tools  and  some  distinguishing  characteristics  of  each. 


CDROM-based  authoring  tools 


Web-based  authoring  tools 


Figure  1 — Classes  of  Authoring  Tools 


Web-Based  Course  Management  Tools 

Another  class  of  Web-based  authoring  tools  known  as  Web-based  course  management  tools  emerged  when  Web-based 
authoring  tools  were  being  increasingly  used  to  create  Web-based  courses  for  online  learning.  The  need  for  a more  integrative 
structure  to  manage  the  delivery  of  such  courses  and  facilitate  the  migration  from  face-to-face  classroom  instruction  to  WBI, 
resulted  in  the  development  of  “one  stop  one  shop”  applications  such  as  WebCt  and  Blackboard.  Unlike  previous  Web-based 
authoring  tools,  those  tools  include  instructor  tools,  learner  tools,  and  technical  administration  tools  allowing  for  different  types 
of  users,  and  for  multiple  Internet-based  activities  embedded  within  the  tool  itself.  Table  3 lists  the  three  classes  of  authoring 
tools  identified  above  and  the  general  features  of  each. 

Table  3 - Features  of  Authoring  Tools 


Category 

General  Features 

Instructional  Products 

CDROM-based  Authoring  Tools 

• Tool-based  interface 

• Computer-Based  or 

• Utilized  with  CDROM  and 

Computer-Assisted  Instruction 

Examples  include: 

videodisc  technologies 

(CBI/CAI) 

HyperCard,  Authorware,  Toolbook  II, 

• Closed  system  (does  not  allow 

• Simulations 

Director 

user  to  go  beyond  the 

• Games 

boundaries  of  what's  there) 

• Microworlds 

• Content  is  generally  stable 

• Tutorials 

• Most  linking  is  internal,  could 

• Individualized  instruction 

have  external  links  requiring 

• Programmed  Instruction 

firing  up  a browser 

• Self-contained  interactive 

• Can  be  Internet -enabled 

modules 

through  “plug-ins” 

• Mastery  learning 

• Require  installation  therefore 

• “Canned”  or  stand-alone 

operating  system  dependent 

instructional  products 

• Require  a steep  learning  curve 

• Standard  testing  programs 

in  order  to  take  full  advantage 

(e.g.  SAT) 

of  their  features 

• Criterion-based  testing 

• Used  mostly  by  developers 

►0 


BEST  COPY  AVAILABLE 


and  instructional  designers  to 
produce  instructional  software 
• Do  not  have  specific  instructor 
or  learner  tools  (only 
developer  tools) 

Web-based  Authoring  Tools 

Examples  include: 

FrontPage,  DreamWeaver,  Claris 
Homepage,  Homesite 

• Browser  interface 

• Utilized  with  Internet -Based 
technologies 

• Open  system  (allows  user  to 
go  beyond  the  boundaries 
through  external  linking  to  the 
WWW) 

• Extensible 

• Dynamic  content 

• Enables  active/collaborative 
media 

• Require  a steep  learning  curve 
in  order  to  take  full  advantage 
of  their  features 

• Used  by  a variety  of  users  to 
develop  Web  sites  for  multiple 
purposes 

• Do  not  have  specific  instructor 
or  learner  tools 

• Singl  e Web  pages  and 
integrated  Web  sites  for  the 
purposes  of  information 
presentation  to  support 
classroom  instruction 

• Structured  Web  sites  resulting 
in  a variety  of  formats  for 
WBI 

• Personal  and  institutional 
homepages 

• Web  publishing 

• Organization  of  Web-based 
resources 

• Complex  animations  and 
interactions  when  used  with 
high  level  scripting  languages 
(Java,  Javascript,  C++)  and 
other  Web  development  tools 

Web-based  Course  Management 
Tools 

Examples  include: 

WebCt,  Blackboard,  TopClass, 
Virtual-U,  LeamingSpace 

• Browser  interface 

• Utilized  with  Internet-Based 
technologies 

• Open  system 

• Easy  to  use 

• Dynamic  content 

• Enables  active/collaborative 
media 

• Have  specific  tools  for 
instructors,  learners  and 
administrators 

• Embedded  communication 
tools  (email,  discussion 
forums,  group  tools) 

• Used  primarily  to  manage  and 
deliver  online  learning  in 
educational  institutions  and 
online  training  in  corporate 
settings 

• Distance  education  programs 

• Courseware  (WBI) 

• Knowledge  networks 

• Asynchronous  & synchronous 
learning  environments 

• Distributed  learning 
environments 

Scalability  and  Usability 

It  is  evident  from  the  table  above  that  authoring  tools  have  evolved  on  three  fronts:  networkability,  level  of  use  (user-base), 
and  ease  of  use.  CDROM -based  authoring  tools  were  primarily  designed  for  software  developers  and  became  popular 
mechanisms  for  supporting  the  production  of  CBI  where  learners  interact  with  an  instructional  program  to  gain  mastery  of  a 
certain  skill  or  knowledge  (Kasowitz,  1998).  The  World  Wide  Web  (WWW)  shifted  the  focus  of  interactivity  from  interaction 
with  an  instructional  program  to  interaction  with  other  learners  (global  interaction)  (Kearsley  & Shneiderman,  1998),  and  from 
accessing  materials  on  bounded  delivery  vehicles  such  as  CDROMs  to  accessing  unbounded  and  dynamic  information  through  a 
network  of  global  resources  on  the  Web  itself  (Clark  & Lyons,  1999;  Hedberg,  Brown,  & Arrighi,  1997). 

The  WWW  also  created  the  need  for  tools  to  develop  Web  pages  which  began  with  simple  text  editors  to  create  HTML  files 
and  evolved  to  Web-based  authoring  tools  which  continued  to  grow  in  functionality  integrating  more  user  features  and  Internet- 
based  technologies  leading  to  the  development  of  Web-based  course  management  tools.  The  learning  curve  dropped  sharply  with 
Web-based  course  management  tools  as  the  interface  became  more  template-controlled  (“choose-it-and-we’ll-do-it-for-you”)  and 
the  functions  more  context  sensitive,  extending  the  user-base  to  multiple  user  profiles.  With  little  or  no  prior  experience  in 
authoring,  instructors,  learners,  university  administrators,  and  corporate  developers  are  all  able  to  easily  explore  the  potential  of 
these  integrated  tools  to  create,  engage -in,  manage,  and  deliver  online  learning. 
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Current  Authoring  Tools 

Authoring  tools  can  be  traced  back  to  the  1960s  when  “computer-assisted  instruction  was  viewed  as  an  economically  viable 
way  to  distribute  teaching  expertise”  (Huntley  & Alessi,  1987,  p.  259).  In  1997,  Kozel  documented  about  50  commercial 
multimedia  authoring  tools  not  counting  highly  specialized  niche  tools.  Most  of  the  market’s  share  was  spread  among  the  most 
popular  tools:  Aimtech’s  IconAuthor,  Allegiant’s  Supercard,  Allen  Communication’s  Quest,  Asymetrix  Toolbook  II  line 
(Asymetrix  is  now  Click2Leam),  Macromedia’s  Director  and  Authorware,  and  mFactory’s  newer  object-oriented  metropolis. 
Currently,  Macromedia  claims  to  own  80%  of  the  authoring  market  between  their  two  tools,  with  Director  possibly  dominating 
this  market  share.  Other  tools  with  a loyal  following  include:  Claris’  HyperCard,  Oracle’s  Media  Objects,  Apple’s  MediaTool, 
and  Discovery  Systems  CourseBuilder. 

In  a survey  on  the  usage  of  Web  authoring  tools  on  the  Web  conducted  by  Security  Space  in  July  of  2000,  the  following 
WYSIWIG  (What  You  See  Is  What  You  Get)  tools  were  listed  in  descending  order  of  percentage  use:  Microsoft’s  FrontPage, 
Netscape’s  Composer,  Adobe  Page  Mill,  NetObjects  Fusion,  HotMetal  Pro,  IBM  HomePage  Builder,  NetObjects  Authoring, 
Macromedia’s  Dreamweaver,  Allaire’s  Home  Site,  and  IBM  NetObjects  TopPage  However  when  taking  the  mind-set  of  a 
professional  Web  developer  responsible  fora  medium-sized  company's  Web  efforts,  Oliver  Rist  of  IntemetWeek  (1998)  selected 
three  WYSIWIG  Web  authoring  tools  that  are  powerful  enough  to  develop  cutting-edge  pages,  yet  visual  enough  to  do  so  quickly 
and  easily.  The  three  tools  were  Microsoft’s  FrontPage,  Adobe’s  Page  Mill,  and  Macromedia’s  Dreamweaver,  with  FrontPage 
coming  out  on  top  in  terms  of  an  “all-in-one  professional-level  design  and  management  tool”,  and  Page  Mill  and  Dreamweaver 
following  closely  behind.  In  a more  recent  roundup  of  Web-based  authoring  tools,  PC  magazine  in  its  May  4^,  2000  issue  gave 
Dreamweaver  first  place  for  advanced  site  design  with  an  average  user  rating  of  9/10,  and  FrontPage  first  place  for  basic  site 
design  with  an  average  user  rating  of  7/10.  Allaire’s  Home  Site  received  an  honorable  mention  in  the  same  issue  and  was  highly 
recommended  for  developers  who  prefer  complete  control  over  their  HTML  code  due  to  its  thorough  code-editing  capabilities. 

Web-based  course  management  tools  (WBCMT)  represent  yet  another  share  of  the  market  which  clearly  lies  in  the  education 
sector  since  the  main  purpose  of  these  tools  is  to  facilitate  the  management  and  delivery  of  online  courses  to  support  e-Leaming 
and  distance  education  programs.  In  a recent  survey  by  the  U.S.  Department  of  Education's  National  Center  for  Educational 
Statistics  (NCES),  it  was  reported  that  the  number  of  distance  education  programs  increased  by  72  percent  from  1994-95  to  1997- 
98  and  that  an  additional  20  percent  of  the  institutions  surveyed  plan  to  establish  distance  education  programs  within  the  next 
three  years  (The  Institute  for  Higher  Education  Policy,  2000).  It  was  also  reported  that  1 .6  million  students  were  enrolled  in 
distance  education  courses  in  1997-98.  It  is  not  surprising  therefore  that  institutions  and  faculty  members  are  increasingly  feeling 
pressure  to  offer  Web-based  courses  to  meet  economic  and  student  demands  and  the  recent  proliferation  of  WBCMT  seems  to  be 
in  answer  to  this  demand.  Examples  of  WBCMTs  include  WebCt,  Blackboard,  Convene,  Embanet,  Real  Education, 
eCollege.com,  Symposium,  TopClass,  WebMentor,  E-Web,  Web  Course  In  A Box,  Internet  Classroom  Assistant,  Lotus  Learning 
Space,  Softarc’s  FirstClass,  Serf,  Virtual-U,  and  Eduprise  to  mention  a few. 

Eduventures.com  (a  leading  e-Leaming  independent  industry  analyst  firm),  in  an  October  2000  industry  research  report 
established  that  Blackboard  had  the  strongest  market  position  at  the  time  the  report  was  published.  The  same  report  also  stated 
that  WebCt  had  also  reached  a sizable  share  of  the  market  and  that  with  its  partnership  with  Thompson  Learning  it  is  in  an 
excellent  position  to  match  or  even  surpass  Blackboard  in  the  coming  months.  The  report  mentions  Campus  Pipeline  and 
Jenzabar  as  other  WBCMT  category  leaders  in  e-Leaming  (Stokes,  Evans,  & Gallagher,  2000).  Eduventures.com  also  predicts 
that  the  higher  education  e-Leaming  business  will  eventually  be  dominated  by  two  or  three  large  players,  or  maybe  even  one 
“killer”  player.  Other  popular  Web-based  course  management  systems  include  Learning  Space,  Virtual-U,  and  TopClass  (Mann, 
1999).  For  a more  comprehensive  list  of  authoring  tools  and  a comparative  analysis  of  their  features,  visit  Bruce  Landon’s 
Website  at:  http://www.ctt.bc.ca/landonline/index.html. 

Future  Implications  of  Authoring  Tools 

As  discussed  at  the  beginning  of  this  paper,  the  aim  of  authoring  tools  is  to  automate  entirely  or  partially  the  courseware 
construction  process  by  supporting  tasks  such  as  the  ability  to  create  screens,  screen  objects  such  as  menus  and  buttons,  link 
content  to  other  content,  and  sequence  material  (Bell,  1998).  The  lack  of  specific  design  principles  however  often  restricts  the 
kinds  of  instmctional  designs  these  tools  support  leading  in  many  instances  to  the  creation  of  simple  drill  and  practice  programs 
or  uninteresting  tutorials.  "The  result  is  a tool  that  supports  a broad  range  of  possible  instmctional  applications,  some  of  which 
may  be  good,  and  some  of  which  are  likely  to  be  poorly  executed,  but  none  of  which  will  have  been  created  with  much  guidance 
from  the  tool  itself’  (Bell,  1998,  p.  76).  Murray  (1998)  further  emphasizes  this  shortcoming  of  authoring  tools  when  he  states  that 
"commercial  authoring  systems  excel  in  giving  the  instmctional  designer  tools  to  produce  visually  appealing  and  interactive 
screens,  but  behind  the  presentation  screens  is  a shallow  representation  of  content  and  pedagogy”  (p.  6).  So  how  do  we  preserve 
the  level  of  usability  and  the  instmctional  methods  that  instmctional  designers  have  become  familiar  with,  and  add  additional 
tools,  features,  and  authoring  paradigms  that  will  allow  more  powerful  and  flexible  learning  systems  to  be  built? 

There  are  two  approaches  that  attempt  to  answer  this  question  both  based  in  an  information  technology  perspective.  The  first 
proposes  that  Intelligent  Tutoring  Systems  (ITS)  need  to  be  embodied  by  authoring  tools  to  allow  additional  levels  of  abstraction, 
modularity,  and  visualization  in  order  to  achieve  more  powerful  and  flexible  authoring  paradigms  (Bell,  1998;  Murray,  1998). 
The  second  proposes  that  new  metaphors  for  authoring  tools  need  to  be  developed  to  match  current  theory  (Hedberg  & Harper, 
1998),  and  that  "of  all  the  metaphors  likely  to  survive,  objects  stand  the  greatest  chance  because  they  reflect  an  evolutionary 
improvement  in  the  software  engineering  world”  (Kozel,  1 997,  p.  42). 


A New  Authoring  Paradigm 

The  object  model  represents  an  inevitable  evolution  in  application  development  since  interactive  multimedia  is  both  created 
in  and  delivered  as  software  making  it  easier  to  author  complex  programs  by  thinking  of  interactive  elements  as  objects.  The 
organizing  metaphor  of  the  authoring  system  has  become  critical  to  the  effective  design  of  the  final  learning  environment 
(Hedberg  & Harper,  1998).  This  calls  for  a new  authoring  paradigm  allowing  users  to  spend  more  time  designing  at  the 
conceptual  and  pedagogical  level  instead  of  focusing  on  the  features  of  the  tools  to  produce  more  engaging  instructional  designs 
(Robson,  2000).  Currently  for  example,  authoring  tools  for  the  construction  of  Web  documents  offer  a page  metaphor  with 
hypertext  as  the  dominant  link  structure  allowing  more  of  a “top  -down’’  design  process  than  the  screen  metaphor  of  more 
traditional  low  level  multimedia  authoring  tools  such  as  HyperStudio  (Hedberg,  Brown,  & Arrighi,  1997).  All  to  say  that  the 
paradigm  or  metaphor  of  an  authoring  tool  can  guide  or  restrict  the  types  of  instructional  designs  possible. 

Object  Oriented  Designs 

Both  the  ITS  approach  and  the  dominance  of  a 'true'  object  metaphor  have  one  objective  in  common:  introducing  a 
pedagogical  layer  to  an  authoring  tool  in  order  to  enable  the  design  of  more  intelligent  and  flexible  learning  environments. 
According  to  Murray  (1998)  this  is  primarily  achieved  by  representing  instructional  strategy  and  instructional  content  separately 
and  by  modularizing  the  instructional  content  for  multiple  use  and  reuse.  This  principle  allows  for  embedding  the  pedagogy  in  the 
tutor  for  the  proponents  of  the  ITS  approach,  and  in  the  properties  of  objects  for  the  proponents  of  the  object-oriented  model. 
Such  tools  (in  which  strategy  and  content  are  separated)  can  facilitate  the  design  of  instructional  actions  by  modifying  the 
behavior  of  an  intelligent  tutor  (in  the  ITS  case),  or  specifying  the  relationship  between  objects  (in  the  object  model)  based  on  the 
specific  needs  of  the  learner  or  the  pedagogical  characteristics  of  the  content  being  taught. 

Currently  most  authoring  tools  limit  the  designer  to  the  pre-programmed  modules  of  the  tool  and  to  the  underlying 
assumptions  of  highly  structured  instructional  design  models  (Hedberg  & Harper,  1998).  Furthermore,  the  typical  ID  process 
makes  it  difficult  for  instructors  to  communicate  their  content  requiring  instructional  designers  to  see  a lot  of  content  in  order  to 
understand  what  the  instructor  wants  (Robson,  2000).  An  object  oriented  approach  would  resolve  this  problem  since  it  would  be 
easier  for  instructors  to  translate  their  content  into  learning  resources  (e.g.  I use  lectures,  assessment  items,  resources,  etc.)  and 
instructional  designers  can  then  create  a prototypical  environment  matching  the  instructor's  expectations  without  seeing  any 
content  at  all  (Robson,  2000). 

Learners  as  Producers  of  Hypermedia  Learning  Systems 

Another  critical  factor  that  could  impact  the  pedagogical  use  of  authoring  tools  is  whether  the  learner  is  perceived  as  the  user 
ox  producer  of  hypermedia  learning  environments.  Hedberg  et  al.  (1997)  argue  that  if  the  activities  of  the  learner  are  regarded  as 
the  central  focus  in  an  educational  context  then  learners  should  be  thought  of  as  software  (courseware)  producers  rather  than 
software  users  in  the  development  of  educational  software  for  both  bounded  CD-ROM  titles  and  unbounded  Web-based 
resources.  They  propose  the  integration  of  learner  tools  that  allow  users  for  example  to  organize  information  in  a meaningful  way 
by  positioning  elements  on  the  screen,  creating  new  links,  and  generating  multimedia  objects.  Such  cognitive  tools  could  include 
a notebook  to  copy,  edit  and  format  text;  a visual  graphics  tool  to  create  marker  buttons  that  point  to  multimedia  elements  such  as 
video,  audio,  or  pictures  and  enable  the  learner  to  manipulate  those  elements;  and  a cognitive  mapping  tool  (concept  mapping 
tool)  allowing  flexible  information  representation.  The  learners  as  producers  concept  supports  a generative  approach  to  learning, 
which  aligns  with  a constructivist  epistemology. 

Learning  Objects  Systems  Architecture 

Learning  objects  systems  architecture  is  also  paving  the  way  to  support  the  generative  use  of  authoring  tools  (Bannan- 
Ritland,  Dabbagh,  & Murphy,  2000).  A learning  objects  system  adopts  an  object-oriented  approach  for  storing  and  metatagging 
instructional  content  and  instructional  strategies.  Uneditable  media  objects  called  'primedia'  can  be  stored  in  a database  and 
accessed  for  multiple  uses  in  multiple  contexts.  Primedia  can  range  from  low  to  high  granularity  depending  on  their  relative  size 
as  a learning  resource,  with  highly  granular  resources  increasing  the  efficiency  of  online  instructional  support  systems  due  to 
their  greater  potential  for  reusability  (Quinn,  2000;  Wiley  et  al.,  1999).  With  database-driven  websites  becoming  increasingly 
popular  it  is  certain  that  the  future  of  hypermedia  learning  environments  will  be  powered  by  such  technologies  instead  of  the 
static,  'hard-coded'  HTML  documents.  Authoring  systems  will  be  designed  for  the  creation  of  generically  encoded  reusable 
information  allowing  the  design  process  to  proceed  by  specifying  learning  resources,  creating  links  among  the  resources  and 
authoring  content  independently  of  format  (Davidson,  1993;  Robson,  2000).  The  idea  is  to  define  learning  objects  or  resources 
such  that  each  learning  resource  has  specific  instructional  properties  enabling  its  pedagogic  integration  with  other  resources. 
Depending  on  who  creates,  assembles  and  links  these  objects,  the  pedagogical  philosophy  of  the  hypermedia  learning 
environment  can  vary  from  an  instructivist  to  a constructivist  approach  resulting  in  a directed  or  open-ended  taming 
environment  as  discussed  earlier  in  this  paper  when  comparing  and  contrasting  CBI  and  WBI. 

Currently  Web-based  authoring  tools  and  Web-based  course  management  tools  do  not  facilitate  the  construction  of  learning 
objects  however  they  do  support  some  reusability  of  content  due  to  the  inherent  archival  nature  of  the  Web  as  a delivery  medium. 
They  also  support  cognitive  tools  that  enable  users  to  engage  in  reflective  and  collaborative  practices.  In  an  evaluation  of  Web- 
based  course  authoring  tools  conducted  by  Dabbagh,  Bannan-Ritland,  and  Silc  (2001),  it  was  revealed  that  the  intersection 
between  pedagogical  considerations  and  the  attributes  of  Web-based  authoring  tools  yields  the  most  educational  impact.  It  was 
suggested  that  a comprehensive  advisement  mechanism  included  within  Web-based  authoring  tools,  and  providing  guidance  in 
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the  areas  of  pedagogical  approach,  instructional  strategy,  and.  on-line  support  and  resources  will  facilitate  more  effective  and 
engaging  instructional  designs.  Perhaps  such  a pedagogical  advisement  layer  can  be  embodied  by  authoring  tools  using  an  ITS 
approach  or  a learning  objects  systems  architecture  approach  in  the  future  in  order  to  enable  the  design  of  more  intelligent  and 
flexible  learning  systems. 

Conclusion 

This  paper  discussed  the  heuristic  paradigms  and  organizing  metaphors  underlying  authoring  tools  and  their  impact  on  the 
design  of  hypermedia  learning  systems.  In  addition,  the  evolution  of  authoring  tools  from  a technological  and  pedagogical 
persp  ective  was  discussed  by  comparing  and  contrasting  the  instructional  attributes  of  the  two  primary  courseware  products 
developed  using  authoring  tools:  CBI  and  WBI.  Three  classes  of  authoring  tools  were  also  identified  and  the  features  and 
associated  instructional  products  of  each  class  were  provided.  Finally,  an  overview  of  current  authoring  tools,  their  market  share, 
perceived  shortcomings,  and  new  authoring  paradigms  and  their  implications  on  the  design  of  intelligent  and  flexible  learning 
systems  was  discussed. 
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Abstract 

This  paper  describes  two  studies  involving  men  and  women  (n  ~ J 3)  who  participated  in  online  discussions  in  a web-based 
graduate  course.  The  purpose  of  both  studies  was  to  examine  the  frequency  and  types  of  responses  these  students  made  during 
the  course  chats  and  threaded  discussions.  The  first  study  was  occurred  at  the  end  of  the  course  in  a two-week  period.  Students 
were  divided  into  2 small  groups  in  which  one  group  discussed  the  assigned  topic  via  chat  while  the  other  group  discussed  the 
same  topic  via  threaded  discussion.  The  following  week,  these  groups  switched  discussion  formats  to  discuss  a second  topic. 
Results  of  the  first  study  indicate  that  both  chats  and  threaded  discussion  (the  primary  focus  of  the  original  study)  were  valued 
and  had  utility.  However,  an  unanticipated  result  indicated  a variation  in  the  number  and  type  of  statements  made  with  one  of  the 
small  groups  versus  the  other.  It  was  found  that  the  majority  female  group  made  more  frequent  statements  than  the  majority  male 
group.  This  finding  was  contrary  to  our  review  of  research  in  regard  to  2 points:  J)  results  and  discussion  that  indicate  that 
online  discussion  is  an  equalizer  between  men  and  2)  results  that  indicate  that  females  were  less  involved.  Hence,  we  examined 
other  discussion  of  students  using  both  online  formats  involving  at  various  points  within  the  same  course  and  semester.  Results 
of  the  second  study  indicate  that  there  were  some  differences  in  the  types  of  statements,  but  he  amount  of  statements  made  by 
both  genders  on  average  was  equivalent  within  mixed-gender,  large  group  discussions  in  the  sampled  4 weeks.  The  findings  of 
this  second  study  confirmed  CMC  studies  that  indicate  gender  equity.  However,  the  initial  study’s  lesults  are  still  viable. 
Perhaps  when  in  same  gender  groups,  the  genders  use  typical  communication  styles  in  discussion  and  when  in  mixed  gender 
groups,  the  discussions  are  equivalent  in  number  of  statements  for  both  genders.  Additional  research  is  necessary  into  this  issue 
of  gender  and  online  discussion. 

Background  of  the  Studies 

Web-based  and  web-blended  (enhanced)  courses  require  active  and  interactive  participation  with  all  participants  (instructor 
and  students)  and  with  instructional  materials  posted  or  linked  to  a web  site.  Active  participation  is  usually  conducted  by 
locating  or  providing  information  and  posting  this  information  to  a designated  web  site,  such  as  shared  documents,  external  links, 
and  online  biography  of  student  and  staff  information.  Interactivity  requires  the  interchange  of  ideas  with  all  participants.  This 
interchange  of  ideas  occurs  mainly  through  online  chats  or  threaded  discussions.  (Davidson-Shivers,  Muilenburg,  & Tanner,  (in 
press),  2000;  Davidson-Shivers,  Morrison,  & Sriwon^ol,  2001;  Davidson-Shivers  & Rasmussen,  1998,  1999). 

The  computer-mediated  communication  (CMC)  literature  documents  the  dynamics  of  online  discussions  by  various  forms  of 
communication  patterns,  processes,  and  language  styles  (Lawley,  1993;  Adkins  & Brasher,  1995;  Sherry,  1999;  Kara,  Bonk,  & 
Angel,  1999.)  Lawley’s  article  discusses  communication  and  computers  in  terms  of  gender  as  a social  construct  and  the  political 
effects  of  technology,  such  as  the  dehumanizing  aspects  of  technology  and  viewing  the  user  as  shaping  the  process  and 
environment,  especially  virtual  environments.  Adkins  and  Brasher’ s study  of  CM  groups  suggests  that  language  style  of 
powerful  and  powerless  speech  affect  interpersonal  perception — being  that  the  powerful  was  perceived  as  being  more  task- 
attractive  and  competent.  Another  finding  indicated  those  members  perceived  of  higher  status  were  given  more  opportunity  to 
talk.  Machanic  (1998)  discussed  the  need  to  have  institutions  and  instructors  establish  procedures  and  safeguards  to  prevent 
harassment  and  nastiness  occurring  in  online  discussions  and  facilitate  a sense  of  safe  community. 

Other  studies  focused  on  gender  differences  in  communication  patterns  in  online  discussions  (Vrooman,  2001;  Proost,  Elen, 
& Lowyck,  1997;  Wojahn;  1994;  McConnell,  1997;  Ross,  1996;  Herring,  1993;  Allen,  1995).  Herring  also  claims  that  the  belief 
that  CMC  is  inherently  more  democratic  than  face-to-face  communications  may  be  overly  op  timistic  with  respect  to  gender.  In 
her  ethnographic  study  of  academic  professionals,  she  found  that  women  did  contribute  less  and  with  shorter  statements  than  did 
the  males.  Ross  also  found  that  females  participated  less  than  males  in  small  group  discussions  and  discussed  their  families 
whereas  men  did  not  contribute  to  messages  referring  to  family.  While  Mahoney  and  Knupfer  (1997)  suggest  that  research  on 
women  and  language  reveal  that  women  can  experience  linguistic  discrimination  within  CMC  and  that  cyberspace  is  not  a 
gender-neutral  space,  a recent  article  by  Vrooman  argues  that  online  environments  can  produce  a more  equitable  area  for 
communication  from  both  genders. 

In  addition,  Allen’s  case  study  of  men  and  women  using  the  electronic  mail  system  found  no  difference  in  number  of 
messages  sent  an  length  of  time  using  email;  however,  females  rated  the  email  more  highly  in  various  categories  and  reported 
learning  from  co-workers  than  men.  In  addition,  Allen  suggested  that  females  tend  to  be  socialized  by  supportive  and  nurturing. 
Proost,  Elen,  and  Lowyck  in  their  survey  found  that  it  was  experience  rather  than  gender  being  responsible  for  different 
perceptions  of  CMC  environments.  Wojhan  compared  the  adult  communication  patterns  by  gender  using  a bulletin-board 
communication  format;  she  found  that  the  length  of  communication  patterns  of  men  and  women  was  very  similar.  McConnell, 
when  comparing  patterns  of  men  and  women  in  mixed  gender  groups,  found  that  men  tended  to  talk  more  and  longest  in 
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computer  conferencing,  but  that  women  may  be  less  disadvantaged  in  conversations  in  online  than  in  face-to-face.  Hence,  there 
was  some  discrepancy  in  terms  of  contributions  within  CMC  environments. 

Based  on  conflicting  information  within  the’ literature,  the  results  of  two  studies  are  being  examined  in  terms  of  gender  equity 
and  possible  influencing  factors.  The  purpose  of  the  paper  is  to  describe  the  methods  employed  in  the  two  studies,  discuss  the 
results  of  each  study,  and  identify  potential  influencing  factors  that  may  affect  (or  determine)  gender  equity  in  online  discussions 
and  that  call  for  ftirther  research. 


Methodology 

Participants 

Participants  in  the  study  were  graduate  students  (n  = 13)  in  a required  course  for  their  degree  programs  of  study  from  a 
southeastern  regional  university  in  the  USA.  Approximately  two-thirds  of  the  students  were  female.  Based  on  the  survey  results, 
the  majority  of  students  reported  that  they  had  computer  experience  with  some  having  less  experience  with  the  Internet  and 
WWW.  Participation  in  the  discussions  was  a course  requirement.  Confidentiality  of  information  was  maintained  by  having 
surveys  collected  and  coded  by  someone  other  than  the  instructor  and  the  analysis  of  the  discussions  occurred  after  final  grades 
were  posted. 

Course  content,  organization,  and  requirements 

The  course  was  an  introductory  course  on  trends  and  issues  in  instructional  design.  The  course  was  organized  by  weekly 
topics  with  assignments  and  questions  being  posted  to  its  website.  Two  or  three  questions  were  given  with  directions  on  how  to 
post  (either  chat  or  threaded  discussion)  answers  and  replies.  Students  had  a week  to  respond  to  any  listserv  question(s)  and  were 
also  required  to  reply  at  least  twice  to  other  students’  responses  during  the  week.  Typically  one  additional  question  was  scheduled 
for  an  hour  and  a half  chat  during  the  week.  Chats  were  large  group  (whole  class)  in  which  most  students  were  able  to  attend. 
Students  were  also  assigned  particular  readings  as  preparation  for  discussing  the  weekly  topic.  They  were  also  encouraged  to 
draw  on  their  own  experiences,  knowledge  and  skills.  Both  threaded  discussion  and  chat  could  be  copied  and  all  of  the  chats  were 
distributed  to  all  members  of  the  class.  After  the  fifth  week  of  the  term,  students  were  assigned  as  discussion  leaders  to  facilitate 
the  weekly  discussions  in  both  chats  and  threaded  discussions  with  guidance  from  the  instructor.  The  instructor  participated 
directly  in  the  online  chats;  however,  less  so  when  another  student  was  the  discussion  leader.  With  the  threaded  discussions,  she 
added  her  comments  to  a summary  at  the  end  of  the  week  rather  than  commenting  during  the  week. 

Data  Gathering  Procedures 

The  following  procedures  occurred  for  gathering  the  data: 

1 . Obtaining  Transcripts  of  chats  and  threaded  discussions. 

Study  1:  Transcripts  of  the  small  group  discussions  for  chat  and  threaded  discussion  for  week  13  and  14  were  coded  using  a 
coding  scheme  developed  by  the  Davidson-Shivers  et  al  (1999)  based  on  the  work  of  Pibum  and  Middleton  (1998)  and  Williams 
and  Meredith  (1996).  See  Table  1 for  the  coding  scheme. 

Study  2:  Transcripts  of  the  discussions  for  4 different  weeks  (weeks  5,  7,  10  & 15)  were  then  coded  using  same  coding 
scheme.  There  were  two  threaded  discussion  questions  and  one  chat  for  each  week;  all  of  these  discussions  were  whole  class 
rather  than  students  being  divided  into  small  groups. 

2.  Training  of  researchers. 

The  researchers  were  trained  to  use  the  coding  scheme  and  then  coded  each  discussion  transcript  independently.  The 
transcripts  were  coded  by  each  completed  statement/thought  made  rather  than  using  a line-by-line  method.  The  coding  and 
analyses  of  the  discussions  did  not  occur  until  after  the  final  course  grades  were  posted.  Complete  sentences,  incomplete 
sentences,  and  short  phrases  were  considered  as  a statement  if  a new  or  different  thought  was  presented  within  them.  Incomplete 
sentences  or  short  phrases  were  often  used  within  the  chat  due  to  the  speed  and  interactive  nature  of  this  format. 

3.  Handling  discrepancies  in  coding. 

Discrepancies  encountered  in  the  coding  were  resolved  by  review  and  discussion  of  the  statement  and  the  researchers  coming 
to  consensus. 

4.  Surveys  were  analyzed  for  demographic  data,  computer  and  web  experience,  and  attitudes  toward  web-based  instruction. 

Table  /.  Types  of  Discussion  Participation  Coding  Scheme 
SUBSTANTIVE:  messages  that  relate  to  the  discussion  content  or  topic. 

Code  1.  Structuring  - Statements  which  initiate  a discussion  and  focus  attention  on  the  topic  of  the 

discussion.  These  statements  are  often  made  by  the  discussion  leader  or  instructor  (i.e.  ‘Today  we 
are  going  to  discuss  . . or  “This  week’s  discussion  will  focus  on  . . .”). 

2.  Soliciting — Any  content-related  question,  command  or  request  which  attempts  to  solicit  a response  or  draw  attention 
to  something  (i.e.  “What  do  you  think  the  author  meant  by  . . .?”  or  “Give  us  some  support  for  that  assertion.”). 

3.  Responding — A statement  in  direct  response  to  a solicitation  (i.e.  answers  to  questions,  commands,  or  requests). 
Generally  these  are  the  first  response  to  a question  by  a given  individual. 

4.  Reacting — A reaction  to  either  a structuring  statement,  to  another  person’s  comments,  but  not  a direct  response  to  the 
question,  (i.e.  “Your  earlier  statement  got  me  to  thinking  about . . .”  or  “I  agree/disagree  with  Bob  because  . . .”). 

NONSUBSTANTIVE:  messages  that  do  not  relate  to  the  discussion  topic  or  content 
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5.  Procedural — Scheduling  information,  announcements,  logistics,  listserv  membership  procedures,  etc. 

6.  Technical — Computer-related  questions,  content,  suggestions  of  how  to  do  something,  not  related  to  the  topic 
directly. 

7.  Chatting — Personal  statements,  jokes,  introductions,  greetings,  etc. 

8.  Uncodable — Statements  that  consist  of  too  little  information  or  unreadable  to  be  coded  meaningfully. 

9.  Supportive — Statements  that  although  similar  to  chatting,  there  is  an  underlying  positive  reinforcement  to  the 
comment!  (i.e.  “Good  idea!”  or  “Excellent  work!”).  Note:  This  type  was  added  when  the  researchers  met  for 
consensus  of  their  coding  of  the  transcripts. 

Source:  Davidson-Shivers,  Muilenburg,  & Tanner,  1999.  Adapted from  Pibum  & Middleton  (1998)  and  Williams  & Meredith 
(1996). 

Results 

Study  1 

In  the  initial  study,  students  were  randomly  assigned  to  either  a small  group  chat  or  threaded  discussion  during  one  week  of 
the  course  (week  13).  In  the  second  week  (week  14),  the  same  groups  switched  discussion  modes  on  another  topic  question. 
Using  qualitative  methods  and  a coding  scheme,  the  researchers  coded  the  transcribed  discussions  to  find  out  whether  of  the 
students’  participation  were  substantive  (directly  related  to  the  topic)  or  non-substantive  (not  directly  related  to  the  content)  in 
nature.  Results  indicated  that  overall  students’  discussions  included  all  9 types  of  substantive  and  non-substantive  comments. 
However,  it  appeared  that  there  was  a remarkable  difference  in  terms  of  types  and  amounts  of  responses  between  these  two 
groups. 

The  majority  female  group  had  greater  numbers  of  responses  and  reactions  (substantive  categories)  in  both  chat  and  threaded 
discussion  than  did  the  majority  male  group.  In  addition,  the  majority  female  group  made  more  chatting  and  supportive 
comments  (nonsubstantive  categories)  than  did  the  males  during  the  two-week  period.  When  reviewing  these  patterns  in  terms  of 
face-to-face  situations,  the  findings  are  not  that  surprising.  Tannen  (1990)  suggests  that  females  in  face-to-face  conversations 
with  each  other,  tend  to  use  more  words  and  elaborations  as  well  as  supportive  comments  than  do  males  in  similar  situations. 
However,  the  results  are  in  sharp  contrast  to  findings  in  computer-mediated  communication  literature,  which  suggests  that 
communication  patterns  of  men  and  women  in  online  discussions  are  similar  (Wojhan,  1997;  McConnell,  1997). 

Study  2 

In  this  second  study,  the  focus  was  to  analyze  the  interactions  of  these  same  students  in  chats  and  threaded  discussions  drawn 
from  4 different  weeks  (week  5,  7,  10  & 15).  These  discussions  involved  a mixed  gender  group.  Results  were  mixed  in  terms  of 
any  differences  in  which  gender  had  greater  amounts  of  these  two  main  categories  over  the  four  weeks.  In  terms,  of  the  non- 
substantive categories,  the  females  tended  to  have  made  slightly  greater  numbers  of  chatting  and  supportive  comments  than 
males.  When  comparing  male  and  female  discussion  leaders  the  female  showed  a greater  amount  in  those  same  two  non- 
substantive categories  than  did  the  male  leader.  Both  males  and  females  showed  the  greatest  amounts  of  messages  in  the 
responding  and  reacting  (2  of  the  substantive  categories)  and  were  fairly  equal  in  amount  overall.  It  appears  then  that  differences 
in  male  and  female  discussions  are  diminished  when  looking  at  substantive  remarks  in  online  discussions  using  a mixed  gender 
group.  The  results  of  this  second  study  supports  the  idea  that  CMC  tends  to  be  an  equalizer  among  men  and  women  when  in 
mixed  gender  groups.  However,  the  findings  of  the  second  study  do  not  necessarily  negate  the  results  of  the  initial  study.  Perhaps 
when  in  same  gender  groups,  men  and  women  tend  to  revert  to  their  natural  patterns  of  conversation. 

Discussion  of  Results 

As  suggested  in  the  review  of  literature,  perhaps  there  is  opportunity  for  women  to  create  a new  model  for  online  discussions 
and  that  females  are  less  disadvantaged  in  online  discussions  than  in  face-to-face  meetings  after  all  (Wojahn;  McConnell, 
Vrooman).  And  based  on  the  initial  study’s  findings,  perhaps  women  can  make  a strong  contribution  in  online  discussion. 
However,  there  remains  the  question  of  why  the  findings  in  both  studies  may  have  occurred. 

Pure  speculation  may  indicate  that  when  groups  are  same  gender  rather  than  mixed  gender,  communication  may  revert  back 
to  typical  gender-based  communication  (Tannen,  1990).  Tannen  suggests  that  men  use  a parallel  form  of  communication  with 
each  other  which  is  when  each  communicate  his  own  ideas  without  necessarily  commenting  on  the  other’s;  it  tends  to  be  a side- 
by-side  type  of  pattern  without  much  discussion  or  interrelationship  between  the  males’  comments..  Women  tend  to  have  a 
relational  type  of  communication,  in  that  one  female  states  something  that  in  turn  is  commented  on  by  the  other  female  and  in  a 
sense  they  take  turns  sharing  and  supporting  ideas  between  themselves.  Women  often  use  more  words  and  talk  more  than  men; 
estimated  at  25,000  per  day  compared  to  a man’s  1 5,000  per  day  (source  unknown). 

The  increase  in  amount  of  supportive  statements  by  the  female  students  may  be  indicative  of  a tendency  for  this  gender  to 
provide  supportive  comments  to  each  other  (Tannen,  1990)  or  acknowledge  points  made  by  each  other  (Wojahn,  1994). 
McConnell  (1997)  also  suggests  that  men  tend  to  support  conversations  at  the  end  whereas  women  tend  support  conversations 
throughout  the  dialogue.  These  findings  of  the  initial  study,  and  to  some  degree  the  second  finding,  are  similar  to  face-to-face 
conversations  within  same  gender  groups  (Tannen,  1990;  Wojahn,  1994;  McConnell,  1997)  as  well  as  in  terms  of  supportive 
comments  being  a female  charactaistic,  whether  biological  or  socially  constructed  (Herring;  Lawley;  Mahoney  & Knupfer). 


98 


107 


Students  having  had  face-to-face  communication  prior  to  going  online  may  also  have  evoked  a feeling  of  community  and 
allowed  for  initial  impressions  and  perceptions  of  individuals  to  be  made  (Adkins  & Brasher;  Herring;  Lawley).  With  such 
impressions,  the  second  study  may  indicate  that  the  powerful  may  not  have  been  the  typical  perception  of  gender-based  power, 
but  that  of  one  who  displays  confidence  and  higher  status  in  the  group  discussions.  In  addition,  the  face-to-face  may  have 
facilitated  safe  feelings  when  going  online  (Machniac,  1998).  Ross  states  that  when  students  with  high  CMC  skills  may  have 
been  sympathetic  to  those  less  skilled  students  who  encountered  higher  rates  of  technical  problems.  This  phenomenon  appeared 
to  occur  in  the  course  as  well,  again  promoting  a sense  of  community  and  support. 

The  instructor’s  teaching  style  and  manner  helped  established  both  in  the  oncampus  class  and  online  may  have  had  an  effect 
on  sense  of  community.  Procedures  and  safeguards  may  have  been  in  place  so  that  students  felt  safe  as  Machniac  suggests.  There 
is  a tendency  for  males  to  do  more  flaming  than  females(Vrooman;  Ross);  yet,  little  to  none  of  that  occurred  within  the  course. 

Further  Research  Needed 

This  set  of  study  yielded  more  questions  than  it  answered  and  indicates  that  further  exploration  of  communication  patterns 
and  gender  in  online  dialogue  is  needed.  First,  the  methodology  of  the  various  studies  shared  in  this  paper  vary  greatly,  simple 
replication  of  the  various  studies  will  enhance  the  discussion  on  the  issue  of  gender  equity  and  online  discussion.  The  type  of 
group — mixed-  or  same-  gender — may  make  a difference  in  the  types  and  amounts  of  communications  between  aid  within 
gender.  Investigations  of  the  discussion  patterns  with  use  of  whole  class  and  small  groups  need  to  occur  as  well. 

In  addition,  the  effect  of  instructor  presence,  gender,  and  teaching  style  may  make  a difference  on  the  gender  equity  in 
discussions  with  participating  students.  Such  factors  may  relate  to  power  of  speech,  perceptions  of  speaker,  and  language  style  as 
discussed  by  Herring  and  other  scholars  in  the  CMC  literature.  All  of  these  factors  may  affect  the  interactions  of  students  in 
online  discussions  and  ultimately,  the  success  of  web-based  instruction.  It  is  an  issue  worthy  of  further  examination. 

Finally,  examination  of  communication  patterns  of  children  and  adolescents  by  gender  as  well  as  course  content  also  needs  to 
be  considered.  The  topic  of  the  course  as  well  as  age  of  the  participants  could  add  a richness  of  information  to  the  discussion  in 
terms  of  gender  and  online  discussions. 
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Abstract 

How  should  people  behave  when  using  the  computer?  How  did  you  learn  "right"  and  "wrong"  in  cyberspace?  We  surveyed 
teachers  and  students  to  understand  how  each  group  views  ethical  online  behavior.  Our  research  indicates  that  while  students 
and  teachers  share  some  insights,  they  do  not  share  the  same  viewpoints  on  all  cyberethics  issues.  This  discussion  will  address 
the  need  for  a common  forum  and  interactive  ways  for  sharing  ideas  on  cyberethics  issues  to  develop  a student's  ethical  decision- 
making abilities. 

Introduction 

Ethical  behavior  online,  particularly  among  young  people,  is  of  increasing  concern  to  various  segments  of  our  society. 
Before  we  can  influence  this  or  any  behavior,  we  need  to  understand  the  factors  that  shape  it,  including  social  influences, 
cognitive  and  emotional  factors  and  attitudes.  The  purpose  of  this  study  was  to  study  students’  knowledge  and  opinions  about 
online  behavior  and  to  ascertain  whether  their  views  differed  from  those  of  the  adults  around  them.  In  order  to  provide  an 
appropriate  context  for  these  questions,  three  areas  need  to  be  explored.  First,  what  are  the  unique  characteristics  of  ethics  in 
cyberspace  - what  are  the  dilemmas,  and  how  do  we  define  ethics  in  cyberspace?  Second,  what  do  we  know  about  moral 
development  in  general,  and  how  does  it  transfer  into  cyberspace?  Finally,  what  is  the  social  context  of  online  behavior,  and  how 
does  it  impact  ethical  behavior? 

Issue  1:  Ethics  and  Cyberspace 

Ethical  Dilemmas  in  Cyberspace:  Students  often  test  the  social  limits  of  what  is  considered  acceptable  behavior,  both 
online  and  in  the  real  world.  While  it  is  the  major  transgressions  that  create  headlines,  routine  decisions  that  have  ethical 
implications  are  made  every  day  by  students  of  all  ages.  Often  these  decisions  are  made  without  careful  consideration  of  these 
implications  for  individual  students  or  for  our  society. 

Three  key  types  of  issues  have  emerged:  intellectual  property,  privacy/security  and  free  speech/hate  speech.  Intellectual 
property  issues  include  plagiarism,  Internet  file  sharing  programs  like  Napster,  and  software  pirating.  Privacy  and  security  issues 
range  from  hacking  to  surveillance  software  such  as  the  FBI’s  Carnivore  and  unauthorized  cookies  deposited  from  websites. 
Freedom  of  speech  issues  can  be  seen  in  the  discussion  of  online  censorship,  cyberpom  and  cyberhate.  All  three  types  of  issues 
host  multiple  opportunities  for  ethical  decision  making.  Indeed  these  same  issues  are  the  subject  of  several  contemporary  works 
by  lawyers,  philosophers  and  computer  scientists.  (Hamelink,  2001;  Lessig,  1999;  Ludlow,  1996;  Sykes,  1999) 

Definitions  of  ‘‘Ethical”  in  Cyberspace:  Regardless  of  the  particular  issue  under  discussion,  both  students  and  teachers 
question  when  it  is  appropriate  to  transfer  our  understanding  of  ethical  behavior  from  the  classroom  to  the  online  environment. 
However,  students  and  educators  do  not  necessarily  share  the  same  viewpoints  on  cyberethics  issues,  nor  do  they  bring  the  same 
skill  sets  to  the  ethical  decision  making  process.  Most  notably,  while  students  often  are  more  experienced  in  using  online 
technology , teachers  are  more  familiar  with  the  process  of  ethical  decision-making.  Finally,  there  are  many  situations  that  do  not 
have  clearly  defined  social  boundaries.  When  is  it  okay  to  download  and  use  a picture?  Is  a web  site  with  negative  comments 
about  fellow  students  an  example  of  free  speech  or  slander?  Is  bypassing  the  security  of  a system  for  a good  cause  heroic  or 
unethical? 

Issue  11:  Moral  Development  and  Moral  Education 

Although  little  research  on  moral  behavior  in  cyberspace  has  been  conducted,  we  do  have  an  idea  of  how  students  develop 
moral  behavior  generally.  James  Rest  (1983)  identified  four  major  processes  that  contribute  to  moral  behavior:  moral  sensitivity, 
moral  judgment,  moral  motivation  and  moral  character.  Moral  sensitivity  relates  to  an  individual’s  ability  to  recognize  the  moral 
dimensions  of  a situation.  Moral  judgement,  which  has  been  studied  extensively  by  Kohlberg  and  Turiel,  involves  the  ability  to 
decide  which  course  of  action  is  more  morally  sound.  Moral  motivation  recognizes  the  level  of  importance  the  individual  places 
on  acting  morally,  and  moral  character  assesses  the  individual’s  ability  to  persist  in  a moral  course  of  action  even  in  the  face  of 
difficulties.  According  to  Rest,  all  four  processes  interact  to  determine  the  observed  final  behavior.  Research  so  far  suggests 
that  moral  sensitivity  (Bebeau,  1994)  and  moral  judgement  (Schlaefli,  Rest  & Thoma,  1985)  can  be  enhanced  by  educational 
programs.  Moral  motivation  and  moral  character  have  not  been  as  clearly  addressed.  One  component  of  moral  motivation  are 
the  social  norms  of  one’s  reference  group. 

Nancy  Willard’s  research  on  cyberethics  makes  use  of  a similar  model  to  Rest’s.  She  has  proposed  examining  the 
relationship  of  empathy  to  moral  motivation,  and  applied  Bandura’s  social  learning  theory  to  processes  of  disengagement,  which 


may  weaken  moral  character.  (Willard,  1997;  Willard,  2000).  Results  of  her  study  are  forthcoming  and  hold  promise  for 
understanding  how  teens  make  ethical  decisions.  (Willard,  2000). 

Issue  III:  The  Social  Context  of  Behavior  in  Cyberspace 

Efforts  to  Study  the  social  context  of  unethical  cyber  behavior  are  limited.  Recent  sociological  work  on  the  hacking 
community  proves  interesting.  While  headlines  often  portray  the  hacking  community  as  a collection  of  “pathological” 
individuals,  two  studies  address  the  complex  social  dimensions  that  define  and  negotiate  the  boundaries  of  acceptable  behavior 
within  this  community  (Himanen,  2001;  Jordan  and  Taylor,  1998).  The  group  ethos  of  this  particular  community  dictates  that 
hacking  should  not  be  used  for  theft  or  individual  gain.  In  fact,  these  studies  note  that  the  hacking  community  has  developed  it’s 
own  set  of  standards  to  define  “ethical”  hacking.  Jordan  and  Taylor  note  that  this  community  ethos  can  distinguish  between 
hackers  and  cybercriminals  --  those  who  seek  personal  gain  from  their  illicit  actions.  Himanen  contends  that  hacking  represents  a 
change  in  approach  to  work.  Hacking  reflects  “a  general  passionate  relationship  to  work  that  is  developing  in  the  information 
age”  and  “...the  hacker  ethic  is  a new  work  ethic  that  challenges  the  attitude  toward  work  that  has  held  us  in  its  thrall  for  so 
long.”  (Himanen,  2001  :ix) 


Purpose  of  the  Study 

The  current  study  attempted  to  elicit  information  relevant  to  the  issues  described  above.  Specifically,  the  study  was  designed 
to  assess  students’  and  educators’  intuitive  understanding  of  ethics  and  ethical  decision-making,  their  ability  to  apply  that 
knowledge  to  cyber  dilemmas,  their  understanding  of  how  to  teach  and  learn  an  ethical  awareness,  and  their  sensitivity  to  the 
social  context  of  cyberethics  issues. 

Method 

Focus  Group  Formation 

Five  focus  groups  were  conducted  using  modified  Socratic  questioning  (see  below  for  a description  of  the  procedure).  Group 
1 consisted  of  5 post-secondary  educators  from  Marymount  University.  Group  2 consisted  of  5 K-12  educators  from  suburban 
Northern  \Irginia  public  schools.  Groups  3-5  were  drawn  from  counselors  and  campers  at  a summer  computer  camp  in 
Washington  D.C.  Group  3 consisted  of  15  late  elementary  and  middle  school  students  and  Group  4 consisted  of  16  high  school 
students.  The  students  in  these  groups  came  from  a variety  of  neighborhoods  around  Washington  D.C.,  however  most  were  of 
higher  socioeconomic  status.  Participants,  who  were  mostly  male,  were  attending  the  camp  due  to  their  interest  in  computer 
technology.  Their  computer  expertise  ^a^ied.  Although  this  group  is  not  representative  of  the  population  as  a whole,  their 
computer  expertise  and  interest  in  cyberissues  was  desirable.  This  study  was  designed  to  define  issues  and  gather  information 
from  those  students  most  likely  to  be  aware  of  situations  which  require  cyberethical  decision-making..  Group  5 consisted  of  13 
computer  camp  counselors.  These  participants  were  college  students,  most  of  whom  had  significant  interest  and  expertise  in 
computer  technology.  As  with  the  younger  groups,  this  groups  expertise  and  interest  level  was  significantly  greater  than  the 
general  population,  but  this  was  desirable  in  order  to  gather  information  from  students  on  the  “front  lines”.  In  addition,  this 
group’s  experiences  counseling  younger  children  on  their  use  of  computers  meant  that  they  in  some  ways  bridged  the  gap 
between  educators  and  students.  The  focus  groups  were  videotaped  with  the  exception  of  Group  5,  which  was  audio  taped.  The 
authors  were  present  as  observers  for  each  group  session. 

Focus  Group  Procedures 

For  each  group,  objects  related  to  computers  and  cyberethics  issues  were  displayed  on  a table.  These  objects  included  items 
with  an  obvious  connection  to  the  issues  discussed  (e.g.  a computer  hard  drive  and  an  Acceptable  Use  Policy)  as  well  as  items 
that  were  more  tangentially  related  e.g.  plastic  “bugs”  and  "worms".  The  group  facilitator  for  all  of  the  groups  was  an  expert  in 
Paideia  methods  of  seminar  discussion.  The  opening  question  invited  the  group  to  examine  the  items  on  the  table  and  discuss 
their  ideas  with  regard  to  these  items.  Follow-up  questions  were  designed  to  encourage  participants  to  elaborate  on  their  own  and 
each  other’s  ideas.  A final  question  was  designed  to  relate  the  topics  discussed  to  the  participants’  own  experiences. 

The  focus  group  discussions  were  transcribed  and  analyzed  thematically  in  order  to  determine  issues  of  central  importance  to 
each  group  as  well  as  to  determine  the  similarities  and  differences  in  the  issues  raised  by  each  group. 

Results  and  Conclusions 

Four  major  themes  were  identified  across  all  of  the  focus  groups  studied.  Each  theme  is  identified  and  discussed  below  and 
illustrated  with  quotations  from  the  focus  group  discussions. 

Theme  1:  The  cyberethics  "authority"  recognized  by  students  is  much  younger  than  the  offline  ethical 
authority. 

When  asked,  “How  do  people  learn  ethics?”  all  groups  recognized  the  importance  of  parents  in  shaping  a child’s  basic 
understanding.  Students  explain:  “I  think  people  learn  a lot  of  their  ethics  from  their  parents.  Whether  they  get  what  their  parents 
are  teaching  them  or  something  else.  But  I think  it’s  a lot  from  their  parents  or  whoever  raised  them,  and  their  parents  give  them  a 
way  to  react  to  what  others  say  so  they  can  take  it  in  think  about  it  and  what  your  primary  values  are  probably  shared  with  your 
parents.”  (High  School)  Teachers  agreed:  “Kids  look  to  teachers,  parents  ...to  set  examples  by  how  they  behave...  Over  a period 


of  time  you  develop  a hierarchy  of  values.  You  can  compare  a given  situation  and  place  in  context,  (You)  learn  from  parents, 
others  important  in  life  that  you  value  their  judgement,  (You)  have  to  value  it  before  it  makes  a difference  to  you,”  (Educators) 
However,  as  children  get  older,  the  relative  importance  of  parents  declines:  “It  changes  at  different  times  in  their  lifetime. 
Young  kids  are  more  likely  to  accept  parents  and  not  question.  They  question  more  as  they  are  influenced  by  others  and  then  they 
question  anything  parents  say  in  adolescence,”  (Educators) 

Teens  acknowledge  that  teachers  aren’t  always  the  ultimate  authority:  “If  you  think  about  it,,,,  your  teachers  haven’t  gone 
through  these  troubles  (referring  to  cyber  dilemmas)  for  years  and  your  Just  going  through  them  right  now  and  maybe  things  have 
changed  from  when  they  were  kids  so  on  some  points  I think  your  friends  are  wiser  about  what  you’re  going  through  than  your 
teachers  are.”  “Friends  might  put  you  on  the  wrong  track,  (but)  teachers  might  too.  Society  has  changed  a lot,  on  some  points,” 
(High  School) 

When  asked  “How  did  you  learn  right  and  wrong  in  cyberspace?”  students  responded:  “I  taught  myself  right  and  wrong, 
especially  because  on  the  computer  I taught  myself  how  to  fax  things  and  how  to  fix  programs,”  (Elem/Middle) 

In  fact  when  it  comes  to  the  Internet,  the  ethical  course  of  action  is  often  defined  by  those  between  19  and  25  years  of  age: 
the  Internet  culture  is  controlled,  not  by  50  year  olds  or  60-year  olds,  it’s  us.  Our  age.  We  talked  about  when  you  sort  of  lose 
your  ethics  when  you  go  to  college...  You’re  like,  alright!  Ethernet  connection  and  you  start  downloading  more  MP3s  than  you 
would  ever  do,  songs  you  hate,  cause  it  was  a fast  connection.  ,,,my  parents  don’t  really  understand  the  Internet  at  all  so  they 
can’t  really  make  ethical  Judgements  based  on  it.  So  the  ethics  come  from  19-year  old  high  school  students,  the  ethics  that  we 
use  on  the  Internet.  We  say  everybody  does  it,  well  everybody  on  the  Internet  or  most  people  are  Internet  are  probably  under  30, 
The  majority  of  that  (group)  is  probably  under  (the  age  of ) 25.”  (College) 

To  summarize,  students  and  teachers  recognize  the  important  role  parents  play  in  shaping  children’s  basic  understanding  of 
ethics.  Both  groups  agreed  that  as  students  get  older,  the  importance  of  parents  and  adults  declines.  Teens  acknowledge  that 
teachers  aren’t  always  familiar  with  contemporary  dilemmas  resulting  from  new  technologies.  In  fact,  when  it  comes  to  the 
Internet,  ethics  are  often  defined  by  young  adults  ages  1 9 to  25. 

Theme  2:  Students  of  different  ages  use  different  mental  frameworks  to  decide  ethical  online  behavior. 

Online  communities,  like  AOL  or  GeoCities,  provide  definition  to  the  social  boundaries  that  middle  school  students 
acknowledge.  “When  I first  signed  on  to  AOL,  there’s  like  email  that  says  ‘Welcome’,.,  Then  you  check  it  and  we  printed  and 
put  it  right  in  the  center  where  everyone  would  see  it  so  they  would  obey  what  AOL  says.  Like,  don’t  hack,  don’t  mess  around 
with  stuff  (The)  long  email  says  what  you  shouldn’t  do.”  (Elementary /Middle)  “At  GeoCities  it  didn’t  give  me  as  long  a set  of 
rules,  but  the  only  reason  you  can  get  on  there  is  to  get  a web  site.  So  it  has  rules  about  what  you  make  (for)  your  web  site.  When 
I started  reading  them,  there  are  all  the  rules,  pretty  much  the  same:  don’t  download  things  that  are  illegal  things,  don’t  download 
MP3s  on  your  site.”  (Elementary/Middle) 

High  school  students  consider  what  is  appropriate  in  terms  of  the  ’’moral"  or  ethical  impact  their  actions  have  on  others  and 
society  at  large.  “I  think  it’s  appropriate  to  use  (it)  for  school,  but  it’s  not  right  to  send  off  a virus.  You  are  destroying  another 
person’s  property”  “I  had  a romantic  image  of  hackers  like  the  movie  Hackers  for  one,  they  bring  down  an  evil  corporation  like 
the  Westerns  where  the  outlaw  would  kill  the  corrupt  sheriff  or  something  but  a lot  of  times  it’s  not  like  that  at  all  because  major 
corporations  can  lose  billions  of  dollars  and  that  can  cause  many  repercussions  like  downsizing”  (High  School) 

College  age  students  often  use  a "legal"  framework  as  their  guide  to  "ethical"  behavior.  After  extensive  discussion  about  how 
to  decide  what’s  right  in  relation  to  the  Internet,  one  college  student  reflected  on  the  relationship  between  laws  and  ethics:  “We 
keep  talking  about  what  is  legal  and  illegal  as  our  moral  code,  ,,,A  lot  of  people  go  by  The  Ten  Commandments  and  some  of 
those  things  are  law,  official  United  States  law,..  But  they  are  a kind  of  a code  of  ethics  that  we  can  go  by  and  live  by,,,,” 
(College) 

These  students  see  the  line  of  distinction  between  ethics  and  laws  as  a personal  decision  “You  have  to  do  it  personally.  You 
have  to  come  to  personal  decisions,,,”  “, ,,the  reason  that  we  come  to  law  is  that  ethics  are  personal,  whereas  laws  are  something 
that  we  share,  that  are  attempting  to  sort  of  approximate  the  general  ethics  without  imposing  on  ethical  differences,”  (College) 
Importantly,  all  age  groups  learn  right  and  wrong  in  cyberspace  from  their  hands-on  experience,  “People  have  to  learn  things 
from  experience  and  I think  that  the  only  way  of  learning  something  no  matter  how  much  you  believe  them,  you  really  need  to 
learn  for  yourself”  (High  School)  In  fact,  students  suggest  that  ethics  in  general  is  learned  through  experience.  “I  don’t  believe 
that  any  form  of  ethics  is  learned  directly,,.  Ethics  is  learned  indirectly  from  various  sources.”  (High  School) 

To  summarize,  students  of  different  ages  use  different  mental  frameworks  to  decide  ethical  online  behavior.  Middle  school 
students  use  online  communities,  like  AOL  or  GeoCities  to  define  social  boundaries.  High  school  students  consider  what  is 
appropriate  in  terms  of  the  ’’moral’’  or  ethical  impact  their  actions  have  on  others  and  society  at  large.  College  age  students  often 
use  a "legal"  framework  as  their  guide  to  "ethical"  behavior.  For  all  age  groups,  right  and  wrong  in  cyberspace  is  learned  from 
hands-on  experience. 

Theme  3:  When  considering  ^’appropriate"  and  "inappropriate"  behavior  online,  some  issues  are  clear  for 
both  teachers  and  students,  however  many  others  are  not. 

Advances  in  technology  create  new  challenges  for  ethical  decision-making.  While  educators  insist  most  cases  are  still  a 
matter  of  ’’basic  values,’’  students  at  the  college  level  suggest  that  cyberrules  are  different.  Discussing  intellectual  property, 
educators  explain:  “ You  need  to  show  (students),,,  they  can  achieve  what  they  want  in  an  honest  way,  these  MP3s  things  are  a 
good  example,  a lot  of  people  understand  why  that  might  be  wrong  because  they  respect  the  artist.  They  don’t  (necessarily)  see 
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intellectual  ideas  as  property  the  way  music  is  the  artists’  property.”  For  this  educator,  it’s  merely  a matter  of  drawing  parallels 
between  intellectual  property  that  students  recognize  and  the  new  situations  that  emerge  in  cyberspace. 

Younger  students  often  express  confusion  over  what’s  "legal,"  particularly  when  it  comes  to  downloading  music:  “It  depends, 
...if  you  download  live  album  tracks,  that’s  illegal.  But  you  can  go  to  places  like  Listen.com  and  it’s  legal  you  can  download  it  to 
a CD.  Places  have  a license  to  have  you  copy  it,  that’s  what  they  are  for.”  (Elementary/Middle)  When  referring  to  using  graphics 
and  music  from  an  online  web  site,  another  high  school  student  explains:  “ If  the  person  tells  you  ok,  the  person  with  the  Web 
page  and  MP3s,  (s^s)...  click  here  if  you  want  it,  (they  are)  giving  (you)  permission.  As  long  as  you  have  your  friend’s 
permission.”  (High  School)  “Like  I was  saying  about  plagiarism  laws,  it’s  really  not  illegal  until  you  start  trying  to  make  money 
off  somebody’s  idea.  I have  copied  things  on  CD  into  MP3  files  because  of  ease  of  access.  I don’t  try  to  sell  it  or  spread  it 
around  online  so  it’s  ok.”  (High  School) 

Educators  are  aware  of  the  challenges  posed  by  these  new  technologies,  but  maintain  that  our  current  standards  offline  apply 
in  the  online  environment.  “ Technology  has  made  it  so  much  more  complex.  There’s  the  issue  of  sampling  in  music,  are  you 
going  to  footnote?  It’s  done  by  serious  musicians,  classical  musicians.  It’s  hard  to  know  where  to  draw  the  lines.  O using 
Photoshop  and  making  something  new  out  of  it.  That’s  serious  art.  How  do  you  acknowledge  that?  We’re  in  difficult  water  here. 
It  was  pretty  clear  about  footnotes  and  books,  but  it’s  harder  with  more  arcane  technology.  There’s  proper  research  and  I think 
it’s  important  that  educators  and  parents  teach  the  proper  way  to  do  research.  We  (teachers)  have  to  define  material  that  is 
correct.” 

In  spite  of  these  new  challenges,  educators  emphasize  the  need  to  use  traditional  techniques.  “That  takes  us  back  to  old 
fashioned  sources  of  good  documentation,  good  references,  how  to  check  out  something  that  might  be  fraudulent.  It’s  harder  now 
because  it’s  click,  click,  click  away.  We  need  to  ground  kids  in  book  culture  until  we  know  how  to  find  our  way  in  the  new 
culture.” 

College  students  clearly  articulate  some  of  the  complexities  involved.  For  example,  they  suggest  that  it  is  "harder  to  see  a 
victim  with  any  online  crime.  It  all  started  with  cassette  tapes.  ...I  mean  nobody  in  their  right  mind  actually  thought  ..  .I’m  doing 
this  horrible  thing  when  they  ...copied  cassette  tapes.  And  then  you  got  the  Internet  and  all  of  a sudden  instead  of  just. ..  copying 
cassette  to  cassette  tape,  you're  copying  CDs,  you're  copying  books,  you're  copying  art.  Everything  suddenly  with  the  Internet 
went  from  a small  issue  to  ...the  point  where  you  can  copy  entire  movies,  you  can  copy  entire  books  in  seconds.  And  I think  our 
ethics  weren't  ready  to  expand.  . . .Anything  I could  copy  in  an  hour  wasn't  that  important."  (College) 

Students  see  the  online  environment  as  distinct  from  the  offline  world.  They  acknowledge  that  the  way  people  interact  with 
one  another  online  adheres  to  different  standards.  “You  act  different  when  you’re  being  ethical  on  the  Internet,  than  when  you  are 
being  ethical  in  real  life.”  “I  mean  I’m  not  ON  the  Internet.  There  are  parts  of  my  life  that  I (am)  showing  through  a website... 
...you  don’t  put  forward  your  whole  self.  ...You’re  maybe  an  image  of  yourself,  but  not  your  real  self  We  can  be  more  free  with 
other  people.  We  take  advantage  on  the  Internet.”  “I  think  basically  what  it  is,  is  the  Internet  is  just  a lot  less  real  to  people  than 
the  real  world.  And  that’s  why  it’s  called  cyberspace.  It’s  not  called  the  real  world.”  “...people  just  don’t  think  of  the  Internet  as  a 
real  thing  with  real  consequences.”  (College) 

But  more  than  just  a place  where  inhibitions  or  even  better  judgement  are  left  by  the  way  side,  students  question  the 
fundamental  assumptions  concerning  what  is  right  and  wrong  in  cyberspace.  When  discussing  intellectual  property  in  the  online 
environment,  they  explain:  "it’s  like  when  you’re  a little  kid  you’re  told  don’t  steal  that  book  because  it’s  his  book.  If  you  steal 
that  book  he  can’t  use  it.  On  the  Internet,  if  you  steal  his  book,  he  still  has  his  copy  of  the  book  and  now  it’s  just  you  have  a copy 
of  the  book  too.  ...the  standard  ethical  rules  you  drive  into  little  children  just  don't  apply  as  much  on  the  Internet  as  they  do  in  the 
working  order  world  that's  out  there  in  other  places."  (College)  These  same  questions  have  been  posed  by  philosophers  at  MIT 
(Ludlow,  1996). 

To  summarize,  some  issues  are  clear  for  both  teachers  and  students.  Other  issues  are  not  so  clear  when  it  comes  to 
considering  "appropriate"  and  "inappropriate"  behavior  online.  Advances  in  technology  create  new  challenges  for  ethical 
decision-making.  While  educators  insist  most  cases  are  still  a matter  of  "basic  values,"  college  students  suggest  that  cyberrules 
are  different. 

Theme  4:  Educators  can  encourage  ethical  behavior  by  recognizing  the  type  of  situations  that  make  unethical  behavior 
attractive  and  challenging  students  to  channel  their  technological  expertise  in  positive  directions. 

Educators  are  well  aware  that  students  have  a great  deal  more  technical  knowledge  than  they  do:  “It  will  take  all  our  brains 
and  energy  to  keep  up  with  them.”  But  they  are  optimistic  that  most  students  want  to  do  the  right  thing.  They  believe  must 
establish  clear  examples  of  what’s  “right.”  “It’s  still  a real  minimal  number  compared  to  the  whole.  98%  do  what’s  right  99%  of 
the  time.  Kids  who  steal  cars  now  and  then.  (The  main)  issue  is  being  consistent  and  clear.  This  is  appropriate  and  acceptable.  It 
should  not  be  a mystery.  Let  kids  know.  It  really  needs  to  be  spelled  out,  taught,  probably  from  kindergarten.” 

Students  suggest  that  educators  focus  on  the  positive  and  keep  the  list  short!  “When  you  first  sign  up  to  AOL  you  get  a 75 
page  email  or  so  anyway,  it’s  really  long  and  you’re  supposed  to  read  the  whole  thing  long  long  thing  (we  were)  so  anyways  if 
you  read  it  there’s  50  million  rules  and  it’s  weird  it’s  annoying  also  they  never  what  they  tell  you  what  you  can  do,  just  what  you 
can’t  do.  It  would  be  better  if  they  sent  the  can  dos  and  then  you  have  a shorter  list  to  read  and  you  can  picture  the  bad  things.” 
(Elementary/Middle) 

Reinforcing  ethical  behavior  requires  educators  to  actively  engage  students  in  a dialogue.  The  younger  generation  learns  by 
doing!  “Past  experience  has  a lot  to  do  with  it.  Have  to  learn  things  the  hard  way,  from  experience.  If  people  just  tell  you 
something  you  really  have  to  see  if  for  yourself”  (High  School)  “You  have  to  do  it  personally.  You  have  to  come  to  personal 
decisions  or  you  can  tell  kids  to  do  that  themselves....  So  I think  that’s  one  way  that  they  interact.”  (College) 
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Educators  recognize  some  situations  make  unethical  behavior  attractive  and  emphasize  the  importance  of  carefully  evaluating 
expectations.  “I  think  the  pressures  on  them  - they  talk  about  how  they  have  to  perform  at  such  a level  in  10  different  activities 
and  be  able  to  sleep  two  hours  a night  and  be  the  king  and  queen...  A lot  of  youngsters  rationalize  - 'I  can  get  this  done,  I just 
need  some  help  from  cyberspace,  a buddy,  a friend,  etc.’  Either  they  don’t  see  that  or  they  do  see  and  the  pressure  is  so  high. 
That’s  their  way  of  meeting  those  demands  placed  on  them.  Especially  in  high  school,  starting  in  about  7th  grade.” 

Still,  there  are  those  students  who  engage  in  unethical  behavior  just  for  the  challenge.  “It’s  a game  to  some  people,  you  can’t 
get  caught,  can’t  get  killed,  like  a game  or  a test  of  their  ability.  “It’s  a challenge  they  need  and  the  Pentagon  is  the  ultimate 
challenge.  Go  to  a more  impenetrable  challenge.”  “...For  a lot  they  are  kind  of  total  losers  in  real  life,  (who)  keep  to  themselves, 

(and  have)  no  power in  cyberspace  I have  power.  What  a power  trip  hacking  into  the  Pentagon.  Kids  with  no  friends.  T beat 

the  Pentagon  hack  protectors.’  ” (High  School) 

Finally,  it  is  important  that  educators  provide  opportunities  that  challenge  those  students  with  technological  expertise.  By 
channeling  their  expertise  in  a positive  direction,  teachers  can  provide  students  with  opportunities  to  take  ownership  and  act 
responsibly.  “One  of  the  problems  at  my  school  is  vandalism.  They  were  taking  the  balls  out  of  the  mice.  It  gets  really  old.  The 

Lab  was  inoperable We  talked  to  SGA  (Student  Government  Association)  to  get  the  students  involved...  What  are  the 

consequences  of  this  kind  of  vandalism?  You’re  behind  on  your  project  because  someone  thinks  it’s  funny,  but  you  are  burdened. 
Only  faculty  and  staff  were  owning  this  problem.  We  asked  them  to  help  us  deal  with  it.  It’s  kind  of  like  when  I first  started 
teaching.  My  father  told  me  to  find  the  biggest  kid  in  the  room  and  make  him  your  friend.  It  was  good  advice.  Find  the  smartest 
computer  kid  in  the  room,  get  help  from  them,  help  them  resolve  problems.  ...Sometimes  they  do  it  for  the  challenge  - that’s  a 
good  value,  looking  to  explore  new  territory.  Adults  have  to  channel  that  into  something  constructive.  ...That  talent  should  be 

directed  somewhere  where  it  doesn’t  destroy  property Channeling  it you  (can)  do  a tech  based  project  and  have  them 

help  you  teach  the  project.  All  of  a sudden  they  have  ownership  and  they  aren’t  going  to  screw  that  up  to  lose  the  power  or  the 
satisfaction.  They  (the  students)  are  teaching  the  teacher.  (It’s)  called  pride.  You  give  them  back  to  channel  it.  They  were  sitting 
there  all  day  putting  inappropriate  pictures  on  the  screen  savers.  Then  you  say  can  you  put  in  some  pictures  of  the  Civil  War, 
using  the  same  skills.  And  citing,  of  course!”  (Educators) 

To  summarize,  educators  can  encourage  ethical  behavior  by  recognizing  the  type  of  situations  that  make  unethical  behavior 
attractive.  In  addition,  it  is  important  to  provide  opportunities  that  challenge  students  to  channel  their  technological  expertise  in 
positive  directions.  Students  who  take  ownership  of  their  work  are  less  likely  to  act  in  ways  that  are  destructive  to  the  online 
community. 

Future  Directions 

At  present,  the  most  common  response  of  educational  institutions  to  the  challenges  of  cyberethics  has  been  to  hand  out  an 
Acceptable  Use  Policy  (AUP)  and  a set  of  plagiarism  guidelines,  require  students  and  parents  to  sign  it,  and  wait  for  the  ball  to 
drop.  As  this  paper  demonstrates,  the  questions  these  issues  raise  cannot  be  addressed  by  a single  statement  of  principles, 
however  well  crafted.  The  answers  are  too  varied  and  unclear  for  easy,  unequivocal  answers.  No  AUP  can  anticipate  every 
possible  problem  that  may  arise.  Instead  of  prescribing  policy,  students  and  educators  need  to  establish  an  ongoing  process  of 
ethical  discourse  when  dealing  with  cyberethics  issues. 

If  we  see  cyberethics  issues  as  more  akin  to  other  types  of  moral  dilemmas,  the  research  on  moral  education  can  provide  us  with 
some_direction.  Narvaez,  Bebeau  & Thoma  (1999)  noted  that  Rest’s  four  stage  model  can  be  used  to  classify  various  approaches 
to  moral  eduation  in  K-12  schools.  According  to  their  analysis,  moral  education  in  the  public  schools  historically  tends  to  reflect 
society’s  particular  moral  concerns  which  have  ranged  from  the  “macromorality”  concerns  of  the  1960s  (the  Civil  Rights 
movement,  anti-war  protests)  to  current  pre-occupation  with  attempting  to  curb  perceived  youthful  antisocial  aggressiveness  by 
emphasizing  self  control  and  “10  Commandment”  type  prescriptions  for  moral  behavior.  Using  the  four  component  framework, 
this  reflects  a shift  in  emphasis  from  moral  judgment  to  moral  character.  The  effectiveness  of  current  “character  education” 
approaches  has  not  yet  been  established  (Leming,  1997  in  Rest,  et.  al.,  1999).  It  would  seem  logical  that  approaches  that  address 
multiple  components  of  moral  behavior  should  be  more  effective  than  those  that  address  only  one  component.  Thus,  any 
approach  to  cyberethics  education  should  address  issues  of  moral  judgment,  moral  character,  moral  motivation  and  moral 
sensitivity. 

Insights  from  brain-based  learning  research  (Caine  and  Caine,  1991)  also  can  guide  cyberethics  education.  This  research 
indicates  that  students  must  connect  what  they  are  trying  to  learn  with  what  they  already  know  to  retain  the  knowledge. 
Traditional  presentations  of  instructional  material  aganize  that  material  into  the  educator’s  categories  of  knowledge.  With  a 
subject  like  cyberethics,  this  traditional  approach  is  particularly  unlikely  to  yield  effective  results  since  the  student’s  categories  of 
knowledge  may  differ  quite  significantly  from  those  of  the  teacher. 

Educators  and  students  need  to  engage  in  dialogue  that  can  help  both  groups  question  their  decisions  concerning  the  use  of 
ethical  standards.  In  this  process  it  is  important  to  recognize  that  everyone  will  not  come  up  with  the  same  answer  or  agree  on  a 
single  “ethical”  course  of  action.  All  can  agree  that  decisions  should  be  based  on  sound  reasoning  and  ethical  principles. 

Where  ethical  standards  can  be  applied,  we  need  to  document  the  harm  of  ignoring  an  ethical  course  of  action.  This  can  help 
students  develop  moral  sensitivity  and  enhance  moral  motivation  as  they  see  the  consequences  of  their  behavior  on  what  are 
usually  perceived  as  “invisible”  victims. 

All  of  us  -students  and  teachers  together-  must  actively  question  the  changing  boundaries  that  result  from  technology 
transfer.  These  questions  cross  many  disciplines  in  the  humanities  and  social  sciences: 

• How  do  students  of  different  ages,  developmental  stages,  and  cultures  grapple  with  “ethical”  in  the  online  environment? 

• How  are  social  boundaries  changing  and  thus  defining  what  is  appropriate  and  inappropriate  online? 
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• What  social  forces  shape  these  definitions? 

• How  do  economic  and  political  pressures  influence  our  judgments  and  behavior? 

Using  the  interactive  nature  of  the  Internet  itself  may  prove  to  be  a very  effective  means  of  facilitating  this  kind  of  ethical 
discourse  about  online  behaviors.  The  linear  features  of  online  technology  allow  students  to  select  multiple  paths  through  new 
material  rather  than  one  set,  prescribed  course  of  study.  Digital  “scaffolding”  techniques  can  assist  students  as  they  apply  abstract 
concepts  to  contemporary  issues.  Scaffolding  refers  to  the  guidance  students  are  given  while  completing  a complex  task  like 
ethical  decision-making.  Students  are  given  only  the  amount  of  help  that  is  needed,  as  that  help  is  needed.  The  teacher  gradually 
withdraws  assistance,  allowing  the  student  to  complete  the  task  independently.  Scaffolding  is  designed  to  support  students 
through  more  complex  tasks  with  the  amount  of  support  they  need,  and  no  more  (Jonassen,  Peck  and  Wilson,  1999),  Using  these 
techniques  to  engage  students  in  brain-friendly  ways,  and  addressing  all  aspects  of  the  four-stage  model  described  above  should 
yield  a rich,  comprehensive  approach  to  exploring  cyberethics  issues  and  developing  students’  moral  sensitivity,  judgment  and 
decision-making  capabilities. 

The  stakes  involved  in  becoming  a more  ethical  society  online  are  not  trivial.  With  our  increasing  dependence  on  computer 
technology  for  bsks  as  significant  as  maintaining  medical  records,  controlling  air  traffic,  ensuring  national  security,  and 
conducting  businesses  large  and  small,  the  unethical  actions  of  a few  have  the  potential  to  impact  society  on  a massive  scale. 
Helping  our  students  acquire  the  awareness,  the  motivation  and  the  means  to  become  ethical  decision  makers  in  cyberspace  is 
critical  in  order  to  prevent  such  impacts  from  occurring. 
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Abstract 

There  is  a growing  interest  among  organizations  in  identifying  and  nurturing  Communities  of  Practice.  However,  defining  a 
Community  of  Practice,  distinguishing  it  from  a team,  and  nurturing  the  growth  of  a forming  Community  of  Practice  can  prove  to 
be  challenging.  By  considering  two  case  studies  in  a posthoc  fashion,  two  potential  communities  of practice  will  be  defined  and 
characterized.  Finally,  a discussion  of  how  best  to  nurture  a newly  formed  community  will  take  place. 


Introduction 

Delineating  what  is  and  what  is  not  a Community  of  Practice  (CoP)  is  not  a clear-cut  task.  Although  many  definitions  of 
Communities  of  Practice  exist,  assessing  the  degree  to  which  a group  meets  the  components  that  make  up  the  criteria  of  these 
definitions  is  often  subjective  and  difficult  at  best.  One  specific  area  of  difficulty  is  distinguishing  between  a team  and  a CoP. 
Because  there  are  strong  similarities  between  the  two,  it  is  often  helpful  to  outline  the  areas  where  teams  and  CoPs  differ.  A final 
area  of  difficulty  with  regard  to  a CoP  is  determining,  if  one  does  indeed  exist,  how  best  to  nurture  its  existence  in  an  effort  to 
provide  fertile  soil  for  its  development.  This  paper  will  look  at  these  three  areas:  (1)  defining  a CoP,  (2)  distinguishing  between  a 
CoP  and  a team,  and  (3)  identifying  ways  to  nurture  a CoP.  The  framework  for  addressing  these  questions  will  be  from  the 
perspective  of  a posthoc  review  of  two  case  studies:  the  ISUnion  Web-design  Team  and  the  actual  virtual  community  developed 
by  the  Web-design  Team.  The  purpose  of  the  Web-design  Team  is  to  develop  an  electronic,  web-based  infrastructure  to  support 
the  development  of  a virtual  community  of  dialogue.  The  purpose  of  the  virtual  community  is  to  nurture  dialogue  among  existing 
practitioners,  faculty  and  current  students  in  the  area  of  performance  technology  and  instructional  systems  design. 

The  ISUnion  is  an  interactive  web  site  designed  and  created  by  graduate  students  of  the  Instructional  Systems  (IS)  Program  at 
an  academic  institution  in  the  Southeastern  United  States.  The  ISUnion  consists  of  components  such  as  an  interactive  forum, 
news,  a knowledge  management/survey  center  and  an  employment  center.  The  original  goal  of  this  web  site  is  to  build  a virtual 
community  of  students,  alumni  and  faculty  to  develop,  improve  and  promote  the  IS  Program  at  the  institution.  This  website  came 
into  existence  in  September  2000. 

Communities  of  Practice 

Diverse  definitions  of  Communities  of  Practice  exist,  yet  they  have  many  similar  characteristics.  The  phrase  "community  of 
practice"  was  first  used  in  1991  by  Etienne  Wenger  and  Jean  Lave.  In  1998  Wenger  published  the  work,  "Communities  of 
Practice:  Learning,  Meaning,  and  Identity"  which  describes  communities  of  practice  as  a "community  created  over  time  by  the 
sustained  pursuit  of  a shared  enterprise.”  Wenger  further  clarified  Communities  of  Practice  as  “groups  of  people  who  share 
information,  insight,  experience,  and  tools  about  an  area  of  common  interest”.  (Wenger,  1998) 

The  Value  of  a Community  of  Practice 

Why  are  communities  of  practice  worth  identifying  and  nurturing?  They  bring  the  power  of  a self-organizing,  social  entity  to 
play  in  an  otherwise  often  highly  structure  world.  “Self-organizing”  means  that  it  is  based  on  a degree  of  intrinsic  motivation  and 
perceived  value.  People  are  a part  if  the  community  because  they  want  to  be.  When  they  no  longer  find  value  in  the  community, 
they  either  change  the  community  or  leave  it.  The  fact  that  a CoP  is  a social  entity  brings  the  dynamics  of  the  complexity  and 
interrelations  of  two  or  more  persons  interacting.  This  relation  is  seen  to  be  exponential  in  its  power  as  opposed  to  linear. 

Many  organizations  including  business,  government,  and  not-for-profits,  have  seen  the  potential  of  communities  of  practice. 
Organizations  such  as  Hewlett-Packard,  the  Department  of  Defense  and  the  World  Bank  have  all  developed  technical  means  of 
supporting  communities  of  practice.  Often  this  takes  the  form  of  discussion  boards  or  electronic  data  sharing. 

Because  of  the  change  in  our  global  economy  from  that  of  being  an  industrial  society  to  that  of  an  information  sharing 
society,  organizations  see  the  value  of  capturing  the  knowledge  that  exists  within  communities  of  practice.  Within  knowledgp 
management  frameworks,  organizations  not  only  attempt  to  capture  existing  data  and  information,  but  they  also  see  the  value  of 
capturing  knowledge,  expertise  and  wisdom.  Today’s  workforce  no  longer  places  their  loyalties  with  a single  company. 
Whereas  a few  years  back  it  was  not  uncommon  for  a worker  to  retire  after  thirty,  forty  or  even  fifty  years  with  a single  company, 
today,  individuals  value  career  moves.  They  develop  specializations.  Workers  move  to  where  the  career  opportunities  allow 
them  to  increase  their  expertise.  From  the  knowledge  management  perspective,  organizations  develop  electronic  forums  for 
communities  of  practice  to  share  their  knowledge,  expertise,  and  wisdom.  Because  of  the  electronic  infrastructure,  this  is  now 
archived  information,  a living  artifact,  which  an  organization  can  now  index  and  search.  When  the  individuals  leave  the 
organization,  at  least  parts  of  their  tacit  knowledge  has  been  captured  by  the  organization. 

But  simply  setting  up  a discussion  board  alone  may  not  provide  a nurturing  environment  for  a community  of  practice  to 
emerge.  Communities  of  practice  take  time  to  form.  They  follow  an  ever-changing  pattern  of  ebb  and  flow.  Indicators  that  a 
community  of  practice  has  formed  would  include  the  nature  of  the  structure,  the  commonalties  and  the  artifacts  or  products  of  the 
community  (Wenger,  1998). 


A CoP  is  formed  spontaneously  as  people  seek  help,  try  to  solve  problems,  develop  new  ideas  and  approaches  (McDermott, 
1999).  People’s  interest,  time  and  energy  direct  their  participation  in  CoP.  A CoP  develops  in  various  stages  with  different  levels 
of  interaction  among  its  members  and  different  kinds  of  activities.  A CoP  is  different  from  a business  or  functional  unit  in  that  it 
defines  itself  in  the  members’  understanding  of  their  practice.  A CoP  is  different  from  a team  in  that  it  is  defined  by  knowledge 
rather  than  by  task.  A CoP  is  different  from  a network  in  that  it  is  not  just  about  social  relationships. 

Communities  of  Practice  are  groups  of  people  who  share  similar  goals  and  interests.  In  pursuit  of  these  goals  and  interests, 
they  employ  common  practices,  work  with  the  same  tools  and  express  themselves  in  a common  language.  What  keeps  people 
together  is  a common  sense  of  purpose  and  a real  need  to  know  what  each  other  knows. 

Members  of  a Community  of  Practice  learn  from  each  other  in  solving  common  problems  and  evolving  a more  skilled  and 
creative  practice.  They  are  committed  to  jointly  developing  better  practice.  Communities  of  Practice  are  pivots  for  the  exchange 
and  interpretation  of  information.  They  leverage  the  preservation  of  tacit  knowledge  and  keep  up  with  the  development  of  the 
cutting-edge  technology. 

Case  One:  The  Web-design  Group  of  the  ISUnion  Project  as  a Potential  Community  of  Practice 

The  Web-design  Group  consists  of  twenty -five  members  who  participate  in  varying  degrees  in  the  design  of  a webspace 
structure  to  support  the  potential,  virtual  community  of  dialogue.  In  reviewing  the  development  of  the  infrastructure  in  a posthoc 
fashion,  it  is  beneficial  to  outline  the  differences  between  a team  and  a community  of  practice.  Several  articles  have  been  written 
to  look  at  these  similarities  and  differences  and  to  determine  how  the  strengths  of  each  can  best  be  utilized  within  an  organization 
(McDermott,  1999  & Sharp,  1999).  Figure  1 below  presents  key  distinguishing  differences  between  teams  and  communities  of 
practice. 
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The  structure  of  the  Web-design  group  is  based  on  outlined  project  plan  and  individual’s  abilities  to  move  the  project 
forward.  While  the  group  was  self-appointed  (volunteer),  there  was  a strong  marketing  effort  by  core  members,  which  included 
benefits  of  participation  such  as  training  on  HTML  coding  and  graphics  applications. 

The  commonalties  of  the  Web-design  group  included  a shared  vision  for  the  desired  outcome.  However,  the  group  was 
driven  by  the  goal  of  completing  the  project.  There  was  no  desire  for  long-term  sustainability  of  the  design  group.  Rather,  a 
workplan,  which  had  a specific  end  point,  was  the  driving  force.  While  many  shared  stories,  language  and  jargon  were  noticed,  it 
is  believed  that  this  is  in  part  due  to  the  common  environment  of  the  participants.  All  were  students  within  the  same  programs, 
attending  classes  with  the  same  professors,  and  moving  through  the  same  track. 

Finally,  the  products  of  the  group  related  directly  to  the  project.  Whereas  with  a community  of  practice  where  the  products 
tend  to  be  a defining  element,  the  Web-design  group  looked  at  the  products  as  completion  points.  They  did  not  define  the  group 
in  any  way. 

Based  on  the  activities,  social  dynamics  and  structure  of  the  group,  and  nature  of  the  group’s  products,  the  ISUnion  Web- 
Design  group  could  be  considered  a team  rather  than  a Community  of  Practice.  The  social  dynamic  as  it  currently  exists,  is  strong 
and  with  the  strong  core  value  of  ’’perceived  value  in  the  idea  of  the  ISUnion"  shared  by  the  group,  the  team  will  most  likely 
develop  into  a CoP  in  time.  However,  as  it  currently  stands,  it  would  be  better  defined  as  a team  with  strong  social  interactions,  a 
shared  belief  in  the  value  of  the  project  and  a firm  knowledge  and  trust  of  the  abilities  and  skills  of  the  other  group  members. 

Case  Two:  A Virtual  Community  as  a Community  of  Practice-  The  ISUnion 

Is  the  ISUnion  (Virtual  Community)  a CoP?  While  it  may  be  too  early  to  answer  this  question,  the  ISUnion  is  a potential 
CoP  because  its  structure  and  the  target  interaction  conform  to  the  characteristics  of  a CoP. 

Organizational  / Social  Structure: 

Members  of  the  ISUnion  are  students,  alumni  and  faculty  of  the  IS  Program  at  a Southeastern  United  States  academic 
institution.  They  are  informally  attached  to  one  another  through  exposure  to  the  common  field  of  instmctional  design  or 
improving  human  performance,  and  they  are  in  common  pursuit  of  solutions  to  improve  human  performance.  They  hold  a 
warehouse  of  knowledge  and  skills  in  Instructional  Systems  and  Human  Performance  Technology.  Additionally,  they  come  from 
the  same  institution,  which  provides  a common  theoretical  underpinning  for  the  study  of  the  particular  discipline. 

But  the  ISUnion  is  different  from  a formal  organization.  People  who  participate  are  all  doing  it  out  of  their  common  interest 
in  solving  human  performance  or  learning  problems,  and  they  all  speak  in  the  ISD  jargons  like  “ instructional  objectives  and 
learning  hierarchy”.  Their  motivation  to  participate  is  intrinsic  as  no  reward  or  incentive  system  exists  to  encourage 
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participation.  People  build  friendships  and  learn  from  each  other  at  virtual  and  face-to-face  social  occasions  in  the  format  of 
chatting,  sharing  experience  and  story  telling. 

Roles  of  Participants  in  the  ISUnion  from  the  Perspective  of  a Commnnity  of  Practice 

Wenger  describes  several  roles  of  an  individual  within  a community  of  practice.  The  are: 

• Inbound 

• Insider 

• Outbound  (Wenger,  1 998) 

“CoPs  arise  out  of  people’s  natural  desire  to  share  ideas,  get  help,  learn  about  new  ideas  verify  their  thinking,  and  hear  the 
latest  “professional”  gossip”  (McDermott,  1998).  They  develop  as  people  have  regular  contact  with  people  who  share  their 
interests. 

Newcomers  of  a CoP  usually  take  an  “observational”  lookout  post,  both  absorbing  and  being  absorbed  in  the  culture  of  the 
community  of  practice.  They  assemble  a general  idea  of  what  constitutes  the  practice  of  the  community.  Their  tasks  are  usually 
short  and  simple  with  small  costs  of  errors  and  little  responsibility.  Newcomers  usually  perform  at  the  end  branches  of  work 
processes  and  start  from  the  ground  for  self-evaluation  (Lave  & Wenger,  1991). 

How  does  this  description  relate  to  the  role  of  a newcomer  in  the  ISUnion?  People  reach  out  to  the  ISUnion  from  different 
angles.  People  may  have  heard  about  the  ISUnion  from  their  department  listserv,  available  flyers,  or  by  word  of  mouth.  They 
begin  with  observing,  which  is  to  check  out  news  first,  to  see  what  is  going  on  out  there  and  what  the  web  site  has  to  offer  to  its 
target  audience.  Then,  they  will  lurk  on  the  discussion  board,  and  jump  in  whenever  they  feel  interested  or  have  time  to  join  the 
discussion  if  they  feel  that  the  discussion  facilitated  by  the  ISUnion  is  of  value  to  them  for  their  professional  development. 

“Inside”  community  members  frequently  help  each  other  solve  problems,  offer  each  other  advice,  and  develop  new 
approaches  or  tools  for  their  business  or  study.  Helping  each  other  makes  it  more  convenient  for  community  members  to  improve 
their  weaknesses  and  learn  together  in  the  “public  space”  of  the  community  (McDermott,  1998).  “As  they  share  ideas  and 
experiences,  people  develop  a shared  way  of  doing  things,  a set  of  common  practices  and  sources  of  individual  identify”  (Lave  & 
Wenger,  1991). 

As  a member  at  full  participation,  insiders  usually  have  a deeper  sense  of  the  value  of  participation  to  the  community. 
Learning  of  an  insider  lies  in  becoming  a part  of  the  community.  Insiders  commit  a greater  amount  of  time  and  an  intensified 
effort.  They  assume  more  and  broader  responsibilities  and  take  more  risky  and  difficult  tasks.  They  enjoy  an  increasing  sense  of 
identity  as  a master  practitioner  (Situated  Learning,  1991). 

People  enter  the  ISUnion  to  check  out  news  regularly,  to  share  experience  bravely  and  to  search  for  answers  to  their  unsolved 
problems.  People  participate  in  the  ISUnion  can  perform  similar  jobs  or  collaborate  on  a shared  task  or  work  together  on  a 
product.  Maybe  some  of  them  work  together  at  the  same  company  or  in  the  same  classes,  maybe  some  are  working  on  a mutual 
assignment.  They  are  peers  in  the  execution  of  the  real  work.  What  holds  them  together  is  a common  sense  of  purpose,  which  is 
to  improve  their  ISD  practice,  and  a real  need  to  communicate  what  each  other  knows  in  order  to  broaden  their  insights  and 
vision  in  the  ISD  field.  They  ask  each  other  for  advice  on  completing  their  ISD  projects,  they  share  successful  or  painful 
experience  in  past  projects  and  discuss  about  the  direction  of  development  in  the  ISD  field. 

Students  talk  about  their  accomplishments  and  concerns,  and  express  their  expectations  and  ambition  for  our  program. 
Alumni  share  stories  of  their  practice  in  the  field  or  make  known  their  admiration  and  loyalty  for  their  alma  mater  reflecting  on 
the  foundation  they  built  while  there,  and  helping  to  strengthen  our  current  program  by  sharing  their  information,  experience  and 
talent  in  the  IS  field.  Faculty  members  maintain  a close  connection  with  students  and  alumni  for  a common  purpose,  to  express 
their  thoughts  on  a program  they  greatly  influence  (ISUnion  Proposal,  2000).  At  the  same  time,  more  experienced  and  well- 
respected  alumni  or  faculty  members  can  act  as  thought  leaders  or  coordinators  of  the  CoP  to  initiate  activities  and  guide  the 
direction  of  people’s  interactions. 

A community  of  practice  is  a living  entity,  in  a “constant  state  of  motion”;  evolution.  As  such,  people  move  towards  the 
edges  of  the  community,  following  an  outbound  path  towards  the  ever-changing  boundary  “defining”  the  community  of  practice 
from  the  outside  “world”.  There  are  several  reasons  why  a member  of  a CoP  would  be  on  an  outbound  path.  These  include: 

• Conflicts  or  shifts  in  time  and  energy 

Since  CoP  is  not  an  official  required  organization  for  people  to  join,  people’s  participation  in  a CoP  depend  on  their 
individual  commitment  of  time  and  energy.  Even  though  participating  in  CoP  is  valuable  to  people,  it  is  easily  ignored 
because  of  other  more  pressing  tasks  (McDermott,  1999).  Also,  if  a person’s  energy  level  does  not  allow  him  to  attend  any 
CoP  activity  after  work,  this  person  will  very  likely  to  choose  to  rest  for  next  day’s  work  instead  of  going  for  a CoP 
meeting. 

• Conflicts  or  shifts  in  values 

Another  important  issue  that  caused  people  to  leave  a CoP  is  the  conflict  or  change  in  the  value  of  an  individual  and  the 
shared  value  of  a CoP.  Sometimes,  a community  member  joins  a CoP  when  his  or  her  temporary  needs  or  value  match  the 
CoP  value.  After  a while,  his  or  her  needs  or  value  may  shift  according  to  the  change  in  his  or  her  situation  such  as  a job 
shift.  This  person’s  value  may  not  match  the  one  of  the  CoP  any  more  after  he  or  she  changed  his  or  her  job,  and  he  or  she 
will  choose  to  leave  the  previous  CoP,  or  maybe  join  another  CoP.  • 

• CoP  not  engaging  or  effective  enough  to  hold  people  together 

Because  of  the  emergent  and  spontaneous  nature  of  a CoP,  sometimes  it  is  not  well  organized  or  managed  efficiently.  If  the 
CoP  activities  are  not  focused  on  knowledge  and  skills  essential  to  both  the  practice  and  the  people;  if  there  is  no 
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respectable  community  members  or  thought  leaders  involvement  or  coordination;  if  personal  or  community  interactions  are 
not  active  or  enthusiastic,  community  members  will  lose  interest  in  the  CoP  activities  and  the  CoP  is  going  to  diminish. 
Specific  to  the  ISUnion,  potential  reasons  why  people  would  be  on  an  outbound  path  are: 

• Not  enough  involvement 

It  is  possible  that  people  will  leave  the  ISUnion  if  it  does  not  have  enough  opportunities  for  individual  or  community 
interaction.  Most  people  would  not  like  to  participate  if  they  see  very  few  people  interact  with  each  other,  and  there  are  no 
major  professional  figures  (experts)  within  the  participants.  They  would  consider  that  this  CoP  is  not  very  popular  among 
the  practitioners  in  the  ISD  field,  and  guess  that  there  might  be  less  value  in  committing  time  and  energy  to  this  CoP. 

• Not  active  or  current 

It  is  likely  that  people  will  stop  observing  or  lurking  on  the  ISUnion  if  they  find  out  that  there  is  not  much  change  in  the 
topics  or  feedback  of  the  discussion  forum,  or  the  number  of  hits  to  the  web  pages  have  very  slow  and  slight  increase.  They 
would  think  that  the  topics  discussed  on  this  web  site  is  not  related  to  the  most  current  core  value  so  that  nobody  would  like 
to  contribute. 

• Not  entertaining 

Due  to  the  self-emergent  and  self-organizing  nature  of  a CoP,  the  stability  of  the  community  can  easily  deteriorate  if  the 
contents  or  activities  provided  by  the  ISUnion  are  not  attractive  enough  to  hold  its  visitors  for  a comparatively  long  period 
of  time.  It  will  be  very  difficult  for  a member  of  the  ISUnion  to  have  to  discipline  him-  or  herself  to  keep  coming  back  to 
visit  the  web  site  if  there  is  not  any  fun  or  interesting  elements  related  to  the  field,  which  appear  on  the  web  site  regularly 
and  frequently. 

• Time  and  energy 

Some  people  may  have  to  stop  attending  the  ISUnion  temporarily  or  permanently  if  their  Job,  study,  or  personal  load  does 
not  give  them  any  space  for  any  other  commitment.  For  example,  there  is  a fluctuation  in  the  number  of  hits  to  the  ISUnion 
between  ordinary  school  days  and  the  week  for  the  final  exam  because  most  of  the  participants  are  working  in  the 
educational  area. 

What  are  the  potential  benefits  of  a CoP? 

Communities  of  Practice  provide  an  opportunity  for  people  to  solve  real  problems  in  ways  that  formal  processes  can  not 
anticipate.  There  is  always  a formal  ideal  way  of  performing  a task  or  completing  a study  assignment  such  as  Service  Standards 
or  the  requirement  on  a course  yllabus;  But  after  we  do  things  in  the  real  settings  for  a period  of  time,  we  may  have  pulled  out 
from  the  Service  Standards  or  syllabus  a gist  of  the  ideal  way  or  have  formed  our  own  thoughts  or  experience  of  performing  the 
task.  This  set  of  informal  or  improvised  knowledge  is  what  we  can  share  with  each  other  within  a Community  of  Practice  to 
become  an  efficient  and  skillful  performer  of  our  tasks,  but  what  we  can  not  always  obtain  from  formal  education  or  instruction. 

Learning  is  more  about  becoming  a member  of  a community  than  it  is  about  absorbing  information.  It  is  considered  a social 
process  built  around  informed  participation.  This  applies  to  the  learning  of  tacit  knowledge,  which  refers  to  “intuition,  judgment, 
and  common  sense — the  capacity  to  do  something  without  necessarily  being  able  to  explain  it”.  Tacit  knowledge  exists  in  the 
special  practices  and  relationships  that  emerge  from  working  together  over  time.  People  draw  out  the  tacit  knowledge  that  others 
possess  by  having  human  interactions  to  construct  a common  context  to  understand  each  other  and  the  trust  to  be  willing  to  share 
ideas.  (Brown  & Gray,  1995) 

Why  do  people  want  to  join  ISUnion  and  Can  we  benefit  from  ISUnion  ? 

• The  ISUnion  is  a badge  of  identity. 

People  who  participate  in  the  ISUnion  identify  themselves  as  a current  or  previous  member  of  the  IS  program  and  the  field. 
These  people  will  be  interested  to  find  out  about  what  happened  in  the  program,  what  progress  the  program  has  made  and 
what  the  most  up-to-date  research  or  development  is  in  the  program  and  the  IS  field.  The  perceived  quality  of  the  program 
is  a reflection  on  all  past,  present  and  future  participants  in  the  IS  Program. 

• The  ISUnion  is  a place  to  meet  people. 

People  involved  in  the  ISUnion  activities  are  the  ones  who  really  would  like  to  meet  (may  be  virtually)  with  other  people 
working  or  studying  in  the  same  field,  and  to  form  a social  relationship  with  them.  Now  more  than  ever,  people  realize  the 
importance  of  networking  with  people  working  in  the  same  field.  They  try  to  improve  their  learning  and  collaboration  by 
building  trust  between  each  other  first.  For  this  purpose,  they  will  actively  take  part  in  most  of  the  activities  proposed  by  the 
ISUnion  for  them  to  meet  people  face  to  face  or  virtually.  An  additional  benefit  of  the  ISUnion  is  the  forming  of  a web  of 
participation  and  networking.  The  ISUnion  web  site  presents  the  most  current  news  of  events  happened  in  the  IS  program. 
It  is  updated  every  month  with  the  most  recent  stories  of  events  and  photographs.  People  come  to  the  ISUnion  to  read  about 
what  has  just  happened  and  what  is  going  to  happen  so  that  they  can  share  the  common  topics  when  they  encounter  other 
members  of  the  community.  People  get  together  face-to-face  when  there  is  any  conference  or  social  event  announced  by  the 
ISUnion,  and  these  events  become  the  major  “real”  (as  opposed  to  virtual)  human  interactions  of  the  CoP. 

• ISUnion  is  a way  to  learn. 

Another  major  reason,  hopefully  the  most  important  one,  for  people  to  be  engaged  in  ISUnion  is  to  learn  from  each  other 
and  to  produce  better  practice.  The  discussion  topics  posted  on  the  interactive  forum  can  be  a big  attraction  to  the  target 
audience.  People  can  raise  their  questions  or  problems  by  posting  them  into  the  forum.  If  other  people  find  the  topics 
interesting  and  relevant  to  their  particular  problems  or  experiences,  they  would  try  to  offer  solutions  or  share  their  similar 
encounter  by  carrying  out  an  asynchronous  on-line  discussion.  Other  members  who  visit  the  web  site  may  also  regard  this 


exchange  of  information  valuable  to  their  job  or  study.  Thus,  learning  will  be  disseminated  within  the  whole  group  of  target 
audience. 

The  ISUnion  offers  an  alternative  way  to  learn  about  the  real  work  experience  from  alumni  and  faculty  members,  about  how 
academia  relates  to  “reality”  and  how  the  combination  of  theory  and  practice  is  dynamic  in  nature.  Summaries  of  interviews  of 
alumni  and  faculty  posted  on  the  web  site  inform  us  about  stories  of  achievements  and  failure.  These  real-world  stories  may  be 
different  from  the  ideal  settings  and  methods  learned  from  the  courses  offered  by  the  IS  program.  These  stories  provide  us  with  a 
context  to  connect  classroom  study  with  the  real-world  requirements,  and  create  a more  realistic  view  of  the  ISD  field. 
Conversely,  a theory  “worked  out”  on  the  web  site  via  dialogue  may  have  a change  effect  on  a current  practice  in  the  field. 

Via  the  ISUnion,  we  have  the  opportunity  to  learn  what  b impossible  for  the  academic  IS  program  to  offer — the  tacit 
knowledge  by  becoming  a member  of  the  community.  The  tacit  knowledge  that  is  embodied  by  the  community  of  the  ISUnion 
refers  to  the  intuition,  judgment  and  common  sense  serving  the  purpose  of  improving  human  learning  and  performance.  This  tacit 
knowledge  can  only  be  acquired  by  involvement  with  the  students  and  practitioners  in  the  ISD  field.  The  ISUnion  offers  us  one 
more  opportunity  to  interact  with  these  people  and  to  have  one  more  exposure  to  and  possibly  working  with  this  embedded 
knowledge. 

Nurturing  the  Community 

Having  demonstrated  that  the  community  for  which  the  ISUnion  is  serving  is  in  fact  a community  of  practice,  and  that  there 
is  a potential  benefit  to  the  community  for  using  ISUnion,  we  must  now  determine  how  to  increase  its  use.  At  present  we  have 
identified  four  areas  of  potential  improvement.  The  four  areas  that  are  currently  lacking  are  involvement,  interaction, 
time/energy,  and  entertainment. 

Involvement,  for  our  purposes,  is  defined  as  communication  between  community  members.  Some  possible  ways  to  increase 
involvement  are  storybuilding,  case  debates,  and  increased  discussion.  Storybuilding  would  consist  of  individuals  ‘writing’  a 
collective  story.  A second  proposed  technique  is  sponsorship  of  case  debates.  The  concept  is  to  post  elements  of  an  instructional 
or  performance  analysis  and  allow  community  members  to  discuss  and  debate  subsequent  actions.  A final  proposal  to  increase 
involvement  is  to  increase  activity  on  the  discussion  board.  A couple  of  ideas  in  this  area  are  to  have  recognized  leaders  and 
experts  in  the  field  moderate  discussions  and  to  restructure  the  mechanics  of  the  discussions  so  that  they  are  not  linearly 
structured. 

Interaction,  for  our  purposes,  is  defined  as  utilization  of  the  site.  Examples  of  ideas  to  increase  community  use  of  the  site  are 
a case  study  database,  electronic  library,  day-in-the-life-of  documentaries,  and  daily  updated  material.  A case  study  database 
could  be  a repository  of  cases  studied  within  the  classroom,  experienced  in  the  Performance  Systems  Analysis  course,  and 
encountered  by  alumni.  Such  case  studies  would  include  as  many  elements  as  possible  from  the  initial  contact  hopefully  through 
a complete  evaluation.  Such  a repository  would  be  of  immense  benefit  for  many  different  members  of  this  community,  from 
students  learning  the  processes  for  the  first  time  to  veterans  of  the  field  doing  research.  A second  idea  to  increase  the  member 
interaction  would  be  to  digitize  current  videos  from  the  Learning  Resource  Library.  Such  videos  represent  seminars  with  leaders 
of  the  field  as  well  as  recorded  courses.  Other  material  could  also  be  added  to  the  library,  such  things  as  lecture  notes, 
PowerPoint  presentations,  and  old  tests.  Another  element  for  increasing  interaction  could  be  documentaries  of  leaders  in  the  field 

and  alumni  of  the  program.  A final  idea  is  to  have  daily  “ ’s”.  This  could  be  a daily  quote,  daily  word,  daily  wisdom,  and 

etceteras. 

Time  and  energy  seem  to  be  in  short  supply  for  many  members  of  this  community.  As  such,  this  site  should  help  reduce  time 
and  energy  expenditures  of  the  community  members.  Some  ways  to  accomplish  this  could  include  posting  projects  from  the 
Computer  Courseware  and  Electronic  Performance  Support  Systems  classes  that  might  be  of  benefit  to  the  community.  Such 
courses  could  include  accessibility  requirements,  copyright  laws,  and  software  introductions.  Instructors  can  use  the  site  within 
courses.  An  example  of  such  use  might  be  to  interview  alumni  online.  Interviews  are  currently  a requirement  in  one  course,  by 
conducting  them  online  there  is  a record  which  others  can  benefit  from.  Furthermore,  course  credit  could  be  given  for  work  done 
on  the  actual  site.  The  program  that  supports  the  site  is  not  very  strong  technically  and  some  students  feel  this  is  a deficit  with  the 
program.  By  providing  credit  for  technical  work  on  the  site,  the  site,  the  student,  and  the  program  gain  some  benefit.  A final 
method  to  reduce  time  and  energy  consumption  is  to  use  the  site  discussion  board.  At  present  the  online  courseware  supports 
discussion  but  does  not  provide  a notification  option.  The  ISUnion  discussion  board  does  provide  this  option. 

Finally,  the  site  could  be  made  a bit  more  entertaining.  This  could  be  accomplished  through  the  use  of  such  elements  as  jokes, 

comic  strips,  and  interactive  games.  There  could  also  be  a -cam.  This  would  be  something  along  the  lines  of  a fish-cam,  but 

with  an  object  of  interest  to  the  community. 

All  of  these  potential  methods  of  nurturing  the  community  are  technical  and  require  deployment  time.  Many  of  them  also 
require  a repository  in  place  prior  to  deployment.  Several  also  require  individuals  to  commit  to  moderating  discussions  or  provide 
material.  However,  much  of  the  material  does  exist  or  would  not  require  much  effort  to  prepare  for  the  web.  The  question  that 
must  be  answered  is  whether  the  extra  effort  provides  a large  enough  benefit  to  the  community  to  be  worthwhile. 

Conclusions 

A Community  of  Practice  takes  time  to  develop.  Communities  of  Practice  are  organic  and  self-organizing.  They  need  time  to 
find  what  kind  of  information  to  share,  who  the  participants  are  and  the  “best”  methods  to  communicate  (McDermott,  1998).  For 
example,  McDermott  mentions  the  experience  of  Shell’s  Corporation  with  the  nurturing  of  successful  communities  of  practice. 
They  started  with  a group  of  six  to  eight  people  meeting  weekly  to  discuss  some  technical  issues.  It  took  them  six  months  for 
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word  to  spread  of  the  value  of  these  discussions.  Then  the  number  of  people  that  attended  the  weekly  meetings  increased  to  about 
forty. 

ISUnion  is  created  as  an  intentional  community  of  practice,  which  is  “intentional  in  its  focus”.  But  to  develop  the  trust, 
connection  and  sharing  of  natural  communication,  it  is  necessary  to  support  the  natural  process  of  community  development 
instead  of  imposing  an  artificial  one  (McDermott,  1998). 

Are  these  two  entities  of  the  ISUnion  destined  to  be  communities  of  practice?  Only  time  will  tell. 
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Abstract 

Current  teacher  educators  are  being  challenged  to  find  opportunities  for  their  preservice  teachers  to  develop  both 
competence  in.  and  confidence  for,  integrating  technology  into  their  curricula.  Given  the  difficulty  involved  in  trying  to  arrange 
successful  classroom  technology  experiences  for  preservice  teachers,  this  study  was  designed  to  examine  whether  electronic 
models  of  exemplary  technology-using  teachers,  presented  via  CD-ROM.  could  provide  a viable  alternative  for  developing  ideas 
about  and  self-efficacy  for  technology  integration.  Sixty-nine  students  enrolled  in  a one-credit  technology  course  completed 
demographic  and  online  survey  instruments  before  and  after  interacting  with  a CD-ROM  that  featured  six  teachers’  classroom 
technology  beliefs  and  practices.  Results  suggest  that  electronic  models  can  significantly  increase  preservice  teachers  ’ ideas 
about  and  self-efficacy  for  technology  integration.  Furthermore,  students’  found  the  examples  of  teachers  included  on  the  CD- 
ROM  to  be  both  realistic  and  relevant.  Implications  are  discussed  as  well  as  suggestions  for  future  research. 

Introduction 

According  to  the  most  recent  report  of  the  National  Center  for  Education  Statistics  (NCES,  2000),  nearly  70  percent  of 
teachers  report  not  feeling  well  prepared  to  use  computers  and  the  Internet  in  their  teaching.  The  1998  Technology  in  Education 
Report  (Market  Data  Retrieval)  noted  that  only  7 percent  of  schools,  nationwide,  boast  a majority  of  teachers  at  an  advanced  skill 
level  (i.e.,  able  to  integrate  technology  into  the  curriculum). 

Even  among  our  newest  teachers,  instructional  use  is  not  as  high  as  might  be  expected.  Contrary  to  popular  belief,  preservice 
and  beginning  teachers  do  not  use  computers  significantly  more  than  their  more  experienced  colleagues  (Hadley  & Sheingold, 
1993;  NCES,  2000;  Sherwood,  1993).  Although  beginning  teachers  report  wanting  to  use  computers,  and  have  gained  adequate 
technical  skills,  they  typically  lack  knowledge  about  how  to  integrate  computers  within  the  more  routine  tasks  of  teaching  and 
managing  their  classrooms  (Hruskocy,  1999;  Novak  & Knowles,  1991). 

Skills  vs.  Ideas 

Clearly,  the  growing  increase  in  teachers’  technical  skills  is  insufficient  to  guarantee  the  effective  use  of  technology  in  the 
classroom  (Carvin,  1999;  Marcinkiewicz,  1994).  In  order  to  translate  skills  into  practice,  teachers  need  specific  ideas  about  how 
to  use  these  skills  to  achieve  meaningful  learning  outcomes  under  normal  classroom  conditions.  Traditionally,  inservice 
technology  training  programs  have  been  software-  rather  than  curriculum-based  (Gilmore,  1995).  Thus,  teachers  completed 
technology  courses  still  not  knowing  how  to  create  or  implement  small-  or  whole-group  activities  that  incorporated  meaningful 
uses  of  technology  (Moersch,  1995).  Unfortunately,  this  also  has  been  true  for  most  teacher  education  technology  courses 
(Moursund  & Bielefeldt,  1999;  Yildirim,  2000).  Although  the  majority  of  teacher  preparation  programs  now  require  that  students 
take  three  or  more  credit  hours  of  technology  instruction,  recent  survey  data  suggested  that  most  teacher  education  faculty  still  do 
not  feel  that  technology  use  is  being  effectively  modeled  for  our  future  teachers  (Schrum,  1999). 

Simply  stated,  few  of  our  current  or  future  teachers  have  either  observed  or  experienced  learning  with  or  from  computers 
(Carlson  & Gooden,  1999).  While  today’s  teachers  are  expected  to  leverage  the  full  potential  of  powerful  conceptual  technology 
tools  to  meet  the  changing  needs  of  their  students,  they  have  been  given  few,  if  any,  opportunities  to  develop  their  own  visions 
for,  or  ideas  about,  meaningful  technology  use. 

The  importance  of  developing  a vision  for  technology  use  cannot  be  overstated  (Ertmer,  1999).  As  noted  by  the  Office  of 
Educational  Research  and  Improvement  (1993):  "Most  teachers  will  find  little  incentive  to  tackle  the  technical  and  scheduling 
problems  associated  with  technology  unless  they  have  a clear  vision  of  how  the  technology  can  improve  teaching  and  learning’’ 
(p.  83).  Once  a clear  vision  is  in  place,  specific  tools  and  strategies  are  needed  to  help  teachers  address  the  many  unique 
challenges  posed  by  the  translation/integration  process:  changing  roles  of  teachers,  students,  and  technology;  classroom 
organization,  management,  and  security  issues;  and  assessment  methods,  among  others.  As  Dexter,  Anderson,  and  Becker  (1999) 
explained,  "For  teachers  to  implement  any  new  instructional  strategy,  they  must  acquire  new  knowledge  about  it  and  then  weave 
this  together  with  the  demands  of  the  curriculum,  classroom  management,  and  existing  instructional  skills"  (p.  223).  Teachers 
need  information  about  how,  as  well  as  why,  to  use  technology  in  meaningful  ways.  Lack  of  knowledge  regarding  either  element 
can  significantly  decrease  the  potential  impact  that  these  powerful  resources  might  have  on  student  learning. 


Self-efficacy  Beliefs 

Yet  even  the  best  ideas  about  technology  use  will  remain  unused  unless  teachers  believe  that  they  are  capable  of 
implementing  them  in  the  classroom.  In  particular,  teachers'  beliefs  about  their  ability  to  use  computers  in  instruction  may  be 
key,  given  the  role  self-efficacy  is  proposed  to  play  in  determining  behavior.  According  to  Eachus  and  Cassidy  (1999),  "self- 
efficacy  has  repeatedly  been  reported  as  a major  factor  in  understanding  the  frequency  and  success  with  which  individuals  use 
computers"  (p.  2). 

Self-efficacy  refers  to  personal  beliefs  about  one's  capability  to  learn  or  perform  actions  at  designated  levels  (Bandura,  1997). 
According  to  Bandura,  self-efficacy  is  based,  not  solely  on  the  level  of  skill  possessed  by  an  individual,  but  on  judgments  about 
what  can  be  done  with  current  skills.  That  is,  self-efficacy  comprises  beliefs  about  what  one  is  capable  of  doing,  not  about 
whether  one  knows  what  to  do.  As  such,  self-efficacy  is  thought  to  mediate  the  relationship  between  skill  and  action.  Therefore, 
without  knowledge  or  skill,  performance  isn't  possible;  yet  without  self-efficacy,  performance  may  not  be  attempted.  According 
to  Bandura,  "beliefs  of  personal  efficacy  constitute  the  key  factor  of  human  agency"  (p.  3).  Thus,  teachers  who  have  high  levels 
of  efficacy  for  teaching  with  technology  are  more  likely  to  participate  more  eagerly,  expend  more  effort,  and  persist  longer  on 
technology -related  tasks  than  teachers  who  have  low  levels  of  efficacy. 

If  self-efficacy  beliefs  are  key  to  performance  and  increased  self-efficacy  can  lead  to  increased  performance  (Christoph, 
Schoenfeld  Jr.,  & Tansky,  1999;  Schunk,  1981),  how  can  we  help  teachers  increase  their  efficacy  for  technology  use? 
Researchers  in  the  area  of  self-efficacy  (Bandura,  1997;  Schunk,  2000)  describe  four  primary  sources  of  information  that  can 
influence  judgments  of  efficacy:  personal  mastery  (successful  task  completion),  vicarious  experiences  (observing  models),  social 
persuasion  ("I  know  you  can  do  this!"),  and  physiological  indicators  (emotional  arousal,  relaxation). 

Next  to  personal  mastery,  vicarious  experience  is  thought  to  provide  the  most  valid  information  for  assessing  efficacy 
(Schunk,  2000).  According  to  Olivier  and  Shapiro  (1993),  "vicarious  experiences  with  the  computer  increase  one's  feelings  of 
control  and  confidence.  These  encounters  also  make  an  individual  want  to  learn  more  about  the  technology,  thus  reducing  and 
eventually  eliminating  the  fear  of  the  unknown  factor.  As  the  fear  and  anxiety  diminish  and  positive  experiences  add  up,  self- 
efficacy  and  the  willingness  to  cope  with  mastering  the  task  will  increase"  (p.  83).  Given  the  logistical  difficulties  involved  in 
providing  preservice  teachers  with  enactive  experiences  related  to  successful  technology  integration,  teacher  educators  have 
turned  to  modeling  as  a feasible,  yet  powerful  method  for  increasing  teachers'  ideas  about  and  self-efficacy  for  technology 
integration  (Schrum,  1999).  Not  only  can  models  provide  information  about  how  to  enact  meaningful  technology  use  but  they 
can  increase  observers'  confidence  for  generating  the  same  behaviors.  Furthermore,  providing  access  to  multiple  models 
increases  both  the  amount  of  information  available  about  how  to  accomplish  the  performance  and  the  probability  that  observers 
will  perceive  themselves  as  similar  to  at  least  one  of  the  models  (Schunk,  2000),  thus  increasing  their  confidence  for  also 
performing  successfully. 

Electronic  Models 

Still,  the  use  of  models  does  not  guarantee  either  learning  or  later  performance.  Many  factors  have  been  shown  to  influence 
observers'  responses  to  models  including  the  prestige  and  competence  of  the  models,  consequences  experienced  by  the  models, 
perceived  similarity  of  the  models  to  the  learners,  as  well  as  learners'  own  self-efficacy  for  performing  the  behaviors  (Schunk, 
2000).  In  addition,  research  has  yet  to  establish  whether  models,  presented  electronically,  can  be  used  to  achieve  results  similar 
to  those  achieved  with  live  models.  Will  learners  perceive  themselves  as  similar  to  models  that  are  presented  electronically? 

Will  they  regard  the  models  as  both  realistic  and  relevant?  Given  the  increasing  potential  to  present  models  of  exemplary 
technology  use  via  multimedia  technologies,  it  is  important  to  determine  the  extent  to  which  pre-  and  inservice  teachers  can 
benefit  from  observing  these  types  of  electronic  models. 

Purpose 

This  study  was  designed  to  examine  the  effects  of  electronic  models  on  preservice  teachers'  perceived  ideas  about,  and  self- 
efficacy  for,  technology  integration.  Specifically,  exemplary  technology -using  teachers  were  presented  via  a CD-ROM  teacher 
development  tool,  called  VisionQuest.  VisionQuest  features  the  classroom  practices  of  six  k-12  teachers  and  is  designed  to 
support  users'  reflections  on  both  the  underlying  beliefs  and  classroom  strategies  that  enable  exemplary  technology  use.  Given 
the  few  opportunities  preservice  teachers  have  to  observe  exemplary  technology  use  in  actual  classrooms  during  student  teaching 
or  observation  sessions  (Carlson  & Gooden,  1999;  Vannatta  & Reinhart,  1999),  VisionQuest  was  developed  to  provide  these 
opportunities.  Specifically,  the  research  questions  guiding  data  collection  and  analysis  included: 

• What  effect  does  observing  exemplary  technology -using  teachers,  presented  electronically,  have  on  preservice 
teachers'  perceptions  of  ideas  about  technology  integration? 

• What  effect  does  observing  electronic  exemplary  technology -using  teachers,  presented  electronically,  have  on 
preservice  teachers'  perceptions  of  self-efficacy  for  technology  integration? 

• What  are  students'  perceptions  of  the  use  of  electronic  models  for  learning  about  technology  integration? 

Methods 

A pretest -posttest  research  design  was  used  to  examine  increases  in  preservice  teachers'  ideas  about,  and  self-efficacy  for, 
technology  integration  following  two  50-minute  class  sessions  in  which  students  used  VisionQuest,  a CD-ROM  teacher 
development  tool  designed  to  present  exemplary  models  of  classroom  technology  use.  Participants'  perceptions  of  their  learning 
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experiences  were  collected  via  classroom  observations  and  interviews  with  10  purposefully  selected  students.  Both  quantitative 
(paired  t-tests)  and  qualitative  (pattern  seeking)  analysis  methods  were  used  to  examine  the  extent  to  which  electronic  models 
offered  a viable  method  for  increasing  preservice  teachers’  capacity  for  technology  integration. 

Role  of  Researchers 

The  research  team  consisted  of  a faculty  member  and  five  students  enrolled  in  an  advanced  educational  technology  research 
course  at  a large  Midwestern  university.  Students  had  varied  background  experiences,  in  both  k-12  and  post-secondary 
classrooms,  and  were  seeking  masters  (7  = 1)  or  doctoral  degrees  (7  = 4)  in  educational  technology.  The  team  worked 
collaboratively  to  design  the  study  and  develop  appropriate  data  collection  instruments.  Each  researcher  collected  survey, 
interview,  and  observation  data  from  students  in  one  of  the  six  course  sections  participating  in  the  study.  Survey  data  were 
combined  and  analyzed  by  the  team;  interview  and  observation  data  were  used  primarily  to  triangulate  quantitative  results. 
Description  of  Site  and  Participants 

Of  the  103  students  enrolled  in  six  sections  of  an  undergraduate  educational  technology  course,  69  students  signed  a consent 
form  and  completed  all  three  data  collection  measures  needed  for  the  study.  Classroom  Applications  of  Educational  Technology 
is  a one-credit  optional  course  offered  as  a "companion"  to  the  required  2-credit  introductory  course.  Participants  ranged  in  age 
from  18-34  years,  with  a mean  of  20  years.  The  majority  of  the  students  were  female  (65%),  sophomores  or  juniors  (71%),  and 
majoring  in  Elementary  Education  (60%).  Eighty-seven  percent  of  the  participants  had  computers  at  home  and,  at  the  time  of  the 
study  (week  10  in  the  semester),  indicated  that  they  used  computers  primarily  for  word  processing  (98%),  electronic  mail  (99%), 
and  browsing  the  Internet  (98%).  When  asked  to  rate  current  levels  of  computer  skills,  from  novice  to  advanced,  75%  of  the 
students  rated  their  skills  at  an  intermediate  level  while  9%  rated  themselves  as  beginners;  16%  rated  themselves  as  advanced. 
None  of  the  students  rated  themselves  as  novice  users. 

Description  of  VisionQuest 

As  an  instructional  tool,  VisionQuest  (VQ)  is  designed  to  provide  opportunities  for  users  to  explore  models  of  effective 
technology  integration.  Users  examine  the  steps  that  three  sets  of  teachers  have  taken  to  achieve  their  current  levels  of 
technology  use.  Cases  include  a high  school  team  of  three  biology  teachers,  a middle  school  music  teacher,  and  an  elementary 
team  of  two  second-grade  teachers  at  a science  and  technology  magnet  school.  Users  examine  how  teachers’  pedagogical  visions 
of  classroom  practice  have  shaped  their  integration  journeys  including  how  they  got  started,  the  roadblocks  and  challenges  they 
faced,  as  well  as  the  incentives  that  moved  them  forward  (highlighted  by  the  Roadmap  components  of  Figure  1).  The  cases 
illustrate  that  technology  integration  can  be  successfully  achieved  in  a variety  of  contexts  despite  differences  in  settings, 
resources,  and  student  backgrounds.  Users  examine,  both  within-  and  across-cases,  the  relationships  among  teachers’  beliefs 
(related  to  classroom  organization;  the  role  of  the  teacher,  students,  and  technology;  curricular  emphases;  and  assessment 
practices)  and  current  classroom  practices  related  to  technology  (illustrated  by  the  Path  and  Destination  components  of  Figure  1). 
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Figure  1.  Screen  shot  from  main  menu  depicting  7 components  of  technology  integration. 
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VisionQuest  initiates  user  reflection  through  the  use  of  video  segments,  augmented  by  electronic  artifacts  (lesson  plans,  student 
products)  from  teachers'  classrooms.  Cases  are  constructed  such  that  users  can  explore  teachers'  classrooms  either  one  at  a time 
(case  by  case)  or  thematically  (i.e.,  comparing  components  of  technology  integration  across  cases).  Each  case  contains  a variety 
of  elements  that  combine  to  illustrate  how  teachers'  visions  for  technology  use  are  translated  into  practice  (see  Figure  2).  Users 
examine  how  teachers  planned  for  integration,  how  they  currently  implement  technology  within  their  classrooms,  and  how  they 
assess  the  impact  of  their  efforts. 


Integration 
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Figure  2.  Screen  shot  from  a content  page  of  VisionQuest. 


At  the  time  of  the  study,  VisionQuest  was  in  beta  format.  Although  navigational  features  were  still  somewhat  rudimentary, 
"workarounds"  were  built  in  so  that  students  could  effectively  use  the  software  to  complete  the  tasks  assigned. 

Procedures 

Demographic  information  was  collected  from  the  participants  during  the  first  class  session  of  the  semester.  During  weeks  10 
and  1 1,  as  part  of  their  normal  class  activities,  all  students  worked  with  VisionQuest,  completing  two  different  tasks.  During  the 
tenth  week,  students  evaluated  VisionQuest  as  an  example  of  professional  development  software.  During  this  week,  questions 
were  specifically  directed  toward  software  quality  and  not  software  content.  Although  the  majority  of  these  data  are  not  relevant 
to  this  study,  two  questions  provided  information  about  students'  perceptions  of  the  VQ  models  and  will  be  discussed  later. 
Students  focused  on  content  the  following  week  when  they  used  VisionQuest  as  a modeling  tool  to  examine  the  beliefs  and 
classroom  practices  of  the  teachers  included  on  the  CD-ROM.  Students  were  asked  to  describe  how  the  different  teachers 
prepared  their  classrooms  for  technology  use,  how  they  used  various  grouping  strategies  to  manage  their  rooms,  how  they 
managed  classroom  "chaos,"  and  so  on.  At  the  end  of  this  session,  students  were  asked  to  list  the  components  of  classroom 
organization  that  the  three  groups  of  teachers  had  considered  prior  to  implementing  technology  in  their  classrooms. 

Data  Collection 

At  the  beginning  of  the  tenth  class  session,  prior  to  evaluating  VQ,  students  completed  an  online  survey  designed  to  collect 
three  types  of  information.  First,  information  was,  collected  regarding  students'  computer  ownership,  current  use,  and  perceptions 
of  skills  and  comfort  using  computers  (e.g.,  "I  enjoy  working  with  computers."  "When  using  computers,  I can  deal  with  most 
difficulties  I encounter.").  Eight  items  comprised  this  initial  section.  The  second  section  included  seven  items  regarding 
students'  ideas  for  technology  use  (see  Figure  3).  Items  were  presented  in  a Likert-style  format;  students  were  asked  to  rate  their 
level  of  agreement  (from  1-strongly  disagree  to  5-strongly  agree)  with  statements  related  to  the  possession  of  specific  ideas 
regarding  technology  use.  The  third  section  used  the  same  seven  items  as  the  second  section  but  with  an  emphasis  on  the 
possession  of  confidence  rather  than  ideas  (e.g.,  "I  am  confident  I can  use  one  computer  effectively  during  large  group 
instruction."  "I  am  confident  I can  use  technology  effectively  to  teach  content.").  Students  used  the  same  rating  scale  (from  1- 
strongly  disagree  to  5-strongly  agree)  to  record  their  levels  of  confidence.  Students'  responses  to  the  online  surveys,  prior  to 
using  VQ,  comprised  pretest  measures  of  students'  perceived  ideas  about,  and  self-efficacy  for,  technology  integration. 
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Please  Indicate  how  strongly  you  disagree/agree  with  each  of  the  following  staternerits. 

I haw  specific  ideas  about  how  to  iise  techiiblo^  as  an  effective  teachifig  tool. 

0 1.  Strongly  Disagree 
0 2.  Disagree 
O '3;  Undecided 
04.  Agree 
03.  Strongly  Agree 


I have  specific  ideas  about  how  to  use  one  computer  effectively  duiing  laige  group  instruction. 


01.  Strongly  Disagiee 

02.  Disagree 
o 3.  Undecided 

.4.  Agre®, 

5.  Ston^  Agree 


■§: 


I have  specific  ideas  about  how  to  develop  effectiw  lessons  that  incotpoiate  technology. 

0 1.  Stipi^y  Disagree 
Q 2.  Dis'a^e 

03.  Undecided 

04.  Agree 

03.  Strongly  Agree 

I have  specific  ideas  about  how  to  use  technology  effectively  to  teach  content  across  the  cunicuium. 

.01.  Strongly  Disagree 

02.  Disagree 
0.3.  Undecided 

04.  Agree  * 

0 3.  Strongly  Agree 

I have  specific  ideas  about  how  to  oveicome  difficulties  using  technology  in  the  classroom  (time,  scheduling  etc:) 

Q 1.  Strongly  Disagree 

02.  Disagree 

03.  Undecided 

04.  Agree 

03.  Strongly  Agree 


I have  specific  ideas  about  how  to  manage  the  grouping  of  students  vrtiile  using  technology  as  a teaching  tool. 


0 1.  strongly  Disagree 
02.  Disagree 
3.  Undecided 
.4.  Agree 

3.  Strongly  Agree 


i 


I have  specific  ideas  about  how  teachers  use  techmlpgy  in  their  classroonw  . 

0 1.  Strongly  Disagree 
02.  Disagree 
0 3.  Undecided 

04.  Agree 

03.  Strongly  Agree 


Figure  i.  Screen  shot  from  online  survey:  Ideas  about  Technology  Integration  Survey 

At  the  end  of  the  eleventh  class  session,  after  students  had  explored  the  ideas  presented  by  the  models  on  VQ  regarding 
classroom  management  strategies,  they  completed  the  second  and  third  parts  of  the  online  survey  again.  These  measurements 
served  as  posttest  indices  of  students'  perceived  ideas  about,  and  self-efficacy  for  technology  integration.  In  addition,  four  items 
were  included  to  explore  students'  perceptions  of  using  VQ  as  a modeling  tool  (e.g.,  "I  can  relate  to  the  examples  of  teachers 
shown  in  VQ."  "I  can  relate  to  the  examples  of  technology  shown  in  VQ."). 

During  both  class  sessions  in  which  students  interacted  with  VQ,  one  or  two  researchers  were  in  attendance,  making 
observations  of  students'  engagement  with  the  software.  Observations  provided  evidence  of  the  "holding"  quality  of  the  software 
and  also  provided  useful  information  for  the  selection  of  interviewees.  Students  were  purposefully  selected  for  interviews  (one  or 
two  per  section)  based  on  noted  levels  of  interest,  with  an  attempt  to  choose  one  highly-  and  one  less-engaged  student  from  each 
section.  Interviews  were  scheduled  at  a time  convenient  to  each  participant  and  were  audiotaped  and  transcribed  by  the 
interviewer.  Interviews  focused  on  identifying  specific  ideas  (about  classroom  organization,  assessment  practices,  etc.)  that 
students  gained  from  VisionQuest  and  the  extent  to  which  they  thought  they  would  use  these  ideas  in  their  classrooms.  We  were 
particularly  interested  in  knowing  whether  students  regarded  the  VQ  models  as  "real"  and  whether  they  believed  that  they  had 
learned  from  them,  just  as  they  might  learn  from  live  models. 
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Data  Analysis 

Demographic  data  were  tallied  and  percentages  calculated  to  identify  general  characteristics  of  participants.  Changes  in 
students'  perceptions  of  their  ideas  about,  and  self-efficacy  for,  technology  integration  were  determined  using  paired  t-tests, 
Pearson  correlations  were  calculated  to  determine  relationships  among  ideas  and  confidence  (pre  and  post)  and  specific 
demographic  characteristics. 

Interview  transcripts  were  analyzed  using  a simple  pattern-seeking  method  to  determine  students'  impressions  of  the  software 
and  the  impact  it  may  have  had  on  their  ideas  and  confidence.  Analysis  efforts  focused  on  comments  that  either  supported  or 
negated  quantitative  findings  in  order  to  validate,  extend,  or  modify  initial  results.  In  addition  we  examined  students'  perceptions 
of  using  electronic  models  to  determine  whether  this  type  of  modeling  tool  might  present  a reasonable  alternative  to  observing 
live  models. 

Issues  of  Validity  and  Reliability 

Reliability  was  increased  through  the  use  of  consistent  data  collection  methods.  Students  in  all  six  sections  of  the  course 
completed  the  same  online  surveys— data  were  electronically  transferred  to  an  Excel  spreadsheet,  thereby  eliminating  possible 
error  in  entering  or  organizing  the  data.  Each  researcher  followed  the  same  procedures  while  introducing  the  study,  conducting 
observations,  and  interviewing  participants.  In  addition,  weekly  online  and  in-class  discussions  among  the  researchers  increased 
the  consistency  of  research  methods  used. 

Chronbach's  alpha  was  used  to  measure  the  internal  consistency  of  the  survey  instruments.  Calculated  Chronbach's  alphas 
were  .80  on  the  Ideas  survey  and  .89  on  the  Self-Efficacy  survey,  suggesting  that  the  instruments  were  moderately  reliable. 
Despite  the  fact  that  the  same  measures  were  used  for  both  pre-  and  post -assessments,  the  possibility  of  experiencing  a testing 
effect  is  minimal.  According  to  Bandura  (2001),  previous  tests  for  reactive  effects  have  demonstrated  that  self-efficacy  does  not 
increase  as  a mere  function  of  assessing  one's  efficacy:  "If  merely  recording  a level  of  efficacy  made  it  so,  personal  change  would 
be  trivially  easy"  (p.  6). 

Survey  measures  were  evaluated  by  an  expert  in  the  area  of  self-efficacy  and  modified  based  on  his  suggestions,  providing 
the  instruments  with  a certain  amount  of  face  validity.  To  further  increase  validity,  multiple  data  sources  were  used  to  triangulate 
findings.  For  example,  observations  provided  a rough  measure  of  students'  levels  of  engagement,  interview  comments  verified 
their  excitement  about  the  software,  and  survey  measures  indicated  that  students  found  the  VQ  examples  relevant.  Together, 
these  data  provide  strong  evidence  of  students'  engagement  in,  and  identification  with,  the  models  provided  on  VQ  and  as  such, 
helped  us  answer  our  third  research  question. 

Results 

Changing  Ideas  and  Efficacy  for  Technology  Integration 

A two-tailed  paired  t-test  {df  = 68)  indicated  a significant  increase  in  students’  ratings  of  perceived  ideas  about  technology 
integration  (/  = 8.85;  p < .0000)  from  pre-  to  post  survey.  Students' judgments  of  their  ideas  for  technology  integration  increased 
from  a pretest  mean  of  3.72  {SD  = .44)  to  a posttest  mean  of  4. 12  {SD  = ,40), 

A two-tailed  paired  t-test  iff  = 68)  also  indicated  a significant  increase  in  students’  ratings  of  perceived  self-efficacy  for 
technology  integration  (/  = 3.46;  p < .000)  from  pre  to  post  survey.  Students'  Judgments  of  confidence  increased  from  a pretest 
mean  of  3.84  {SD  = .52)  to  a posttest  mean  of  4.05  {SD  = ,47), 

Since  it  is  fairly  easy  to  achieve  high  correlation  coefficients  with  larger  samples,  significance  levels  were  set  relatively  high 
in  order  to  discount  high  coefficients  that  were  not  meaningful.  That  is,  we  did  not  consider  coefficients  to  be  significant  unless 
the  probability  of  occurrence  was  less  than  p = ,0005.  Thus,  based  on  a critical  r value  {df=  66)  of  .35,  correlations  among 
demographic  characteristics  and  pre-  and  post-  ideas  and  self-efficacy  indicated  no  significant  relationships  among  age,  gender, 
or  year  in  school  (freshman,  sophomore,  etc.)  and  ratings  of  computer  skills,  ideas,  or  self-efficacy  (see  Table  1),  Although  one 
might  expect  advanced  college  students  (e.g..  Juniors  and  seniors)  to  have  more  skills,  ideas,  or  confidence,  this  was  not  the  case 
in  this  study.  Furthermore,  there  were  no  significant  relationships  between  gender  and  any  variables  examined  in  this  study. 

Table  L Correlation  Coefficients  among  Selected  Demographic  Variables  and  Pre/Post  Measures  of  Ideas  and  Self-Efficacy 


Age 

Class 

Gender 

Computer 

Skills 

Pre 

Ideas 

Post  Ideas 

Pre  SE 

Post  SE 

Age 

Class 

.73 

Gender 

.18 

.14 

Computer 

-.15 

-.13 

-.02 

Skills 

Pre  Ideas 

-.09 

-.09 

-.12 

.34 

Post  Ideas 

-.19 

-.14 

-.09 

.28 

.61^ 

PreSE 

.05 

.02 

-.10 

.18 

J2* 

.52* 

Post  SE 

-.14 

-.09 

-.12 

.26 

.48^ 

.84* 

.50* 

1.00 

♦ p < .0005 

Note:  SE  = self-efficacy 


Significant  correlations  were  found  between  students'  perceptions  of  their  ideas  for  technology  integration,  before  and  after 
using  VisionQuest  (r  = .61);  similarly  students'  perceptions  of  self-efficacy  for  technology  integration  (r  =.50)  were  significantly 
correlated  before  and  after  using  VisionQuest.  Additionally,  perceptions  of  ideas  and  perceptions  of  confidence  were 
significantly  correlated.  Students  who  began  with  greater  perceptions  of  ideas,  also  tended  to  have  higher  levels  of  confidence  (r 
= .72).  This  relationship  was  even  stronger  at  the  time  of  the  posttest  (r  = .84).  The  coefficient  of  determination  (r-squared  = 
.71)  suggests  that  71%  of  the  variance  in  students'  confidence  ratings  could  be  "explained"  by  students'  perceptions  of  their  ideas 
for  technology  integration.  In  other  words,  the  more  ideas  students  have  about  technology  integration,  the  stronger  their  belief 
that  they  can  be  successful  integrating  technology  into  the  classroom.  As  ideas  increase,  so,  too,  does  confidence  for 
implementing  the  ideas. 

Interestingly,  judgments  of  computer  competency  (skills)  were  not  highly  correlated  with  either  ideas  or  confidence  for 
technology  integration.  This  supports  earlier  research  findings  (Moursund  «fe  Bielefeldt,  1999;  Yildirim,  2000)  that  suggest  that 
simple  skill  training  is  insufficient  to  prepare  students  to  use  technology  in  the  classroom.  Although  judgments  of  computer 
skills  were  moderately  correlated  with  students'  ideas  for  using  technology  prior  to  VQ  (r  = .34;  p < ,001),  this  relationship  was 
not  significant  after  using  VQ  (r  = .28;  p > .001).  Furthermore,  skill  competency  did  not  seem  to  translate  into  confidence  for 
achieving  integration  either  pre  or  post  VQ  (r  = .18  and  .26  respectively).  Just  because  students  know  how  to  use  word- 
processing, email,  and  the  Internet,  does  not  mean  that  they  know  how  to  use  these  skills  within  classroom  instruction  or  that  they 
are  confident  trying  to  do  so. 

Perceptions  of  Using  Electronic  Models 

Interviews  with  10  students,  as  well  as  data  obtained  through  four  post-survey  items  and  two  software  evaluation  questions, 
were  used  to  answer  our  research  question  regarding  students'  perceptions  of  using  electronic  models  to  learn  about  technology 
integration.  Interviewees  were  representative  of  the  students  enrolled  in  the  class;  interviewees  included  both  male  and  female 
students  who  ranged  in  age  from  18-34  years,  in  skill  levels  from  beginner  to  advanced,  and  in  confidence  levels  from 
“somewhat”  to  “very”  confident. 

Two  questions  on  the  software  evaluation  form  asked  students  to  rate  the  relevance  of  the  activities  and  models  observed  on 
VQ.  On  a scale  from  1 (strongly  disagree)  to  5 (strongly  agree),  students  agreed  to  strongly  agreed  that  "activities  regarding  the 
use  of  technology  were  realistic"  (mean  = 4.46)  and  that  "the  video  cases  of  teacher  interviews  and  class  activities  were  relevant" 
(mean  = 4.31).  Four  similar  questions  included  on  the  post  survey  averaged  a 3.96  rating  indicating  that  students'  perceived  the 
VQ  models  to  be  both  realistic  and  relevant. 

Although  students  had  suggestions  for  improving  the  software  (particularly  in  terms  of  navigation,  which  was  unfinished  at 
the  time),  interview  comments  were  overwhelmingly  positive.  Students  viewed  the  models  as  realistic,  indicating  that  they  felt  as 
though  they  were  in  the  classrooms  with  the  teachers.  Students  described  the  "life-like"  quality  of  the  videos  and  how  they  felt 
that  the  teachers  were  talking  directly  to  them  (S:  I felt  like  they  were  talking  to  me  as  a teacher  and  not  as  a student).  As  an 
example,  one  student  stated: 

I liked  it.  I liked  how  I got  involved  when  it  showed  you  (the  clips)  and  you  felt  like  you  were  right  there  in  the 
classroom  with  the  students  watching  them.  It’s  like  you’re  in  a movie  theater  almost  because  they  have  such  good 
(videos)...  and  it  shows  the  students  and  it  shows  the  teachers  - and  you  feel  like  you’re  right  there  in  it. 

Because  our  survey  instruments  did  not  provide  information  about  the  specific  ideas  that  students  may  have  gained  using  VQ, 
we  asked  students  to  describe  these  ideas  in  our  interviews.  Students  indicated  that,  by  observing  the  classroom  examples  on 
VisionQuest,  they  had  gained  ideas  about  "using  stations,"  "assessment,"  "group  work,"  "using  different  activities  to  teach  the 
same  content,"  "integrating  computer  research  into  a music  classroom,"  "using  technology  to  work  with  different  levels  of 
students -special  ed  and  those  who  excel,"  "using  HyperStudio  in  a music  class,"  and  "establishing  a good  climate  in  the  class." 
Students  made  many  comments  about  how  VQ  allowed  them  to  see  how  things  were  done  in  a classroom.  Three  representative 
comments  follow: 

I think  actually  seeing  the  teachers  in  the  video  clips  and  how  the  students  are  actually  using  it  and  how  the  teachers  are 
using  it  and  incorporating  it  in  their  lessons-I  think  was  really  good  for  me.  I had  ideas  going  into  VisionQuest  of  how 
I could  use  technology  in  my  classroom  but  actually  seeing  teachers  using  it  gave  me  some  new  ideas  of  'Oh,  I didn’t 
think  of  using  it  like  that. 

Seeing  the  teachers  use  technology  helped  me  to  understand  how  it's  done... it's  one  thing  to  hear  someone  talk  about 
different  methods,  but  seeing  the  classes  actually  use  the  technology  —that  really  made  me  think  of  how  I could  do  it 
next  year. 

I liked  the  examples  and  the  students'  points  of  view.  They  had  a lot  of  good  ideas  about  what  they  were  doing.  It’s  a 
good  way  to  teach  us  about  what  you  can  do  with  computers.  And  we  used  a computer.  This  was  a good  way  for  us  to 
see  what  goes  on  in  a classroom.  I could  see  doing  things  like  they  did. 

Students  agreed  that  it  was  beneficial  to  hear  the  teachers  in  addition  to  seeing  them.  Exploring  teachers'  beliefs  helped 
students  understand  why  teachers  made  the  decisions  that  they  did,  and  provided  cognitive  modeling  of  the  integration  task.  For 
example,  one  student  noted: 
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I think  it’s  really  neat  how  you  have  the  different  clips  in  there,  the  different  classrooms  and  you  have  the  students’ 
opinions  and  the  teachers’  opinions.  It's  got  their  different  beliefs  and  teachers  can  take  that  and  maybe  it  will  change 
their  philosophy  and  they  can  interpret  new  things  into  their  classroom.  I think  it’s  a very  good  program  and  it’s  got  a 
lot  of  potential. 

I liked  it.  I thought  it  was  pretty  cool  the  way  you  could  see  what  they  were  doing  in  all  those  schools.  The  interviews 
were  really  good  because  you  get  a chance  to  see  what  they  think  about  their  own  classrooms  and  they  talk  about  what 
they  want  to  do.  You  could  click  on  the  materials  or  the  interviews. 

Based  on  these  results  it  appears  as  though  students  both  enjoyed  and  benefited  from  observing  the  electronic  models 
provided  on  VisionQuest.  Interview  comments  suggest  that  preservice  teachers  perceived  that  the  use  of  electronic  models  was  a 
positive  approach  that  provided  "life-like"  learning  experiences.  There  were  no  comments  to  suggest  that  the  students  found  it 
difficult  to  identify  with  the  models  presented  via  CD-ROM  technology. 

Discussion 

This  study  examined  preservice  teachers’  perceived  ideas  about  and  self-efficacy  for  technology  integration  before  and  after 
observing  electronic  models  of  exemplary  technology -using  teachers.  Sixty-nine  students,  enrolled  in  a one-credit  technology 
course,  completed  online  demographic  and  survey  instruments  and  then  used  a CD-ROM  electronic  modeling  tool  during  the 
tenth  and  eleventh  weeks  of  the  semester.  To  measure  changes  in  students’  perceived  ideas  and  self-efficacy,  the  online  surveys 
were  completed  again  at  the  end  of  the  second  session. 

The  results  of  this  study  support  our  hypotheses  that  electronic  models  can  be  used  to  increase  preservice  teachers’  ideas 
about  and  self-efficacy  for  technology  integration.  Even  though  students  used  VisionQuest  for  a relatively  short  period  of  time 
over  the  course  of  two  class  sessions  (approximately  90  minutes  total)  and  were  unable  to  explore  the  entire  content  of  the  CD- 
ROM,  students  showed  significant  increases  in  both  their  perceived  ideas  about  and  self-efficacy  for  technology  integration. 
Interview  and  software  evaluation  comments  indicated  that  students  found  the  models  to  be  both  realistic  and  relevant.  Students 
described  a number  of  specific  ideas  that  they  gained  from  the  models  and  furthermore,  described  their  intent  to  apply  these  ideas 
within  their  future  classrooms. 

The  69  students  who  participated  in  this  study  were  not  novice  computer  users;  in  fact,  the  majority  of  our  participants  (n  = 
63)  rated  themselves  as  either  intermediate  or  advanced  computer  users.  In  addition,  initial  ratings  of  perceived  ideas  about  and 
self-efficacy  for  technology  integration  were  not  exceptionally  low  (x  ~ 3.72  and  3.84,  respectively).  Still,  ratings  of  perceived 
ideas  and  self-efficacy  increased  significantly  from  pre-  to  post-  VisionQuest  suggesting  that  students  were  able  to  gain 
additional  ideas  and  confidence  by  observing  the  models  on  the  CD-ROM. 

Students'  pre-  and  post-  ratings  of  their  ideas  and  confidence  were  not  significantly  correlated  with  their  judgments  of  skill 
levels,  suggesting  that  computer  skill  competency  does  not  translate  directly  into  ideas  or  confidence  for  classroom  technology 
use.  In  fact,  students'  perceptions  of  the  direct  usefulness  of  their  skills  may  have  decreased  after  seeing  how  the  teachers  on 
VisionQuest  were  not  dependent  on  high  skill  levels,  although  this  conjecture  requires  further  examination.  However,  there  were 
significant  correlations  between  students’  perceived  ideas  and  confidence,  especially  at  the  time  of  the  posttest  (•  = .84) 
suggesting  that  as  students  see  new  ways  to  use  technology  and  develop  new  ideas  about  technology  integration,  they  develop 
higher  levels  of  confidence  about  their  ability  to  use  technology  in  a variety  of  ways. 

Based  on  the  correlations  obtained,  providing  preservice  teachers  with  specific  integration  ideas  (e.g.,  how  to  organize  a 
classroom  that  uses  technology,  how  to  assess  student  technology  products)  via  electronic  observations  of  technology -using 
teachers  may  be  more  effective  than  skills  training  for  increasing  their  self-efficacy  for  technology  integration.  Furthermore,  by 
increasing  future  teachers’  self-efficacy,  we  increase  the  probability  that  these  behaviors  will  be  implemented  in  their  future 
classrooms.  According  to  Olivier  and  Shapiro  (1993),  "Self-efficacy  has  been  shown  to  be  an  excellent  predictor  of  behavior. 
Individuals  with  a low  sense  of  self-efficacy  will,  more  often  than  not,  shy  away  from  the  best  alternative,  and,  instead,  choose  an 
alternative  that  they  believe  they  can  handle"  (p.  84).  Even  when  practicum  and  student  teachers  possess  "positive  dispositions 
towards  computer  use,"  they  often  lack  confidence  in  their  ability  to  teach  successfully  with  computers  (Albion,  1999).  This  lack 
of  confidence  for  teaching  with  computers  has  been  shown  to  influence  the  levels  of  computer  use  by  student  and  beginning 
teachers  (Albion,  1996;  Handler,  1993). 

The  lack  of  significant  correlations  among  age,  class,  gender,  and  skills  and  pre-  and  post-measurements  of  ideas  and  self- 
efficacy  (see  Table  1)  suggests  that  all  of  the  students  in  this  study  were  able  to  gain  ideas  and  confidence  through  their 
interactions  with  the  electronic  models.  That  is,  no  one  group  of  student  was  more  or  less  likely  to  have  more  ideas  or  confidence 
for  technology  integration.  Previous  research  has  suggested  that  using  a variety  of  models  increases  the  possibility  that  students 
will  find  at  least  one  model  they  can  identify  with  (Schunk,  2000)  and  also  provides  additional  information  about  a number  of 
effective  strategies  that  can  be  used  to  achieve  integration. 


Educational  Implications 

The  results  of  this  study  suggest  that  preservice  teachers  can  benefit  from  observing  teacher  models  presented  via  multimedia 
case  examples,  such  as  those  featured  on  VisionQuest.  Whether  delivered  via  the  Web  or  CD-ROM,  multimedia  models  are 
becoming  more  readily  available  for  use  by  teacher  educators.  These  types  of  examples  can  be  incorporated  into  an  educational 
environment  for  self-paced  exploration,  as  a small  group  reflection  tool,  or  as  an  instructor-led  activity.  From  an  instructor’s 
perspective,  electronic  models  can  positively  impact  the  authentic  nature  of  a course  and  simultaneously  increase  the  confidence 
and  integration  beliefs  of  students.  This  type  of  modeling  can  help  preservice  teachers  develop  a vision  for  what  technology 
integration  looks  like  in  real  classroom  as  well  as  strategies  for  implementing  those  visions. 

Limitations  and  Suggestions  for  Future  Study 

The  primary  limitation  of  this  study  relates  to  our  inability  to  isolate  specific  cause  and  effect  variables.  Because  VQ  was 
part  of  the  course  curriculum  for  the  students  who  participated,  and  was  scheduled  to  be  used  at  a specific  time  in  the  semester, 
we  were  unable  to  create  a control  group  for  this  study.  However,  future  efforts  will  include  use  of  a cross-over  design,  that  is, 
one  that  would  introduce  VQ  to  a control  group  at  a later  time  in  the  semester.  This  would  allow  for  a more  systematic  look  at 
the  effect  of  VQ  on  students'  perceived  ideas  and  efficacy. 

Participants  in  this  study  were  fairly  homogeneous;  generalization  to  g'oups  differing  in  age,  gender,  ethnicity,  or  levels  of 
computer  competency  may  not  be  appropriate.  As  an  example,  it  is  unclear  whether  the  use  of  exemplary  models  will  be  equally 
effective  with  novice  users  who  are  likely  to  begin  their  explorations  of  the  CD-ROM  with  much  lower  levels  of  ideas  and  self- 
efficacy.  There  is  some  evidence  to  suggest  (Snoeyink,  2000)  that  teachers  need  at  least  a very  basic  skill  and  confidence  level 
before  they  can  benefit  from  observing  exemplary  others.  Because  our  study  did  not  include  participants  who  rated  themselves  as 
novice  users,  we  were  unable  to  answer  this  question. 

An  additional  limitation  of  this  study  relates  to  our  inability  to  determine  if  students'  perceptions  of  having  many  ideas  for 
technology  integration  actually  translate  into  classroom  application,  as  hoped.  Although  student  worksheets  and  interview 
comments  suggest  that  students  gained  new  ideas,  additional  work  is  needed  to  determine  the  extent  to  which  these  students  are 
able  to  carry  out  these  ideas  when  they  actually  assume  leadership  of  their  own  classrooms.  Still,  according  to  social  learning 
theory  (Bandura,  1997),  building  self-efficacy  is  an  important  first  step  toward  developing  the  capacity  to  perform  a particular 
skill.  Without  a sufficient  level  of  self-efficacy  for  performing  computer  tasks,  technology  integration  may  not  even  be  attempted 
(Olivier  & Shapiro,  1993).  Models  can  serve  informational  and  motivational  functions  for  observers  (Schunk,  2000),  yet  further 
research  is  needed  to  verify  the  long-range  benefits  of  increasing  self-efficacy  through  the  use  of  electronic  models. 

Conclusion 

Teachers  today  face  a number  of  challenges  as  they  begin  integrating  technology  into  their  classrooms,  not  the  least  of  which 
include  a lack  of  specific  ideas  about  how  to  organize  and  manage  integrated  classrooms,  uncertainty  about  how  to  implement 
new  roles  within  current  classroom  routines,  as  well  as  a lack  of  confidence  for  implementing  these  new  types  of  ideas  and  roles. 
Even  as  our  teachers  are  gaining  more  computer  skills,  they  continue  to  report  feeling  unprepared  to  use  technology  in  the 
classroom  (NCES,  2000).  As  educators  begin  to  realize  that  skill  training  is  not  enough  to  prepare  teachers  to  integrate 
technology  within  the  curriculum,  their  attention  must  turn  to  helping  both  pre-  and  inservice  teachers  gain  specific  ideas  and 
confidence  for  technology  integration.  How  then,  can  this  be  accomplished  in  the  most  effective  and  efficient  way? 

Although  self-efficacy  theory  suggests  that  personal  successful  experience  with  technology  in  the  classroom  is  the  most 
powerful  means  for  building  teachers'  self-efficacy  (Bandura,  1997),  this  is  almost  impossible  to  achieve  in  practice.  Simply 
trying  to  arrange  field  observations  of  exemplary  technology -using  teachers  is  fraught  with  difficulty.  Even  if  we  were  able  to 
find  sufficient  numbers  of  exemplary  technology -using  teachers  who  were  willing  to  allow  visitors  in  their  classrooms,  handling 
the  logistics  related  to  scheduling  classes,  transporting  students,  and  arranging  appropriate  times  to  visit  would  be  a nightmare. 
The  use  of  multimedia  materials  that  incorporate  examples  of  effective  classroom  use  of  technology  helps  eliminate  these 
logistical  concerns.  Data  from  this  study  suggest  that  providing  preservice  teachers  with  opportunities  to  interact  with  exemplary 
technology  users,  through  electronic  models,  is  a viable  means  for  increasing  capacity  (ideas  and  self-efficacy)  for  technology 
integration. 
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Introduction 

This  Study  examines  the  articles  from  current  journal  literature  in  the  field  of  instructional/educational  technology  to  ascertain 
the  number  of  articles  written  by  women  each  year  and  the  content  that  women  are  addressing  in  their  articles.  A previous  study 
of  the  journal  literature  (Foley,  Gurney,  & Branch,  1994)  examined  women’s  contributions  to  the  literature  from  1988-1992. 
Both  the  current  and  previous  analyses  used  methods  similar  to  Ely’s  work  in  analyzing  the  literature  of  the  field  (Ely,  1996). 
This  study  uses  an  analysis  of  the  topics  of  articles  to  develop  themes  found  in  the  literature  and  compares  these  to  the  trends  and 
issues  identified  by  Ely  (1996)  and  Molenda,  Russell,  & Smaldino  (1998).  Finally  this  study  takes  a critical  look  at  these  articles 
from  a feminist  perspective  in  regard  to  gender  issues. 

Rationale 

Instructional  Technology  Research  Agenda 

A rationale  for  investigation  of  women’s  contributions  to  the  literature  in  educational/instructional  technology  comes  from 
several  sources.  Molenda  (1996)  describes  an  agenda  for  research  in  the  field  in  order  to  “determine  where  the  knowledge  gaps 
are...”  (p.  38)  and  calls  for  an  examination  of  the  “adequacy  of  journals  and  periodicals”  (p.  40)  as  a means  of  describing 
communication  within  the  profession.  In  addition,  Molenda  (1996)  calls  for  an  examination  of  external  forces  that  influence  the 
field.  Specifically  he  asks,  “What  societal  forces  are  affecting  the  profession?  What  are  the  forces  that  assist  or  impede  the 
appropriate  adoption  of  technology  in  education?”  (p.  41). 

Our  personal  experiences  in  higher  education  and  in  the  field  have  yielded  real  concerns  about  the  field  of  educational 
technology  and  specifically  instructional  design  in  terms  of  how/if  they  incorporate  women’s  perspectives.  Most  articles,  books, 
and  literature  in  the  field  that  instructional  design  students  are  exposed  to  continue  to  be  written  by  men  from  a clearly 
masculinist  perspective.  That  the  field  has  the  potential  to  change  can  be  evidenced  by  the  introduction  of  these  texts  by  women 
(Driscoll,  1994;  Driscoll,  1998;  Seels  & Richey,  1994;  & Glasgow,  1990;  Smith  & Ragan,  1999)  used  in  the  foundational 
courses  in  the  instructional  design  program  at  a leading  university  (personal  correspondence,  2001). 

A Focus  on  Women’s  Contributions 

There  are  valid  reasons  why  a study  of  the  field  should  include  a focus  specifically  on  women’s  contributions.  As  is 
documented  in  recent  statistics,  teaching  is  (still)  a female  dominated  profession.  Findings  from  the  1993-94  survey  by  the 
National  Center  for  Education  Statistics  show  that  73%  of  teachers  in  public  K-12  education  and  75%  of  teachers  in  private  K-12 
education  are  women  (National  Center  for  Education  Statistics,  2001).  Ely  also  states,  “There  is  an  insistence  that  teachers  must 
become  technologically  literate”  (1996).  As  noted  in  both  Ely’s  (1996)  and  Molenda’ s (1998)  trends,  computers  are  pervasive  in 
education  settings  and  educational  technology  is  seen  as  a “major  vehicle  for  educational  reform”  (Ely,  1996).  Women,  then  are 
potentially  prime  purveyors  of  the  educational/instructional  technology  literature  as  it  relates  to  K-12  education. 

Reflections  on  Previous  Studies 

The  foundational  study  of  women’s  contributions  to  the  literature  (Foley  et  al.,  1994)  came  out  of  graduate  experience  that 
involved  instructional  design  classes  populated  predominantly  by  women  students  without  reference  to  one  female  author.  At 
that  time  it  was  found  that  the  percentage  of  articles  by  women  in  1 1 instructional  technology  journals  ranged  from  68%  to  20%, 
with  “no  discernable  patterns  [found]  within  each  journal  over  the  five-year  period  [1988-1992]”  (p.  59).  An  analysis  of  the 
content  of  articles  written  by  women  revealed  that  most  articles  concerned  computers  (11%)  and  computer  assisted  instruction 
(5%)  and  the  non-specific  category  of  design  and  development  (8%).  Only  2%  of  all  the  articles  written  by  women  in  the  same 
period  addressed  gender  issues.  The  fact  that  so  few  women  in  the  field  were  not  writing  about  gender  raised  the  question 
concerning  “the  experiences  of  women  attempting  to  publish  in  the  field  of  instructional  technology”  (p.  59). 

Ely  has  completed  a series  of  analyses  of  the  publications  of  the  field.  Most  recently  he  identified  eight  trends  (1996). 
Molenda  et  al.  (1998)  identified  ten  key  issues  from  their  analysis  of  the  literature  of  the  field.  A comparison  of  Ely’s  and 
Molenda’s  findings,  shown  in  Table  1 below,  identifies  many  similarities  and  some  differences. 
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Table  1 Comparison  of  Ely  s Trends  and  Molenda 's  Issues  in  Instructional  Technology 


Ely  (1996) 

Molenda  et  al.  (1998) 

Computers  are  pervasive  in  schools  and  higher  education 
institutions.  Virtually  every  student  in  a formal  education 
setting  has  access  to  a computer. 

Networking  is  one  of  the  fastest  growing  applications  of 
technology  in  education. 

Access  to  television  resources  in  the  school  is  almost 
universal. 

Advocacy  for  the  use  of  educational  technology  has 
increased  among  policy  groups. 

t tt  t 

Incorporation  of  computer-based  media  into  the 
instructional  mainstream. 

Incorporation  of  telecommunications-based  media  into 
the  instructional  mainstream. 

Incorporation  of  traditional  audiovisual  media  into  the 
instructional  mainstream. 

Acceptance  and  support  of  the  concept  of  Educational 
Technology. 

Educational  technology  is  increasingly  available  in  homes 
and  community  settings. 

— 

The  home  as  locus  of  technology  -based  learning. 

New  delivery  systems  for  educational  technology 
applications  have  grown  in  geometric  proportions. 

Application  of  advanced  interactive  technologies  (multi- 
media,  hypermedia,  virtual  reality. 

Educational  technology  is  perceived  as  a major  vehicle  in 
the  movement  toward  education  reform. 

Restructuring/Reengineering  of  basic  organizational 
processes. 

Unique  Trends  and  Issues 
Ely  (1996) 

Molenda  et  al.  (1998) 

There  is  a new  insistence  that  teachers  must  become 
technologically  literate. 

Updating  paradigms  and  procedures  for  instructional 
systems  design. 

Growing  interest  in  learner-centered,  inquiry -based 
instruction. 

Commitment  to  increasingly  authentic  assessment. 

Methodology 

Research  Questions 

Two  questions  guided  this  study: 

1.  What  percentage  of  articles  are  written  by  women  each  year  in  leading  educational  technology  journals? 

2.  What  topics  are  women  writing  about  in  leading  educational  technology  journals? 

Journal  Selection 

Selection  of  seven  journals  for  inclusion  in  this  analysis  were  based  on  Ely’s  (1999)  identification  of  those  publications  read 
by  educational  technologists.  The  seven  journals  included  in  this  analysis  are;  British  Journal  of  Educational  Technology, 
Educational  Technology,  Educational  Technology  Research  and  Development,  Innovations  in  Education  and  Training 
International,  Journal  of  Research  on  Computing  in  Education,  Learning  and  Leading  with  Technology  (formerly  published  as 
Computing  Teacher),  and  TechTrends.  Six  of  these  seven  journals  are  cited  as  leading  educational  technology  journals  for 
publishing  in  the  field  (Price  & Maushak,  2000);  Innovations  in  Education  and  Training  is  not  included  in  the  latter  list.  Analysis 
was  conducted  in  these  journals  over  a six-year  period,  1995-2000. 

Quantitative  Analysis 

The  process  for  descriptive  quantitative  analysis  involved:  1).  documenting  the  number  of  articles  written  in  each  journal;  2). 
ascertaining  which  articles  were  written  by  women  as  either  first  or  second  author;  and  3).  determining  the  percentage  of  articles 
that  were  written  by  women  annually  for  each  journal.  Editorials  and  regularly  published  columns  were  not  included  in  this 
analysis.  Selection  of  inclusion  for  an  article  was  based  on  first  or  second  authorship  by  a woman.  While  many  articles  were 
written  by  larger  groups  of  people  and  women’s  names  appeared  as  authors  in  the  group,  ERIC  protocol  limits  two  authors  in 
their  citations  and  experience  has  shown  that  lead  authorship  is  most  often  ascertained  by  order.  Because  this  is  a study  of  the 
contributions  of  women  in  the  field  we  wanted  the  articles  included  in  this  study  to  reflect  a clear  leadership  role  of  the  women 
authors,  both  in  terms  of  content  and  context  of  the  article. 

Determination  of  gender  was  based  on  analysis  of  the  first  names  of  one  of  the  first  two  authors.  Some  journals 
(conveniently)  provided  pictures  of  the  primary  authors  or  brief  biographies  that  often  included  a feminine  or  masculine  pronoun. 
Difficulties  arose  when  authors  were  identified  by  first  initials  only  and  for  those  European  names  that  are  typically  used  by 
either  women  or  men  (Robin,  Leslie,  Lyn,  etc).  In  addition,  because  neither  of  the  coders  were  familiar  with  Asian,  Arabic, 
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Indian,  and  some  European  names,  a list  of  unfamiliar  names  was  distributed  to  colleagues  from  these  geographic  areas  for 
further  identification.  For  those  cases,  where  gender  could  not  be  determined  for  first  or  second  author,  the  article  was  eliminated 
from  analysis  in  this  study. 

Content  Analysis 

This  study  employed  a content  analysis  methodology  based  on  Ely’s  (1996)  and  Foley’s  et  al.  (1994)  previous  studies.  The 
process  of  content  analysis  involved:  1).  identifying  the  total  number  of  articles  each  year,  excluding  regularly  published  columns 
and  editorials;  2).  ascertaining  the  number  of  articles  written  by  women  as  either  first  or  second  author;  3).  describing  the  content 
of  the  article  by  two  coders;  and  4).  negotiating  final  category  placement  through  a dialogic  process.  Both  coders,  authors  of  this 
article,  are  experienced  instructional  designers.  Coders  examined  each  article  authored  by  women  and  identified  a category  that 
best  described  the  focus  of  the  article.  Whenever  possible  coders  used  the  author’s  words  to  assign  a content  category.  In  cases 
where  different  words  described  the  same  thing,  (i.e.  computer  assisted  instruction/  computer  based  instruction)  one  category  was 
created  for  both  (computer  assisted  instruction).  After  independent  coding,  coders  negotiated  differences  in  selection  of  category, 
making  a case  for  one  or  the  other  or  collaboratively  choosing  an  alternate  category  that  satisfied  both  coders. 

One  difficulty  in  determining  category  arose  when  addressing  process/product  over  context.  Articles  describing  the  use  of 
technology  or  computers  in  special  education  were  placed  in  the  category  of  Education  K-12.  Finally,  because  the  purpose  of  this 
study  was  to  determine  if  women  are  writing  about  women’s  issues  in  the  field,  gender  was  a primary  category  regardless  of 
context.  Because  negotiation  was  an  essential  part  of  the  process,  analysis  had  to  be  conducted  in  a timely  fashion.  Articles  had 
to  be  independently  coded  and  discussed  within  a short  working  period  so  that  coders  could  remain  familiar  with  the  content  of 
each  article  during  negotiations. 


Findings 

What  percentage  of  articles  are  written  by  women  each  year  in  leading  educational  technology  journals? 

A total  of  1809  articles  were  published  from  1995-2000  in  the  seven  journals  included  in  this  study.  Of  these,  993  articles 
were  written  by  women  as  first  or  second  author.  The  percentages  found  in  Table  2 below  reflect  annual  averages  for  each  of  the 
journals  included  in  the  study.  The  percentages  of  articles  written  by  women  range  from  a low  of  33%  to  a high  of  74%.  An 
analysis  of  the  percentages  shows  no  discemable  trends,  increases/decreases  from  year  to  year.  Three  journals  (Educational 
Technology  Research  and  Development,  Journal  of  Research  on  Computing  in  Education,  and  Learning  and  Leading  with 
Technology)  consistently  had  an  annual  average  of  50%  or  more  articles  written  by  women;  one  journal  {Educational 
Technology)  had  an  average  of  less  than50%  of  articles  written  by  women  over  the  6-year  period.  It  should  be  noted  that  two  of 
these  journals  are  published  internationally,  although  no  differences  were  reflected  in  their  annual  percentages  from  those 
published  in  the  US.  All  of  the  journals  except  one  had  a research  focus;  Learning  and  Leading  with  Technology^  the  exception, 
is  a K-12  education  practitioner  oriented  journal.  Given  the  predominance  of  female  teachers  h public  and  private  schools 
(NCES,  2001),  no  doubt  affects  the  predominance  of  women  authors  in  this  journal.  This  journal  was  unique  in  another  feature  - 
it  published  over  600  articles  in  the  6-year  period  of  this  study,  nearly  one-third  of  the  total  number  of  articles.  Of  these  295  were 
in  the  category  of  curriculum  development.  It  was  felt  that  this  inordinate  number  of  articles  in  one  category  from  one  journal 
would  skew  the  findings  and  so  this  number  was  not  used  in  the  analysis  of  content,  but  was  included  in  the  quantitative 
calculations  of  percentages  of  articles  written  by  women.  Clearly  there  is  a difference  between  research  and  practitioner  based 
journals  both  in  terms  of  quantity  of  articles  published  and  in  the  gender  of  the  primary  authors  as  shown  in  the  figures  below. 


Table  2 Percentage  of  Articles  Written  by  Women,  1995-2000 


Journal  Title 

1995 

1996 

1997 

1998 

1999 

2000 

Means 

Learning  and  Leading  with  Technology 

54% 

50% 

69% 

74% 

74% 

66% 

64% 

Journal  of  Research  on  Computing  in  Education 

62% 

50% 

73% 

63% 

64% 

50% 

60% 

TechTrends 

42% 

55% 

55% 

41% 

71% 

59% 

54% 

Educational  Technology  Research  and  Development 

52% 

53% 

56% 

50% 

55% 

50% 

53% 

Innovations  in  Education  and  Training  International 

54% 

45% 

45% 

49% 

63% 

63% 

53% 

British  Journal  of  Educational  Technology 

50% 

47% 

39% 

58% 

48% 

69% 

52% 

Educational  Technology 

39% 

37% 

46% 

33% 

34% 

40% 

38% 

What  topics  are  women  writing  about  in  leading  educational  technology  journals? 

An  examination  of  the  content  analysis  was  done  on  two  levels.  Categories  were  sequenced  by  the  number  of  articles  found 
in  each  category.  From  the  perspective  of  individual  categories  there  were  nine  content  categories  most  frequently  cited  in  Table 
3 below. 
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Table  3 Most  Frequent  Topics  of  Articles  Written  by  Women 


Content  Category 

Frequency 

Distance  Education 

49 

World  Wide  Web  - Internet 

40 

Instructional  Strategies 

35 

Multimedia 

35 

Integration  of  Technology  in  Education 

32 

Effects  of  Technology  (in  any  context) 

30 

Teacher  Education 

29 

Computer  Software 

27 

Professional  Development 

26 

Further  analysis  clustered  individual  categories  together  into  the  themes  found  in  Table  4.  As  can  be  seen  by  this  table,  a 
quarter  of  the  articles  written  by  women  in  this  study  addressed  technology,  with  a focus  on  uses  of  the  World  Wide  Web  and  the 
Internet  Nearly  a quarter  of  the  articles  addressed  instructional  delivery  systems,  with  the  greatest  focus  in  this  theme  describing 
the  delivery  of  distance  education.  Education/training  was  also  a frequently  cited  theme  and  included  pre-service  teacher 
education  and  in-service  professional  development  in  both  education  and  business  contexts.  The  theme  of  design,  development, 
and  evaluation  spanned  the  gamut  of  tasks  these  processes  describe  with  no  concentration  in  any  one  area.  Interestingly,  society, 
the  theme  that  included  gender,  multicultural,  and  minority  issues,  was  not  addressed  in  many  of  the  articles  written  by  women  in 
these  journals. 

Table  4 Themes  and  Percentages  of  Total  Articles  Written  by  Women 


Themes 

Percentages 

Technology  - Internet,  Media,  and  Computer  Software 

25% 

Instructional  Delivery  Systems 

24% 

Education  and  Training 

15% 

Design,  Development,  and  Evaluation 

10% 

Society 

8% 

Learners  and  Learning 

5% 

Utilization  Processes 

4% 

Research  and  Theory 

3.7% 

The  Field 

3% 

Management 

1.4% 

Comparison  of  Findings 

The  themes  described  for  this  study  can  be  compared  with  those  of  Ely  (1996)  and  Molenda  et  al.  (1998)  found  in  Table  1 
previously  cited  in  this  article.  In  concurrence  with  these  recent  studies,  this  study  found  that  computers  and  technology  are 
clearly  the  most  frequently  addressed  topics  by  women  writing  in  the  field.  Technology  is  addressed  on  several  levels: 
development  (software,  multimedia),  implementation  (distance  education  and  instructional  strategies),  and  preparation  (teacher 
education,  continuing  education,  and  professional  development).  This  makes  intuitive  sense,  as  technology  addresses  the  main 
focus  of  research  in  the  field.  More  specifically,  women  also  wrote  about  computer  networking  and  applications  of  technology, 
the  World  Wide  Web  and  the  Internet,  telecommunications,  multimedia,  and  hypermedia.  In  addition,  much  of  the  content  of  the 
literature  in  this  study  focused  on  the  uses  of  technology  to  deliver  instruction,  specifically  in  the  development  of  distance 
education  as  a means  to  effectively  reach  a wider  audience  of  learners. 

As  Ely  (1996)  found  in  his  study,  women  are  writing  about  the  need  for  more  training  and  education  in  the  use  of  technology, 
as  evidenced  by  the  numbers  of  articles  on  professional  development,  teacher  education,  and  continuing  education.  Many  of  the 
articles  in  this  study  addressed  the  integration  of  technology  in  classrooms  and  curriculum,  specifically  computer 
assisted/computer  based  instruction  and  web-based  instruction  and  the  use  of  expert  systems.  Similar  to  Molenda’s  et  al.  findings 
(1998),  the  women  in  this  study  addressed  how  best  to  instruct  students,  with  a focus  on  authentic,  learner-centered  activities, 
instructional  strategies  that  support  active  and  interactive  learning  and  problem  solving.  Women  in  this  study  addressed  the 
integration  of  technolo^  in  education  and  the  effects  of  technology  as  a way  of  moving  education  reform  and  meeting  current 
needs  for  meeting  teaching  and  learning  standards. 

Contrary  to  Molenda’s  findings  (1998),  articles  in  this  study  rarely  addressed  policy  or  advocacy  in  the  use  of  technology 
nor  did  they  write  about  management  issues.  An  examination  of  the  roles  of  women  in  the  field  might  explain  this  lack  of  focus. 
How  many  women  are  in  management  and  administrative  positions  where  policy  is  addressed?  The  women  in  this  study  rarely 
cited  instructional  systems  design  (ISD),  or  research  paradigms,  methodology,  or  theory.  Are  women’s  articles  more  practitioner 


1261  J 5 


focused  than  others?  How  does  this  lack  of  articles  reflect  the  discriminatory  notion  that  women  are  not  suited  to  abstract 
thinking?  This  study  revealed  no  articles  addressing  the  role  of  computers  or  technology  in  homes  or  communities.  Is  this  a 
counter  reaction  to  women’s  traditional  place  in  the  home  and  community  and  outside  of  the  professional  technological  arena? 
Unlike  Molenda’s  findings,  there  was  no  strong  focus  on  authentic  assessment,  although  there  were  articles  that  addressed 
assessment  in  general. 

There  were  several  surprising  omissions  found  in  the  content  analysis  of  these  articles.  Unlike  Ely’s  and  Molenda’s  previous 
studies  the  articles  analyzed  here  did  not  address  the  more  traditional  audio/visual  media.  The  articles  included  in  this  study 
tended  to  describe  technology  with  an  uncritical  gaze,  without  reflection  on  theoretical  frameworks  or  critiques.  Finally,  given  the 
statements  of  concern  voiced  by  many  feminists  as  to  the  gender  inequity  in  the  field,  this  was  not  a concern  that  was  addressed 
by  many  of  the  authors  in  this  study  (1 1 articles,  1.5%).  Comparing  this  figure  to  previous  findings  of  2%  (Foley  et  al.,  1994), 
there  is  no  real  change  and  in  fact  a slight  decrease  in  the  percentage  of  articles  regarding  gender  issues.  Yet  others  (Bryson  & de 
Castell,1995;  Damarin,1994;  Henderson,! 996;  Knupfer,1997)  are  writing  about  gender  bias  and  the  design  and  applications  of 
technology,  indicating  the  issues  are  not  resolved  in  the  field.  The  omission  of  gender  issues  in  the  literature  points  to  a gap  of 
knowledge  that  needs  to  be  addressed  more  frequently.  While  in  general  gender  issues  are  seen  to  be  part  of  women’s  domain  of 
concerns,  women  in  the  educational/instructional  technology  show  slight  interest  in  developing  our  awareness  of  these  issues  in 
the  field. 

Conclusion 

Nearly  all  of  the  journals  cited  in  this  study  had  annual  percentages  of  50%  or  more  articles  written  by  women.  This  marks 
great  improvement  over  past  studies  and  recognizes  women’s  contributions  to  the  field.  However,  it  should  be  noted  that  despite 
these  aggregated  annual  statistics  there  were  individual  issues  of  journals  that  contained  no  contributions  by  women.  There  were 
some  similarities  between  the  topics  of  articles  described  by  Ely  and  Molenda  et  al.,  however  there  were  some  striking 
differences.  A curious  ancillary  finding  of  this  study  is  that  only  1.5%  of  articles  written  by  women  addressed  gender  issues  and 
technology.  This  finding  correlates  to  the  previous  study  by  Foley  et  al.  (1994)  where  2%  of  the  articles  written  by  women 
addressed  gender.  Questions  arise  as  to  why  this  is  not  a more  pervasive  theme  of  women’s  contributions  to  the  literature. 
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Abstract 

The  purpose  of  this  study  was  to  determine  the  needs  of  AECT  members.  A total  of  590  individuals  completed  a web-based 
survey  after  receiving  an  e-mail  invitation  from  AECT  This  survey  was  active  between  October  30  and  November  10,  2000.  The 
survey  was  categorized  into  three  parts:  demographics;  publications;  and  conference.  Resulting  data  was  categorized,  coded, 
and  compiled.  The  study  yielded  five  major  findings.  First,  more  than  half  of  the  respondents  are  affiliated  with  higher 
education.  Second,  the  AECT  conference  is  important  to  members  with  or  without  a trade  show.  Third,  restructuring  efforts  are 
incomplete.  Fourth,  respondents  noted  communications  problems  in  the  organization.  Fifth,  online  services  need  to  be 
improved.  Improved  communication  within  the  organization  is  imperative. 

Purpose  of  the  Study 

In  order  to  determine  needs  of  AECT  members  and  to  identify  the  association’s  strengths  and  weaknesses,  we  conducted  a 
survey.  We  have  shared  the  results  of  this  study  with  AECT  leadership  and  now  summarize  the  findings  for  AECT  members  in 
order  to  provide  feedback  on  recent  changes  in  the  association,  and  to  help  guide  decisions  about  AECT’s  future. 

Research  Questions 

Seven  research  questions  were  investigated  in  this  study: 

1 . What  services  does  AECT  provide  well? 

2.  What  are  the  needs  of  AECT  members? 

3.  How  satisfied  are  the  members  with  the  quality  of  AECT  publications  (TechTrends  & ETR&D)! 

4.  How  satisfied  are  the  members  with  the  AECT  online  conference  registration  service? 

5.  What  conference  networking  opportunities  are  helpful  to  members? 

6.  How  important  is  the  trade  show  at  the  conference  to  members? 

7.  How  satisfied  are  the  members  with  the  decision  of  restructuring  and  renaming  divisions? 

Method 

Instrument 

We  interviewed  the  current  Executive  Director  of  AECT  in  October,  2000.  He  shared  a list  of  questions  that,  after  discussion 
and  clarification,  were  incorporated  into  the  list  of  questions  included  in  this  needs  analysis  survey.  We  also  used  two  questions 
from  the  1999  member  needs  survey  to  see  how  members  would  respond  to  the  same  questions.  Then  we  designed  the  sixteen- 
question  survey  which  included  three  demographic  questions,  five  five-point  Likert-type  scale  questions,  four  multiple  choice 
questions  and  four  open-ended  questions. 

Participants 

AECT  maintains  a database  comprised  of  members  and  individuals  who  subscribe  to  the  periodicals  published  by  AECT.  E- 
mail  was  sent  to  3084  names  in  the  database,  of  whom  1846  of  were  members  as  of  November,  2000.  The  e-mail  message 
invited  individuals  to  complete  an  online  Web  survey. 

Data  Analysis 

We  ended  the  survey  on  November  1 1,  2000,  and  prepared  both  quantitative  and  qualitative  data  files  for  analysis.  After 
examining  the  data,  our  team  discarded  15  responses  from  the  total  number  to  eliminate  multiple  responses  by  individuals;  and 
this  yielded  590  member  and  nonmember  unique  responses  to  the  survey.  We  analyzed  the  quantitat  ive  data  that  were  generated 
by  multiple  choice  and  five-point  Likert -type  scale  questions  by  SPSS.  Then  we  conducted  a content  analysis  with  the  qualitative 
data  generated  by  open-ended  questions. 

The  following  paragraphs  describe  how  we  defined  categories  for  open-ended  questions. 


Question  4.  What  does  AECT  do  for  you  now  that  you  find  most  beneficial? 

In  response  to  the  question  of  what  respondents  felt  was  the  most  beneficial  aspect  of  AECT,  respondents’  comments 
indicated  8 categories  of  responses. 

Responses  that  mentioned  written  journals,  ETR&D^  or  TechTrends  were  placed  in  the  category  of  “Publications  . 
Typical  comments  from  this  category  include  the  following: 

• AECT  keeps  me  informed  of  the  most  recent  findings,  studies  and  research. 

• These  present  broader  views  and  information  that  I would  have  trouble  finding  elsewhere. 

• TechTrends 

Responses  that  mentioned  any  aspect  of  the  national  conference  were  placed  in  the  category  of  “Conference”.  Typical 
comments  from  this  category  include  the  following 

• The  annual  conference  is  the  most  beneficial  part  of  AECT  As  a media/info-tech  administrator  I always  am  able  to  get 
GREAT  information  that  I can  apply  when  I return  home. 

• The  national  conferences  were  always  of  value  in  helping  one  understand  new  theories,  techniques,  and  ways  in  which 
to  use  media  in  the  teaching  and  learning  process. 

• Annual  Conference 

Responses  that  mentioned  any  aspect  of  social  or  professional  interaction,  whether  as  part  of  a conference, 
presentations,  or  informal  gatherings  outside  of  the  conference  were  placed  in  a category  named  “Networking”.  Typical 
comments  from  this  category  include  the  following: 

• Networking  with  colleagues  and  diversity  of  programming  when  it  is  good. 

• Provides  a venue  for  the  exchange  of  knowledge  and/or  research  in  our  field.  Also  provides  opportunities  to  network 
and  interact  with  colleagues. 

• Provides  contacts  with  colleagues,  old  and  new,  which  leads  to  opportunities  to  visit  and  work  at  institutions  outside 
the  U.S. 

Responses  that  mentioned  any  aspect  of  professional  improvement,  current  issues,  trends,  or  keeping  up  with 
developments  were  placed  in  the  category  entitled  “Professional  Development.”  Typical  comments  from  this  category  include  the 
following: 

• Keep  up  with  latest  development  in  educational  technology. 

• AECT  helps  to  keep  me  updated  as  to  what  is  current  in  the  field. 

• Keeps  my  coworkers  up  to  date  on  current  issues  and  trends. 

Responses  that  mentioned  any  aspect  of  electronic  communication  through  the  Internet  or  the  World  Wide  Web,  email 
or  listserv  communications  were  placed  in  the  category  entitled  “Web  Services”.  Typical  comments  from  this  category  include 
the  following: 

• The  job  postings  site  is  most  useful. 

• The  Website  is  useful.  I get  most  of  the  information  from  the  AECT  Web. 

• listserv  newsletter. 

Responses  that  included  the  concept  of  employment -related  issues  were  placed  in  the  category  of  “Employment 
Information.”  Typical  comments  from  this  category  include  the  following: 

• It  provides  employment  leads. 

• Job  listing  (when  current). 

Responses  stating  that  the  respondent  does  not  like  some  general  aspect  of  AECT  were  placed  in  the  category  entitled 
“Negative”.  Typical  comments  from  this  category  include  the  following: 

• At  this  time  nothing.  Several  years  ago  the  journal  and  conference  provided  good  opportunities  for  learning. 

• I do  not  currently  benefit  from  AECT. 

• I have  found  nothing  of  value  the  last  two  or  three  years...  the  organization  has  been  in  transition  and  cannot  function 
until  the  transition  is  complete. 

Responses  that  did  not  fit  into  any  of  the  categories  that  had  been  established  for  this  question  were  placed  in  one 
category  that  was  entitled  “Others”.  Each  of  the  responses  could  possibly  make  its  own  category  and  stand  alone  in  this  report, 
but  our  research  team  felt  that  they  should  be  joined  in  the  “Other”  category.  Typical  comments  from  this  category  include  the 
following: 

• The  membership  is  diverse,  freeing  me  from  the  parochial  view  of  one  section  of  the  profession;  participation  in  the 
organization  and  its  governance  is  encouraged  (most  national  organizations  discourage  it  with  red  tape);  the  people 
are  wonderful,  supportive  and  generous  with  their  expertise;  the  summer  leadership  institute  is  excellent. 

• Phil  Harris,  with  his  staff,  and  the  AECT  President  and  Board  of  Directors  are  actively  looking  for  ways  to  support  and 
encourage  the  AECT  International  Student  Media  Festival.  Charles  White  shares  the  vision  of  ISMF  that  it  serves  as  a 
natural  introduction  to  AECT  for  teachers  and  students  interested  in  the  role  of  media  in  education. 

• TechTrends,  copyright  issues,  new  products,  how  a new  technology  works  in  real  situations.  Ed  issues  and  On-line 
classes.  New  topics  on  Internet  usage.  Teacher  training. 


i 3-3  BEST  COPY  AVAILABLE 

o 

ERIC 


129 


Question  5:  Please  list  the  most  important  thing  AECT  should  be  doing  for  you  now  that  it  does  not  currently 
do. 

In  response  to  question  five  in  which  respondents  were  asked  what  it  was  they  felt  was  the  most  important  thing  that 
AECT  should  be  doing  that  they  were  not  currently  doing.  Member  comments  separated  into  7 categories  of  responses. 
Responses  that  indicated  some  kind  of  need  that  was  associated  with  the  AECT  National  Conference  were  placed  into  the 
category  entitled  “Conference”.  Typical  comments  from  this  category  include  the  following: 

• Thought  that  the  lack  of  exhibits  at  national  conferences  was  big  weakness,  but  that  will  be  satisfied  in  Denver,  I 
assume.  Organization  is  now  doing,  or  about  to  do,  most  of  the  things  that  / think  a national  professional  organization 
should  do. 

• I would  like  to  participate  in  a conference  but  time,  location,  and  funding  are  an  issue.  / would  like  to  see  AECT 
sponsor  an  online  conference  or  include  online  conference  participation  at  its  f2f  conferences. 

• Providing  a scholarly  conference  that  is  respected  and  has  credibility.  Provide  timely  information  on  topics  relevant  to 
our  field. 

Responses  that  included  a mention  of  communications,  response  or  lack  of  response  to  submitted  questions,  or  a flow 
of  information  were  put  into  the  category  entitled  “Communications”.  Typical  comments  from  this  category  include  the 
following: 

• Communicate  with  members  better.  I sent  in  my  membership  months  ago,  and  I’ve  started  receiving  pubs,  but  I don  7 
remember  receiving  any  acknowledgment  of  my  membership.  / was  wondering  about  it  until  I started  receiving 
Tech  Trends. 

• Create  a clear  link  of  communication  between  the  members  and  administration  so  members  can  become  more  involved 
in  policy  and  know  what  to  expect  from  one  conference  to  the  next. 

• Provide  data/opportunities/support  for  better  communication  throughout  the  year. 

Responses  that  mentioned  publications,  articles,  ETR&D,  TechTrends,  or  journals  were  placed  in  the  category  entitled 
“Publications”.  Typical  comments  from  this  category  include’ the  following: 

• AECT  should  be  continuing  to  publish  a scholarly  journal  I have  not  yet  received  an  issue  of  ETR&D. 

• More  publications. 

• AECT journals  need  to  improve  their  quality  of  articles.  Emphasize  research-based  reviews  of  educational  technology 
uses  in  the  classroom  to  help  K-12  teachers  in  implementing  technology  in  the  classroom. 

Responses  that  referenced  electronic  material  including  websites,  listservs,  electronic  communications,  or  email  were 
placed  in  a category  entitled  “Web  Services.”  Typical  comments  from  this  category  include  the  following: 

• Message  board  or  listserv  for  ongoing  member  interaction/discussion. 

• Provide  me  with  the  electronic  capability  to  reach  out  to  my  state  people  and  to  other  colleagues  individually  and 
updating  me  electronically  on  the  state  of  the  organization.  Our  name  says  communication  and  technology  or, 
communication  through  technology. 

• Access  to  electronic  materials  e.g.  back  issues  of  journals  etc. 

Responses  that  included  a discussion  of  the  restructuring  efforts,  organizational  efforts,  or  changes  within  the  AECT 
were  placed  in  a category  entitled  “Organizational  Restructure,”  Typical  comments  from  this  category  include  the  following: 

• Restructure  organization  so  it  can  best  serve  its  members  (e.g.  conference  coordination,  quality  control  of  its 
publications,  etc.). 

• Be  more  organized  and  professional. 

• Provide  a viable  organization  that  combines  ’clout’  with  an  outlet  for  academic  research. 

Responses  that  indicated  a need  for  development  of  professional  skills  or  services  or  for  keeping  up  with  trends  were 
placed  in  a category  entitled  “Professional  Development,”  Typical  comments  from  this  category  include  the  following: 

• Providing  a sense  that  this  organization  is  keeping  up  with  the  latest  trends  in  instructional  design  throughout  the  year. 

• More  professional  development  activities  on  the  regional  level. 

• Professional  Development.  I know  many  people  are  involved  in  the  leadership  and  other  efforts  within  AECT,  but  I 
don  V know  how  to  get  into  that  ’inner  circle.  ’ I’ve  always  felt  like  an  outsider,  except  for  participation  in  the  annual 
conference. 

Responses  that  noted  single  instances  of  a suggestion  were  sorted  into  the  category  entitled  “Other”.  There  was  no 
discernible  trend  noted  in  this  category,  and  so  they  were  combined  into  a single  category  of  suggestions.  Typical  comments 
from  this  category  include  the  following: 

• Role  of  Media  Specialists  in  the  changing  school  environment. 

• Continue  to  seek  member  involvement  in  ways  like  this. 

• Uncertain  what  it  could  do  for  me  since  I’m  a bit  away  from  the  center  of  things  and  haven ’t  been  able  to  attend  a 
conference  for  several  years.  Mainly  - just  keep  information  coming. 

• lam  not  aware  of  services  that  would  be  beneficial  for  me  that  I am  not  getting  from  somewhere  else  that  AECT  should 
provide. 
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Question  15,  How  did  you  first  become  aware  of  AECT? 

In  response  to  the  question  of  how  respondents  first  became  aware  of  the  AECT,  comments  were  sorted  into  6 categories  of 
responses.  Responses  that  indicated  the  individual  had  first  learned  about  the  organization  while  attending  a graduate  program  or 
that  they  had  learned  about  the  organization  in  graduate  school  were  included  under  the  category  entitled  “In  Graduate  School.” 
Typical  comments  from  this  category  include  the  following: 

• From  faculty  members  during  graduate  school 

• As  a graduate  student  some  35  years  ago. . . . 

• Through  discussion  with  faculty  in  graduate  school. 

Responses  indicating  that  initial  awareness  of  the  AECT  organization  had  come  from  contacts  with  professional 
colleagues  and  peers  were  grouped  under  the  category  “From  Colleagues  and  Peers.”  Typical  comments  from  this  category 
include  the  following: 

• Fellow  professionals. 

• Recommended  by  my  colleagues. 

• My  father  has  been  a member  for  many  years,  and  then  when  I entered  the  field,  my  boss  was  a member. 

Responses  from  individuals  who  named  a variety  of  different  sources  that  did  not  fit  easily  under  existing  categories 

were  grouped  under  the  category  entitled  “From  Various  Sources.”  Typical  comments  from  this  category  include  the  following: 

• As  a beginning  professional  by  attending  a regional  conference. . . 

• Link  from  another  web  site 

• In  1980  we  hosted  the  then  active  Missouri  Affiliate  in  Cape  Girardeau  MO.  I have  been  a member  ever  since. 
Responses  from  members  who  had  been  a part  of  the  organization  for  such  a long  period  of  time  that  they  couldn’t 

remember  the  original  exposure  to  the  organization  were  included  under  the  category  entitled  “Cannot  Remember.”  Typical 
comments  from  this  category  include  the  following: 

• Don  V remember. . . 

• Don  V recall . . 

• ...I  don  V remember  how  I became  aware  of  it. 

Responses  from  individuals  who  first  learned  of  the  AECT  by  means  of  written  publications  or  journals  are  organized 
under  the  category  entitled  “From  Publications.” 

• learn  from  TechTrends  and  join  by  mail. 

• In  search  of  publication  outlet. 

• Via  periodics  (sic)  at  University ’s  library. 

Responses  from  individuals  who  had  first  learned  of  the  existence  of  the  AECT  while  attending  an  AECT  conference 
are  included  under  the  category  entitled  “At  the  AECT  Conference.”  Typical  comments  from  this  category  include  the 
following: 

• Convention  in  Miami  in  1979. 

• Through  exhibits  at  the  WEMA  conference  in  Wisconsin. 

• Really  got  involved  after  attending  an  AECT  Region  6 leadership  conference  in  1981... 

Question  16.  Please  list  any  other  comments  that  would  benefit  the  AECT  leadership  in  determining  future 
directions. 

In  response  to  the  invitation  asking  respondents  to  list  any  general  comments  that  would  benefit  the  AECT  leadership  in 
determining  future  directions,  respondents’  comments  were  sorted  into  1 1 general  areas.  Responses  that  indicated  the  AECT 
needs  to  focus  efforts  in  the  restructuring  process  were  reported  under  the  category  entitled  “We  need  to  define  AECT’s  focus.” 
Typical  comments  from  this  category  include  the  following: 

• AECT  must  clearly  define  its  purpose.  Direct  activities  to  that  purpose.  Make  its  membership  services  supportive  of 
that  purpose. 

• AECT  needs  to  decide  who  its  members  are  and  provide  opportunities  for  those  members  to  meet... 

• Find  your  niche,  and  stick  to  the  knitting.  Don't  try  to  be  all  things  to  all  people,  which  the  old  AECT  never  learned. 
Based  on  what  I see  in  the  new  division  structure,  Fm  not  sure  AECT  has  learned  it  yet. 

Responses  that  spoke  of  perceived  communication  difficulties  between  the  administration  of  AECT  and  the  membership 
were  grouped  under  the  category  entitled  “AECT  needs  to  improve  communications  with  the  membership.”  Typical  comments 
from  this  category  include  the  following: 

• Please  have  someone  available  at  the  telephones  at  the  office.  I have  tried  to  reach  a person  on  several  occasions  and 
only  left  messages  which  were  not  returned. 

• To  strengthen  AECT  membership,  I believe  AECT  should  acknowledge  its  members  by  sending  a membership  card... 

• There  appears  to  be  no  way  to  become  involved  in  the  governance  and  workings  of  AECT  without  ‘knowing  someone. ' 
Please  promote  opportunities  for  people  to  become  involved  in  the  divisions  and  councils  or  in  planning  the 
conference. 

Responses  that  indicated  problems  or  solutions  to  difficulties  encountered  during  attendance  at  the  national  conference 
are  grouped  under  the  heading  “Conference  needs  improvement.”  Typical  comments  from  this  category  include  the  following: 


• ...improve  the  organization  of  the  conference.  I overheard  several  people  commenting  on  how  unorganized  it  seemed 
to  be.... 

• The  program  needs  to  be  better  organized.  Denver  was  a mess  with  sessions  listed  in  one  place,  changed  in  the 
addendum  in  another,  and  shown  on  the  TV  monitors  in  a third.... 

• ...I  go  to  this  conference  because  of  its  diversity  - which  was  down  a good  bit  this  time.  Frankly,  it  has  overpriced 
itself  at  this  point.  I would  not  have  attended  this  conference  at  this  price  if  I were  not  involved  in  governance  areas. 
Responses  that  indicate  problems  or  issues  that  have  been  apparent  within  the  AECT  website  are  included  under  the 

category  entitled  “Website  needs  improvement.”  Typical  comments  from  this  category  include  the  following: 

• ...keep  the  WWW  site  up-to-date  (online  job  listings  are  great...  member  directory  on-line  would  be  nice). 

• Please,  please  upgrade  the  website. . . 

• The  AECT  website  needs  some  dramatic  fixing  by  people  who  know  good  graphic  and  user -interface  design  and  good 
content  editors.... 

Responses  of  a broad  nature  that  addressed  very  specific  circumstances  and  yet  were  unable  to  be  grouped  with  other 
responses  formed  a category  entitled  “Miscellaneous.”  Typical  comments  from  this  category  include  the  following: 

• / joined  AECT  as  a graduate  student  and  attended  conventions.  At  the  225  dollar  price  of  Denver,  I as  a graduate 
student  would  never  have  been  in  AECT  or  participated  for  so  many  years. 

• Get  great  speakers  instead  of  so  many  social  functions.  Cut  the  exhibits  if  we  can't  get  enough  vendors  to  make  it 
worthwhile.  Confirm  all  presenters  in  advance.  Cut  the  costs  if  possible. 

• We  MUST  have  a better  process  for  letting  people  know  if  their  papers  were  accepted.  We  did  not  find  out  until  we  got 
the  program  in  Denver  that  one  of  ours  was  accepted. 

Responses  from  individuals  who  expressed  a general  sense  cf  pleasure  with  the  organization  are  grouped  under  the 
category  entitled  “Pleased  with  AECT.”  Typical  comments  from  this  category  include  the  following: 

• lam  glad  to  see  that  AECT  is  moving  forward  again. 

• Keep  up  the  great  work. . . 

• AECT  still  has  an  important  role  to  play  in  the  evolution  of  technology-based  education/training,  but  perhaps  not  as  an 
independent  society  any  longer.  A merger  with  AACE  or  ISP!  may  be  necessary  to  accomplish  what's  needed. 
Responses  from  individuals  who  expressed  their  appreciation  of  the  fact  that  AECT  conducted  this  survey  to  determine 

the  interests  of  AECT  members  are  grouped  in  the  category  entitled  “Like  the  survey.”  Typical  comments  from  this  category 
include  the  following: 

• Can't  think  of  anything...  this  strikes  me  as  a good  survey. 

• lam  pleased  to  see  this  survey  being  done  — it  speaks  well  for  the  leadership. 

Responses  from  individuals  who  felt  that  AECT  was  not  strong  enough  in  getting  positive  aspects  of  the  AECT  program 
out  to  the  attention  of  the  membership  are  included  in  the  category  entitled  “Marketing  needs  improvement.”  Typical  comments 
from  this  category  include  the  following: 

• Some  steady,  dependable  "marketing"  will  be  needed. 

• Are  you  marketing  AECT  to  the  school  districts?  Is  AECT  represented  at  library  and  technology  conferences?  I do  not 
recall  seeing  it  at  those  I attend.  I hope  this  helps.  Thank  you  for  all  the  support  I have  received  through  AECT. 
Responses  from  individuals  who  feel  that  the  general  leadership  of  the  AECT  has  diminished  over  time  and  that  it 

needs  to  be  asserted  are  included  in  the  category  entitled  “Leadership  needs  improvement.”  Typical  comments  from  this 
category  include  the  following: 

• / want  AECT  to  resume  a strong  leadership  position  once  more  in  the  field  of  education  & communications  technology. 

• We  need  to  have  divisions  take  a strong  leadership  role  overall  and  in  conference  planning  specifically. 

Responses  that  mentioned  ETR&D,  TechTrends,  or  other  journals  were  grouped  under  the  category  entitled 
“Publications.”  Typical  comments  from  this  category  include  the  following: 

I think  ETR&D  should  publish  more  variety  and  a larger  number  of  articles.  For  example,  it  should  cover  more  articles 
related  to  network-based  learning  or  Web-based  learning. 

I think  we  should  have  more  publications,  and  that  we  should  start  a research  mentoring  program.  Also,  I would  like  to 
see  a new  journal  that  focuses  on  on-line  learning.  That  journal  could  be  on-line. 


Findings 

Of  the  590  respondents,  487  (82.5%)  respondents  reported  being  current  members,  47  (8%)  respondents  reported  not  being 
current  members  and  56  (9.5%)  respondents  were  not  certain  whether  or  not  they  were  members  (“Not  Sure”).  With  respect  to 
whether  or  not  the  respondent  resides  in  the  U.S.,  563  (95.4%)  respondents  stated  that  they  reside  in  the  United  States  while  27 
(4.6%)  respondents  reported  residing  outside  the  U.S. 
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Question  3:  What  is  your  employment  setting? 


• Approximately  two-thirds  of  the  respondents  reported  Higher  Education  attiliation,  which  includes  junior/community 
college  or  technical  institute,  college  or  university  campus,  or  graduate  student. 

Question  4:  What  does  AECT  do  for  you  now  that  you  find  most  beneficial? 
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Respondents  identified  4 major  areas  that  were  most  beneficial.  These  include  Publications,  Conferences, 
Networking,  and  Professional  Development. 

5:  Please  list  the  most  important  thing  AECT  should  be  doing  for  you  now  that  it  does  not  currently  do. 
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Question  6:  AECT  publishes  Tviiigjuuiiiuiis-jcvri  Cf  •m.s  uiiu  kj  m.  * Aauvw  j %riA  receiving  your  publications 

in  a timely  manner? 
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Of  the  580  surv«58ttsponses,  419  (72.2%)  respondents  indicated  that  they  have  received  the  publications  in  a timely 
manner.  It  is  interesting  to  note  that  31  respondents  chose  the  item”  I am  not  a current  AECT  member.” 


Question  7:  How  satisfied  are  you  with  the  quality  of  recent  attitudes  in  Tech  Trends? 
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Of  the  560  survey  responses,  400  (71.5%)  respondents  expressed  satisfaction  with  the  quality  of  recent  articles  in 
TechTrends.  There  were  60  respondents  (10.7%)  expressed  some  degree  of  dissatisfaction  with  the  quality  of 
TechTrends. 


Question  8:  How  satisfied  are  you  with  the  quality  of  recent  attitudes  in  ETR&Dl 
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Of  the  439  survey  responses,  271  (61.7%)  respondents  expressed  satisfaction  with  the  quality  of  recent  articles  in 
ETR(SlD.  There  were  only  22  respondents  (5%)  who  expressed  some  degree  of  dissatisfaction  with  quality  of  ETR&D. 
Approximately  one-third  of  the  respondents  indicated  that  they  were  undecided  with  regard  to  their  satisfaction  of 
qualities  in  ETR&D. 
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Question  9:  Have  you  attended  an  AECT  National  Conference  within  the  past  3 years? 
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If  YES,  please  indicate  which  of  the  following  were  helpful  for  meeting  people  (i.e.,  networking)? 
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Respondents  addressed  the  question  of  what  was  the  most  important  thing  that  AECT  could  do  to  promote  meeting 
people,  the  concept  of  networking  with  others.  Four  networking  opportunities  stood  out  from  the  responses,  and  these 
included  meeting  people  at  presentations  and  roundtables,  the  Wednesday  night  Roundup  (AECT  Reception  at  the 
conference).  Division  and  Council  receptions,  and  University  receptions. 
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Question  10:  Were  you  satisfied  with  the  new  online  AECT  Conference  registration  service  available  through  the  AECT 
web  site? 
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A total  of  216  (53.4%)  respondents  expressed  some  degree  of  satisfaction  with  the  online  registration. 

Note  that  almost  a third  of  respondents  were  undecided  with  respect  to  their  satisfaction  with  the  online  registration 
service. 
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Question  11:  AECT  changed  its  conference  date  from  early  February  to  late  October,  in  order  to  meet  concurrently  with 
the  National  School  Board  Association,  so  that  AECT  members  could  visit  the  large  trade  show.  Do  you  agree  with  this 
decision? 
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There  was  a ratio  of  approximately  three  to  one  among  respondents  who  agreed  with  the  decision  to  change  the  date  of 
the  AECT  conference  so  that  it  met  concurrently  with  the  National  School  Board  Association  (NSBA), 


Question  12:  How  do  you  feel  about  the  trade  show  at  the  AECT  Conference? 


V^ry  lm|>ortant 
Important 
Undecided 
Not  Imporlant 
Not  at  all  Important 

( 

N=50S 


28.7% 

fl 

^ D 

£ 


S 20.8% 


9.5% 


37  J% 


10 


20 


30 


40 


• 336  (56.4  %)  respondents  believed  that  the  trade  show  is  important  or  very  important.  Question  13:  Do  you  know 

that  AECT  has  recently  established  a toll-free  number? 
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More  than  three-fourths  of  respondents  were  not  aware  that  the  AECT  had  obtained  a toll-free  number. 
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Question  14:  AECT  made  the  decision  in  1999  to  restructure  and  rename  divisions  and  councils.  How  do  you  feel  about 
that? 
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• A total  of  195  (36,3%)  respondents  expressed  satisfaction  with  the  restructuring  efforts.  This  included  187  (34.8%) 
respondents  who  were  satisfied  and  8 (1.5%)  respondents  who  were  very  satisfied.  It  is  important  to  note  that  a greater 
percentage  cf  respondents  expressed  some  degree  of  satisfaction  with  the  restructuring  (36.3%)  than  those  who 
expressed  some  degree  of  dissatisfaction  with  the  restructuring  efforts  (9.1%).  A slight  majority  of  respondents 
expressed  neither  satisfaction  nor  dissatisfaction  with  the  restructuring  efforts. 


Question  15:  How  did  you  first  become  aware  of  AECT? 
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• The  greatest  percentage  (61%)  learned  of  the  organization  from  their  graduate  school  experience,  and  1 16  of  those 
respondents  heard  of  AECT  directly  through  a graduate  school  professor. 

Question  16:  Please  list  any  other  comments  that  would  benefit  the  AECT  leadership  in  determining  future  directions. 
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• Respondents  were  given  the  opportunity  at  the  end  of  the  survey  to  express  themselves  on  any  topic  that  they  felt  was 
beneficial  to  the  organization.  The  largest  group  (31%)  felt  that  defining  AECT’s  focus  is  mission  critical. 

• 1 5%  of  respondents  felt  that  AECT  needs  to  improve  communication  with  its  members  and  22%  of  respondents  felt 
that  improvement  is  needed  in  the  AECT  national  conference. 
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Discussion 


Higher  Education 

It  is  important  to  note  the  fact  that  more  than  half  of  the  respondents  are  affiliated  with  higher  education  at  the  community 
college  and  university  level.  Of  the  K- 12  respondents  to  this  survey,  they  often  found  that  there  was  little  of  value  for  them  at  the 
trade  show  in  conjunction  with  the  conference.  As  AECT  continues  the  restructuring  process,  it  will  be  important  to  determine 
the  role  of  K-12  members  in  the  organization.  If  the  association  wishes  to  include  the  K-12  members  in  a significant  manner, 
they  must  examine  methods  to  do  so.  It  is  clear  that  the  organizational  emphasis  is  currently  on  providing  services  to  the 
majority  of  their  membership,  which  is  involved  with  Higher  Education  as  evidenced  by  this  survey. 

More  than  half  of  the  membership  of  AECT  joined  because  of  an  introduction  that  occurred  during  their  work  in  graduate 
school.  Of  this,  about  a third  mentioned  that  they  had  first  heard  of  the  AECT  through  their  professors.  AECT’s  efforts  at 
getting  out  the  word  in  the  graduate  schools  that  teach  courses  having  a technology  base  seems  to  pay  off  in  increased 
membership.  Anything  that  AECT  can  do,  such  as  giving  breaks  to  graduate  students  for  conference  participation  or 
membership,  is  likely  to  realize  a benefit  for  the  organization  as  a whole  later  on. 

Conference 

AECT  members  state  that  they  found  the  conference  to  be  the  second  greatest  benefit  of  membership.  The  conference  was 
also  stated  to  be  the  second  highest  service  that  AECT  offered  requiring  improvement.  Members  were  vocal  in  offering  remedies 
for  the  conference,  and  AECT  administration  would  be  well  advised  to  examine  these  comments  thoroughly. 

Communication 

An  issue  that  emerges  clearly  across  all  of  the  survey  questions  is  the  issue  of  communication  between  the  administration  of 
AECT  and  the  membership  of  the  organization. 

The  issue  that  is  mentioned  most  often  is  the  frustration  of  individuals  who  are  unable  to  receive  a response  to  queries  for 
information.  A good  example  of  this  is  the  fact  that  9.5%  of  the  individuals  to  whom  this  survey  was  sent  were  uncertain  of  their 
membership  status.  AECT  uses  time  and  resources  to  contact  the  individuals  who  are  a part  of  their  database,  and  it  is  strongly 
suggested  that  they  clarify  this  situation  at  the  earliest  opportunity. 

Most  people  were  pleased  with  the  online  registration  services  that  were  available  for  the  conference.  There  were  some 
important  exceptions  noted.  Of  the  respondents  who  mentioned  that  a purchase  order  was  needed  so  that  their  sponsors  could 
pay  for  registration,  it  seems  that  such  a service  was  not  offered.  Additionally,  concern  was  expressed  about  the  security  of  the 
web  site  and  people  were  uncertain  that  their  personal  information  would  be  kept  in  confidence.  AECT  might  consider 
explaining  the  security  issue  in  future  registration  offerings  to  address  these  concerns. 

A final  issue  exists  with  respect  to  the  online  services  area.  Concern  was  expressed  that  the  AECT  should  be  a leader  in  the 
field  of  website  technology,  and  many  felt  that  the  current  website  was  lacking  in  information,  design,  and  services.  It  was 
mentioned  that  AECT  should  spend  resources  to  see  that  the  website  would  address  these  issues  in  a state  of  the  art  manner  so  as 
to  present  a model  website  to  the  world. 

Most  people  did  not  know  that  the  AECT  has  a toll  free  number.  If  the  Association  wants  this  number  to  be  highly 
visible,  they  must  consider  placement  of  the  number  in  a visible  manner  through  all  of  the  ways  that  they  contact  the 
membership.  This  would  include  the  website,  email  communications,  journals,  and  personal  communications. 

Publications 

AECT  publishes  two  journals,  TechTrends  and  ETR&D.  The  first  is  available  with  membership  and  the  second  only  by 
special  subscription.  This  must  be  thoroughly  explained  to  all  who  receive  these  publications.  There  was  some  confusion  about 
these  journals  at  time  of  this  study.  Further,  publication  and  delivery  of  the  journals  in  a timely  manner  should  be  a priority  to  the 
organization. 

Leadership 

The  task  of  restructuring  AECT  to  better  meet  the  needs  of  the  membership  is  not  complete,  and  efforts  in  this  area  need  to 
continue.  It  is  essential  that  restructuring  efforts  be  communicated  to  the  membership  of  AECT  and  that  the  membership  must 
become  more  actively  involved  at  more  stages  in  the  restructuring.  If  it  is  at  all  possible,  AECT  administration  should  initiate  a 
quality  control  action  aimed  at  assuring  members  that  their  needs  are  being  addressed. 
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Abstract 

The  concepts  of  usability  and  user-centered  design  (UCD)  have  grown  in  popularity  over  the  past  20  years  as  measured  by 
the  number  of  research  and  mainstream  articles  devoted  to  their  discussion.  As  with  all  new  developments,  however,  there  is 
always  the  question  of  how  things  work  in  practice  compared  to  theory.  A survey  on  83  software  developers  mostly  in  small-to- 
medium  sized  companies  in  a variety  of  industries  was  conducted  to  examine  software  developers ' views  on  UCD  and  usability 
practices  and  to  illuminate  how  current  practices  relate  to  theory.  Results  of  a descriptive  analysis  of  the  22  Likert-scale  attitude 
question  items  suggested  that  respondents  had  moderately  positive  attitudes  towards  UCD  activities  and  discipline.  The  Likert- 
scale  items  were  subsequently  factor-analyzed  and  the  results  suggested  that  the  respondents  tended  to  agree  that  UCD  is  worth 
the  effort  and  cost.  They  also  tended  to  agree  that  it  is  important  to  conduct  many  use  test  sessions  and  they  learned  a lot  about 
their  products  from  user  test  sessions.  Software  developers  who  reported  that  their  companies  followed  important  UCD 
practices  were  more  likely  to  agree  with  the  view  that  UCD  is  worth  the  effort  and  cost.  Those  who  have  attended  usability  test 
sessions  were  more  likely  to  agree  that  user  test  sessions  are  valuable,  and  that  UCD  is  worth  the  effort  and  cost.  However, 
those  who  have  attended  usability  test  sessions  also  were  more  likely  to  agree  that  UCD  is  more  work  and  costs  more  than 
conventional  development  activities.  Also,  significantly  more  good  usability  practices  were  reported  by  software  developers  who 
worked  on  teams  that  either  hired  usability  consultants  or  had  a usability  specialist  on  their  teams  compared  with  those  who  had 
no  usability  specialists  at  all.  While  software  developers  held  positive  attitudes  towards  UCD,  it  was  notable  that  they  did  not 
report  that  their  companies  used  practices  that  are  central  to  UCD.  It  appears  that,  while  many  software  developers  agree  that 
UCD  is  a good  idea,  it  tends  not  to  be  implemented  fully  in  practice. 

Introduction 

The  concepts  of  usability  and  user-centered  design  (UCD)  have  grown  in  popularity  over  the  recent  20  years  as  measured  by 
the  number  of  research  and  mainstream  articles  devoted  to  their  discussion.  As  with  all  new  developments,  however,  there  is 
always  the  question  of  how  things  work  in  practice  compared  to  theory.  The  opportunities  for  UCD  to  have  a significant  impact 
on  the  rapid  developments  in  information  technology  are  infinite  in  number  and  will  only  increase  as  new  technologies  emerge. 
To  ensure  that  this  impact  is  realized,  however,  organizations  must  understand  how  to  best  organize  themselves  and  their 
practices  to  take  full  advantage  of  what  UCD  has  to  offer. 

The  purpose  of  this  study  is  to  examine  software  developers’  views  on  UCD  and  usability  practices  in  order  to  illuminate 
how  current  practices  relate  to  theory.  The  results  of  this  study  will  help  researchers  and  organizations  understand  what  factors 
are  critical  to  integrating  an  effective  UCD  approach  in  the  development  lifecycle. 

Deflning  the  Concepts 

User-centered  design  is  an  approach  to  product  development  that  emphasizes  keeping  the  end  user  “front  and  center” 
throughout  the  product  development  process.  Unlike  the  specific  techniques  and  methods  that  make  it  up,  UCD  is  a philosophy 
toward  designing  products  (Norman,  1988),  the  underlying  theme  of  which  is  that  developers  who  stay  attuned  to  the  concerns, 
thought  processes,  habits,  and  preferences  of  the  people  targeted  to  use  their  products  will  develop  interfaces  and  services  that  are 
easier  to  use,  have  greater  utility,  and  are  more  enjoyable  for  their  customers  (Rubin,  1994). 

If  UCD  is  the  philosophy  that  guides  an  effective  development  process,  usability  may  be  seen  as  the  end  result.  Once 
known  simply  as  “user-friendliness”  (Norman  & Draper,  1986),  the  concept  of  usability  has  attracted  much  attention  over  recent 
years  and  is  currently  considered  to  consist  of  the  following  five  attributes:  (a)  leamability,  (b)  efficiency,  (c)  memorability,  (d) 
errors,  and  (e)  satisfaction  (Nielsen,  1993). 

Perhaps  one  of  the  most  valuable  tools  in  the  designer’s  UCD  toolbox  is  usability  testing.  This  method  affords  the  design  team 
the  unique  opportunity  to  observe  the  actions  of  the  target  user  population  first-hand.  Usability  testing  allows  designers  to 
observe  authentic  users  performing  authentic  tasks  and  scenarios.  While  many  tests  occur  in  a laboratory  environment  to  make 
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observation  and  data  collection  easier,  field  visits  to  the  users’  actual  workplaces  provide  the  additional  benefit  of  an  authentic 
context  as  well. 

Dumas  and  Redish  (2000)  identify  five  characteristics  that  define  usability  testing: 

1.  The  primary  goal  is  to  improve  the  usability  of  a product 

2.  The  participants  represent  real  users 

3.  The  participants  perform  real  tasks 

4.  You  observe  and  record  what  participants  do  and  say 

5.  You  analyze  the  data,  diagnose  the  real  problems  and  make  recommended  changes  to  fix  those  problems  (p.22). 

The  Importance  of  “Getting  Close”  to  Your  Users 

The  activity  of  “requirements  gathering”  has  long  been  a core  element  of  the  software  design  process  (Boehm,  1988),  yet 
critics  of  poorly  designed  software  point  out  that  gathering  requirements  through  focus  group  discussions  or  by  talking  to 
management  often  fails  to  identify  what  is  needed  to  make  a usable  product.  The  only  way  to  accurately  define  what  people  will 
be  able  to  use  is  to  gather  information  directly  from  the  users  themselves.  As  with  so  many  things  in  life,  however,  all  user- 
centered  design  activities  are  not  created  equal.  Sbme  methods  are  more  successful  than  others  at  bringing  users  and  designers 
close  together. 

It  would  seem  that  the  simplest  way  of  gathering  information  from  users  is  to  ask  them  what  they  want.  Unfortunately,  we 
know  that  users  do  not  always  know  what  they  want.  Indeed,  Andre  & Wickens  (1995)  cite  a host  of  studies  demonstrating  that 
users  not  only  don’t  know  what  they  want,  but  that  they  frequently  make  bad  choices  as  well.  In  one  study  of  six  different 
interface  designs,  users  consistently  indicated  a preference  for  those  designs  that  they  performed  most  poorly  on  (Bailey,  1995). 
The  results  emphasize  how  important  it  is  to  include  empirical  data  on  performances  in  addition  to  asking  users  what  they  like. 

Conducting  needs  analysis  interviews  and  performing  content  sorting  activities  with  users  have  also  been  found  to  bring  users 
and  designers  closer  together  (Corry,  Frick,  & Hansen,  1997).  These  activities  have  the  added  benefit  of  being  able  to  be 
performed  early  in  the  design  process,  allowing  multiple  iterations  to  follow. 

Low-fidelity  or  paper  prototyping  is  a technique  that  involves  users  early  in  the  design  process  (Sugar  & Boling,  1995)  and 
has  been  shown  to  be  just  as  effective  as  prototyping  exercises  that  employ  a more  completed  electronic  version  (Virzi,  Sokolov, 
& Karos,  1996).  The  fact  that  paper  prototypes  of  a computer  system  interface  are  obviously  unfinished  allows  users  to  freely 
comment  and  contribute  their  ideas  for  improvement  to  designers  (Datz-Kauffold  & Henry,  2000). 

Testing  electronic  prototypes  or  even  an  fully  functioning  system  has  certain  advantages  over  low-fidelity  prototyping.  On- 
screen interactions  no  longer  need  to  be  simulated  and  colors,  resolution,  modes,  and  system  operating  speed  can  be  evaluated 
more  accurately.  Misanchuk,  Schwier,  & Boling  (2000)  describe  how  usability  testing  the  working  version  of  an  electronic  book 
on  instructional  multimedia  led  to  the  discovery  of  multiple,  desirable  features  that  were  missing. 

Often,  there  are  factors  affecting  usability  that  cannot  be  observed  in  a lab  or  test  environment.  Contextual  inquiry  (Beyer  & 
Holtzblatt,  1998)  attempts  to  overcome  this  problem  by  having  designers  observe  users  in  their  natural  work  environment  in 
order  to  fully  consider  the  many  variables  that  may  influence  how  a product  is  ultimately  used. 

Inviting  users  to  participate  on  the  actual  design  team  is  another  strategy  for  bringing  users  and  designers  close  together. 
Known  as  participatory  design,  this  approach  typically  has  designers  and  users  work  side  by  side  throughout  the  development 
cycle.  Benefits  of  participatory  design  have  been  shown  to  include  a sense  of  ownership  among  users  and  an  increased 
understanding  of  users  by  designers  (Williams  & Traynor,  1994).  Clement  & Van  den  Besselaar  (1993)  stress,  however,  that  for 
participatory  design  efforts  to  succeed,  users  must  be  allowed  to  take  an  independent  position  on  problems  and  they  must 
participate  in  the  process  of  decision  making. 

The  Current  State  of  Practices  and  Attitudes 

While  much  has  been  studied  and  written  regarding  usability  evaluation  methods  and  design  practices,  very  little  work  has 
been  done  in  determining  actual  practices  and  attitudes  of  those  in  industry.  Gould  & Lewis  (1985)  surveyed  447  software 
developers  attending  a human  factors  workshop  to  see  whether  they  identified  three  basic  principles  of  designing  for  usability  as 
a common  part  of  their  own  design  processes.  The  principles  included  an  early  and  continual  focus  on  users  and  tasks,  empirical 
measurement,  and  iterative  design.  The  results  revealed  that  developers  either  did  not  identify  with  the  three  principles  or  did  not 
understand  them  well  enough  to  implement  them  as  intended. 

In  another  survey  of  current  practices  at  the  time,  Dillon,  Sweeney,  & Maguire  (1993)  conducted  a survey  of  the  software 
industry  in  the  United  Kingdom,  gathering  data  on  four  themes:  respondents’  backgrounds,  their  interpretation  and  appreciation 
of  the  concept  of  usability,  current  practice  with  regard  to  usability  evaluation,  and  problems  and  requirements  for  support  in 
conducting  usability  evaluations.  The  authors  found  a widespread  awareness  of  usability  among  respondents,  but  what  seemed  to 
be  only  a superficial  application  of  Human  Factors  methods. 

Differences  in  attitudes  toward  usability  were  considered  in  a study  that  combined  survey  and  qualitative  interview  research 
of  three  Management  Information  System  (MIS)  managers  and  125  end-users  of  commercial  software  systems  (Morris  & Dillon, 
1996).  Interviews  with  the  managers  revealed  an  emphasis  on  costs  and  system  features  when  designing  or  selecting  new 
technologies  for  their  organizations.  This  was  in  significant  contrast  to  users’  main  concerns  of  contextual  and  environmental 
issues  that  affect  the  software’s  usability. 
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Methodology 

The  following  questions  were  considered  during  the  course  of  this  investigation:  What  are  software  developers’  attitudes 
toward  user-centered  design?  What  are  the  actual  methods  utilized  by  software  developers  who  report  using  UCD?  Does  there 
exist  any  correlation  between  the  practice  of  user-centered  methods  and  developers’  attitudes? 

Data  Collection 

Three  survey  forms  were  sent  to  each  of  500  software  companies  (1,500  forms).  The  companies  that  the  survey  forms  were 
sent  to  were  selected  from  the  Software  Publishers  Association  membership  directory.  The  survey  forms  were  comprised  of 
questions  concerning  the  respondents  job  classification  (type  of  software  designer),  type  of  training  (if  any)  in  usability,  history 
of  participation  in  usability  tests,  types  of  usability  procedures  utilized  by  respondents,  size  of  the  company  the  respondent 
worked  for,  and  attitudes  concerning  usability  testing.  Most  of  the  questions  required  respondents  to  check  boxes  indicating  the 
appropriate  answer,  except  for  the  attitude  questions  which  contained  22  question  items  measuring  the  subjects’  response  to  a 
given  statement.  These  items  were  measured  on  a five-point  Likert  scale  from  strong  disagreement  to  strong  agreement. 

Subjects 

Eighty -three  software  developers  responded  to  our  survey.  This  was  an  effective  return  rate  of  5.5  percent. 

Results  of  the  Study 

Descriptive  Analysis  for  Software  Developers’  Current  Practices  of  UCD 

(1)  Respondents’  Position  as  a Software  Developer 

As  shown  in  Table  1,  the  majority  of  respondents  worked  in  commercial  applications,  instructional  software,  and  entertainment 
software.  Software  developers  who  belonged  to  the  above  three  positions  accounted  for  86  percent  of  the  respondents. 


Table  L Software  developers ' Position 


Position 

Frequency 

Commercial  applications 

33 

Instructional  software 

24 

Entertainment  software 

14 

Other 

9 

Technical/programming  language 

1 

Instructional  and  Entertainment 

1 

Online  and  Commercial 

1 

Operating  systems 

0 

Online  documentation/help 

0 

Total 

83 

(2)  Companies  that  Participated  in  the  Survey 

Among  the  500  companies  to  which  the  mail  surveys  were  sent  out,  results  were  returned  from  56  different  companies.  Most 
of  the  companies  participated  in  the  survey  had  one  person  respond;  whereas  17  companies  surveyed  had  2-3  software  developers 
respond. 

(3)  Number  of  Employees  in  the  Company 

More  than  half  of  the  respondents  (55%)  worked  in  companies  with  1-50  employees,  and  about  a quarter  of  the  respondents 
worked  in  companies  with  51-250  employees.  The  remaining  18  percent  worked  in  companies  with  greater  than  251  employees. 
Thus,  most  of  the  respondents  (82%)  worked  in  smaller  companies,  with  less  than  250  employees. 

(4)  Number  of  People  Assigned  to  Software  Development  Team 

More  than  half  of  the  respondents  worked  in  core  development  teams  with  4-8  members,  and  about  one-third  of  the 
respondents  worked  on  teams  of  3 or  less.  Relatively  few  respondents  (10%)  worked  on  larger  teams  with  more  than  8 members. 

(5)  Company’  Exp  ectation  on  Usability  in  Their  Products 

Eighty-two  percent  of  those  surveyed  reported  that  their  company  always  expected  their  software  development  teams  to 
ensure  usability  in  their  products.  Seventeen  percent  indicated  that  their  companies  sometimes  had  such  expectations.  No 
respondent  indicated  that  such  an  expectation  did  not  exist  in  his  or  her  company. 

(6)  Use  of  Specific  Process  for  Incorporating  UCD  into  Product  Development 

Thirty-seven  percent  of  the  respondents  answered  that  their  companies  dways  used  UCD  processes  in  their  product 
development,  and  47  percent  responded  their  companies  used  UCD  processes  sometimes.  Meanwhile,  14  percent  indicated  their 
companies  never  had  such  a process  in  their  software  development.  Thus,  it  can  be  seen  that  the  majority  of  the  respondents 
reported  that  their  companies  used  specific  UCD  processes  for  software  development. 
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(7)  Experience  in  UCD  Practice  for  Software  Development 

Respondents  had  an  average  of  approximately  4 years  of  experience  in  UCD  practice  (M=  4,02,  SD=1 .70),  ranging  from  0 to 
over  7 years.  More  than  half  of  the  respondents  indicated  that  they  have  practiced  UCD  1 -7  years, 

(8)  Experiences  in  Attending  User  Test  Sessions 

For  the  question  asking  for  their  experiences  in  attending  user  test  sessions,  80  percent  of  respondents  indicated  that  they  had 
attended  at  least  more  than  one  user  test  session.  In  contrast,  20  percent  responded  that  they  had  never  attended  user  test 
sessions.  There  was  considerable  dispersion  in  the  reported  number  of  times  that  they  attended  user  test  sessions,  ranging  from  1 
to  1 00  test  sessions. 

(9)  Experiences  in  Helping  Conduct  User  Test  Session 

Sixty-four  percent  indicated  that  they  helped  conduct  a user  test  session  more  than  once,  while  36  percent  answered  they 
never  helped  conduct  user  test  sessions.  Those  who  had  conducted  user  test  sessions  themselves  reported  widely  dispersed 
occurrences,  ranging  between  1 and  200  times. 

(10)  Software  Development  Activities  Being  Practiced 

Respondents  were  asked  to  check  all  of  the  kinds  of  activities  in  which  their  companies  were  engaged  in  developing  their 
products.  These  activities  were  ranked  according  to  how  frequently  they  were  mentioned  by  the  respondents  in  Table  2.  “Using 
common  sense”  was  most  frequently  checked  (77),  while  “using  constructive  interaction  techniques”  was  least  frequently 
indicated  (18).  These  activities  were  coded  with  +,  0,  and  - by  the  investigators  (the  respondents  did  not  see  these  codes),  A “+** 
activity  means  that  the  user  is  actively  involved  in  the  software  development  process,  A activity  indicates  that  the  user  is  not 
involved  in  the  process.  A “0”  activity  means  a user  is  somewhat  involved  or  neutral.  The  median  split  from  Table  2 reveals  that 
the  majority  of  the  activities  above  the  median  do  not  involve  users  in  the  design  process  (7  out  of  9),  and  the  4 activities  in 
which  users  actively  participated  ranked  among  the  bottom  6 (below  the  median). 


Table  2.  Software  Development  Activities  Reported  to  Be  Practiced 


Code* 

Development  Activities 

Number  of  Responses 

- 

Using  common  sense 

77  (top  ranking) 

- 

Setting  major  goals 

67 

0 

Using  computer  prototypes 

65 

- 

Interviewing  representative  users  and  asking  whether  they  like  the  product 

63 

- 

Soliciting  feedback  from  “seed  sites”  or  “beta-testers” 

62 

- 

Performing  competitive  analyses  of  competing  products 

58 

0 

Testing  out  major  design  issues  with  users 

58 

- 

Following  GUI  guidelines 

58 

- 

Following  standard  interface  guidelines 

56 

- 

Passing  screen  shots  to  other  developers 

50  (median  ranking) 

0 

Using  paper  prototypes 

44 

- 

Following  company’s  interface  guidelines 

43 

- 

Expert  walkthroughs 

39 

+ 

Doing  field  studies/visits  of  user’s  work  environments 

38 

+ 

Having  a real  user  on  the  design  team 

29 

0 

Performing  task  analysis  of  user’s  tasks 

27 

0 

Recording  user’s  actions  with  a program 

25 

+ 

Using  think-aloud  protocols 

22 

+ 

Using  constructive  interaction  techniques 

18  (lowest  ranking) 

* + indicates  the  user  is  actively  involved  in  the  software  development  process, 

- indicates  the  user  is  not  involved  in  the  software  development  process. 

0 indicates  a user  is  somewhat  involved  or  neutral  to  the  development  process. 


(11)  Attending  Formal  Training  in  Usability  and  Sources  of  Training 

Less  than  one-third  of  the  respondents  indicated  that  they  had  any  formal  training  in  usability;  the  remaining  70  percent 
reported  not  having  attended  any  formal  training  in  usability.  When  asked  their  sources  for  obtaining  training  in  usability  - 
including  informal  types  of  training  - the  respondents  were  most  likely  to  get  their  training  from  books/joumals,  and 
conferences/workshops. 

(12)  Accessibility  to  a Usability  Specialist 

Sixty  percent  of  those  surveyed  answered  that  their  development  teams  did  not  have  access  to  usability  specialists.  On  the 
other  hand,  17  percent  had  access  to  usability  consultants,  and  17  percent  had  a specialist  on  their  team;  thus,  one-third  of 
respondents  indicated  having  access  to  usability  specialists. 

Software  Developers’  Attitudes  Towards  UCD 
(1)  Factor  Analysis 

A factor  analysis  of  the  22  Likert-scale  attitude  questions  was  conducted  using  the  image  factoring  method  with  varimax 
rotation,  resulting  in  seven  factors  as  shown  in  Table  3.  In  addition,  a reliability  analysis  on  each  factor  conducted 


(Cronbach’s  a for  internal  consistency).  The  Likert-scale  values  of  the  items  that  had  negative  loadings  on  a factor  were  reversed 
when  factor  scores  were  computed.  Results  of  the  reliability  analyses  ranged  from  .5952  to  .7784  (see  Table  3). 

Table  3.  Results  of  Factor  Analysis  and  Reliability  Analysis 


Factor 


Items 


Reliability 

coefficient 


1.  User-centered  design  (UCD) 
is  more  work  and  costs 
more. 


2.  User  test  sessions  are 
valuable. 

3.  Epiphany:  experience  of 
UCD  changed  my  mind. 


4.  Learned  little  from  user  tests 
and  UCD. 


5.  Many  tests  important, 
learned  a lot  about  my 
product. 

6.  UCD  is  worth  the  effort  and 
cost. 


7.  Usability  specialists  are 
helpful  in  improving 
product. 


16.  My  team’s  UCD  activities  tend  to  lengthen  development  time  for  a = .7784 

our  product.  (n  = 74) 

5.  The  UCD  activities  that  I have  participated  in  did  not  generally  add 
time  to  product  development  cycle.* 

13.  UCD  is  more  expensive  than  traditional  product  development. 

1 . UCD  activities  make  extra  work  for  me  as  a developer. 

14.  Participating  in  user  test  sessions  is  a positive  experience.  a = .7554 

21 . Overall,  I do  not  enjoy  participating  in  user  test  sessions.*  (n  = 74) 

18.  I usually  have  confidence  in  the  results  of  user  test  sessions. 

2.  Users  in  the  test  lab  behave  just  the  way  I expected  them  to  before  I a = .6438 

started  attending  user  test  sessions.*  (n  = 59) 

15.  Once  I became  involved  with  UCD  activities,  I changed  my  mind 
about  what  UCD  is. 

17.  After  the  first  user  test  session  I observed,  I found  that  I had  an 
altered  view  on  my  users. 

3.  User  test  sessions  usually  do  not  give  me  new  insights  about  my  a = .5952 

program.  (n  = 67) 

1 1.  Participating  in  UCD  activities  had  little  effect  on  my 
understanding  of  this  discipline. 

10.  It  is  important  to  conduct  user  test  sessions  many  times  throughout  a = .605 

product  development.  (n  = 74) 

12. 1 usually  learn  a lot  about  my  product  as  a result  of  user  test 
sessions. 

8.  In  general,  I would  not  recommend  that  other  development  teams  a = .574 

spend  effort  on  UCD  activities.*  (n  = 77) 

6.  In  my  opinion,  UCD  activities  are  worth  the  effort. 

7.  The  expenses  incurred  by  UCD  activities  are  offset  by  savings 
elsewhere  in  the  development  process  or  life-cycle  of  the  product. 

9.  In  my  development  work,  I find  that  the  extra  time  it  takes  for  UCD 
activities  does  not  frequently  enhance  my  products.* 

19.  Most  usability  specialists  are  primarily  interested  in  improving  the  a = .641 

overall  quality  of  my  program.  (n  = 57) 

4.  Usability  specialists  do  not  do  much  except  point  out  the  “mistakes” 

of  my  programs.* 


*These  items  negatively  loaded  on  a factor,  and  were  reverse-coded  when  computing  scale  scores  for  each  factor. 


(2)  General  Attitude 

Results  of  a descriptive  analysis  of  the  Likert-scale  attitude  questions  suggest  that  the  respondents  had  moderately  positive 
attitude  towards  UCD  activities  (l=strongly  disagree,  2=disagree,  3=undecided,  4=agree,  5=strongly  agree).  As  seen  in  Table  4, 
the  respondents  tend  to  strongly  agree  that  UCD  is  worth  the  effort  and  cost  (M=  4.38,  SD=  .48).  They  also  tend  to  strongly 
agree  that  it  is  important  to  conduct  many  user  test  sessions  and  that  they  learned  a lot  about  their  products  from  user  test  sessions 
(M=  4.1 1,  SD=  .68).  Likewise,  respondents  tend  to  disagree  that  they  learned  little  from  user  test  sessions  and  UCD  (M=  1.77, 
SD=  .66).  The  overall  mean  was  3.79,  which  suggests  moderately  positive  attitude  towards  usability. 


Table  4.  Descriptive  Statistics  for  the  UCD  Attitude  Factors 


Factor 

N 

Minimum 

Maximum 

Mean 

SD 

1.  User-centered  design  (UCD)  is  more 

74 

1.25 

5.00 

3.15 

.94 

work  and  costs  more. 

2.  User  test  sessions  are  valuable. 

74 

2.00 

5.00 

3.90 

.61 

3.Eiphany:  experience  of  UCD  changed 

59 

2.00 

5.00 

3.51 

.69 

my  mind. 

4.  Learned  little  from  user  tests  and 

67 

1.00 

4.50 

1.77 

.66 

UCD. 

5.  Many  tests  important,  learned  a lot 

74 

2.00 

5.00 

4.11 

.68 

about  my  product. 

6.  UCD  is  worth  the  effort  and  cost. 

77 

3.00 

5.00 

4.38 

.48 

i453 


7.  Usability  specialists  are  helpful  in 
improving  product. 

Overall  attitude 


57  2.00 


5.00 


3.95 


.78 


45  3.05  4.91  3.79  .37 


(3)  UCD  Practices  and  Attitudes 

Point-biserial  correlations  were  done  between  the  user-active  UCD  methods  (presented  in  Table  2 with  the  + codes)  and  the  7 
components  of  user  attitudes  toward  UCD  (presented  in  Tables  3 and  4).  None  of  these  correlations  was  significant  at  the  0.05 
level.  A new  variable,  “good  UCD  practices,”  was  then  constructed  by  totaling  the  number  of  items  checked  from  the  following 
practices:  1)  testing  out  major  design  issues  with  users,  2)  doing  field  studies/visits  of  users’  work  aivironment,  3)  using  paper 
prototypes,  4)  using  computer  prototypes,  5)  using  think -aloud  protocols,  and  6)  recording  users’  actions  with  a program.  A 
significant  positive  correlation  was  found  between  the  number  of  “good  UCD  practices”  and  the  software  developer  attitude  that 
“UCD  is  worth  the  effort  and  cost”  (r  = .235,  p < .05).  However,  none  of  the  other  six  UCD  attitudes  was  significantly  correlated 
with  the  number  of  “good  UCD  practices,”  nor  was  the  overall  attitude  towards  UCD  correlated  with  the  number  of  “good  UCD 
practices.” 

(4)  Developers’  Background  and  their  Attitudes 

1 . Position  and  Company  Size 

An  analysis  of  variance  (ANOVA)  was  conducted  to  see  if  there  were  any  differences  in  UCD  attitudes  according  to  software 
developer  position.  Results  showed  that  instructional  software  developers  more  strongly  agreed  that  “user  test  sessions  are 
valuable”  than  commercial  applications  software  developers  (F  = 6.874,  p < .002).  Furthermore,  instructional  software 
developers  tended  to  more  strongly  disagree  that  they  learned  little  from  user  test  sessions  and  UCD  than  did  entertainment 
software  developers  (F  = 3.618,  p < .033).  However,  no  significant  difference  was  found  among  the  software  developer  positions 
and  their  overall  attitude  towards  usability  (F  = 2.471,  p = .098).  Also,  there  was  no  difference  in  attitudes  towards  UCD 
according  to  size  of  company  (F  = 1 . 1 64,  p = .34 1 ). 

2.  Formal  Training  in  Usability 

Results  of  an  ANOVA  showed  that  there  was  no  significant  difference  in  the  respondents’  attitudes  towards  usability 
between  the  group  who  have  had  any  formal  training  in  usability  and  those  who  have  not  received  any  formal  training  in  usability 
(F=.213,p=  .647). 


3.  Experiences  in  User  Test  Sessions 

Results  of  an  ANOVA  revealed  that  those  who  have  attended  user  test  sessions  tend  to  more  strongly  agree  that  “user  test 
sessions  are  valuable”  (F  = 3.934,  p=  .051),  and  that  “it  is  important  to  conduct  many  user  test  sessions  and  they  have  learned  a 
lot  about  their  products  from  the  user  test  sessions”  (F  = 3.337,  p = .072).  However,  those  who  have  attended  user  test  sessions 
also  tend  to  more  strongly  agree  that  UCD  is  “more  work  and  costs  more”  than  conventional  development  activities  (F  = 4.634,  p 
= .035).  In  addition,  those  who  have  e><perience  in  helping  conduct  the  user  test  session  also  tend  to  more  strongly  agree  that 
“UCD  is  worth  the  effort  and  cost”  (F  = 5.555,  p = .021)  than  those  who  have  no  such  experience. 

4.  Access  to  Usability  Specialists  and  the  Developer’s  “Good  UCD  Practices 

An  ANOVA  was  conducted  between  the  group  who  had  a usability  specialist  either  in  their  team  or  as  a consultant  and  those 
who  had  no  access  to  a usability  specialist  in  order  to  compare  the  number  of  “good  UCD  practices”  between  those  two  groups. 
The  results  showed  that  significantly  more  good  UCD  practices  were  reported  by  software  developers  who  worked  on  teams  that 
either  hired  usability  consultants  or  had  a usability  specialist  on  their  teams  compared  with  those  who  had  no  usability  specialists 
at  all  (F  = 10.047,  p = .002).  The  means  for  each  group  was  3.9  for  the  group  who  had  access  to  usability  specialists,  and  2.7  for 
the  group  who  had  no  such  access,  respectively. 

Discussion 

Results  suggested  that  while  the  respondents  considered  the  UCD  process  more  work  and  additional  cost,  they  viewed  UCD 

as  a positive,  worthwhile  practice.  The  software  developers  who  attended  user  test  sessions  and  helped  conduct  sessions  reported 
more  positive  attitudes  than  others.  This  would  suggest  that  active  participation  in  usability  tests  may  be  a factor  in  developers’ 
positive  outlook  concerning  usability  tests. 

We  were  intrigued,  however,  with  the  apparent  lack  of  findings  in  many  of  the  areas  for  which  we  performed  analyses.  For 
example,  we  had  expected  attitude  differences  between  those  who  have  had  formal  training  in  UCD  and  those  who  have  not.  No 
difference  in  attitude  was  found. 

Most  intriguing,  however,  was  the  software  developers’  apparent  lack  of  use  “good  UCD  practices.”  That  is,  while  the 
respondents  reported  UCD  as  a positive  and  beneficial  practice,  this  was  not  correlated  with  reported  numbers  of  good  UCD 
practices. 

Perhaps  most  important  is  having  a usability  specialist  either  as  a consultant  or  as  a team  member,  since  this  is  associated 
with  greater  numbers  of  good  UCD  practices.  Given  that  only  30  percent  of  software  developers  have  received  any  formal 
training  in  usability  — and  when  they  did,  it  was  most  likely  from  books  and  journals  — it  appears  worthwhile  to  have  usability 
specialists  on  the  development  team. 

Finally,  this  study  is  limited  by  the  generalizability  of  its  findings.  We  do  not  know  if  those  who  responded  to  the  survey  are 
representative  of  software  developers  in  general.  Also,  our  data  were  collected  in  1994-96,  and  it  is  possible  that  software 
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developers’  attitudes  and  their  company’s  UCD  practices  may  have  changed  since  then.  However,  our  results  seem  to  be 
consistent,  in  part,  with  a recent  survey  of  HCI  professionals  in  North  American  industry  in  which  respondents  were  asked  to 
identify  what  organizational  approaches  and  usability  methodologies  they  perceived  to  be  most  effective  in  having  a strategic 
impact  on  corporate  decision-making  (Rosenbaum,  Rohn,  & Humburg,  2000).  The  size  and  type  of  company  along  with  the  size 
of  the  usability  group  within  it  were  all  considered,  but  no  statistically  significant  relationships  proved  to  exist  between  the 
demographic  data  and  the  organizational  approaches  and  usability  methods  employed.  In  our  study , we  found  no  relationship 
between  software  developers’  attitudes  toward  user-centered  design  and  good  UCD  practices. 
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Abstract 

The  rapid  increase  in  the  use  of  computer  technology  to  facilitate  alternative  forms  of  educational  delivery,  often  called 
distance  education  (DE),  represents  a major  change  taking  place  in  education.  The  potential  for  decoupling  the  traditional 
requirements  that  student  and  teacher  be  present  in  the  same  room  at  the  same  time  has  never  been  greater. 

Many  teachers  will  soon  decide  to  design,  or  be  asked  to  design,  courses  for  distance  education  in  virtual  environments  for 
the  first  time,  but  they  will  not  have  many  published  principles  to  guide  them.  This  research  project,  which  represents  portions  of 
my  doctoral  dissertation,  describes  an  instrumental  qualitative  case  study  examining  the  process  that  an  educator  must  resolve 
when  designing  a distance  education  course  from  a preexisting  traditional  course.  My  research  focused  on  the  transformation  of 
a widely- taken,  lecture-based  course  to  a largely  asynchronous,  web-based  course. 

During  this  study,  the  principal  guiding  question  was:  What  aspects  of  content,  design,  and  andragogy  would  an  instructor 
consider  when  transforming  a lecture-based  course  to  one  incorporating  web-based  instruction?  In  addition  to  this  overarching 
question,  a number  of  related  questions  regarding  the  selection  of  team  experts  and  devising  suitable  methods  for  regulating 
student — student  and  teacher — student  interactions  were  considered. 

A number  of  key  themes  and  issues  manifested  themselves  during  the  course  transformation  process,  primarily  those  concerned 
with  unexpected  problems  and  technical  difficulties,  and  those  focused  on  the  pedagogical  novelty  involved  with  distance 
education. 

The  course  displayed  far  fewer  technical  problems  than  anyone  might  have  imagined.  However,  issues  regarding  teacher 
evaluation  and  the  possible  consequences  for  tenure  evaluation  were  totally  unexpected  by  the  principals  involved.  The  issues 
surrounding  the  newness  of  distance  education,  for  both  student  and  instructor,  demonstrated  enormous  complexity.  Pedagogical 
matters  on  how  time  and  space  affected  the  course,  on  what  "interacting''  means  in  a distance  education  environment,  and  the 
new  varieties  of  limitations  and  constraints  imposed  by  distance  education  must  be  considered  by  future  dstance  education 
instructors. 

Background 

As  we  approach  the  new  millennium,  all  objective  signs  point  to  the  fact  that  education,  as  well  as  a number  of  other  social 
institutions,  appear  to  be  undergoing  radical  transformation.  These  changes  may  involve  all  levels  of  the  educational  system  from 
pre-kindergarten  through  higher  education.  Many  believe  that  these  upcoming  changes  will  inaugurate  the  greatest  systematic 
modifications  since  the  Middle  Ages,  when  universities  shifted  fundamentally  from  locations  that  congregated  assemblies  of 
great  scholars  and  thinkers  to  those  seeking  to  become  great  repositories  of  the  latest  technological  education  innovation:  the 
printing  press  and  books  (Nyiri,  1997). 

One  major  change  taking  place  in  education  is  the  rapid  introduction  of  computer-assisted  educational  delivery,  often  called 
distance  education  (DE).  Other  terms  for  distance  education,  such  as  “open  learning,”  “distributed  learning,”  “web-based 
instruction,”  and  “flexible  learning,”  have  also  been  used,  and  these  terms  often  compete  and  substitute  for  one  another 
indiscriminately.  Many  of  these  terms  show  subtle  distinctions  that  can  best  be  discerned  by  considering  the  environment  and  the 
frame  of  reference  used  by  the  institution  offering  the  instruction. 

Regardless  of  the  particular  variety,  distance  education  platforms  all  share  a number  of  characteristics  in  common  (Peters, 
1993):  (a)  teachers  and  students  are  apart;  (b)  the  learning  often  takes  place  in  the  home  of  the  student;  (c)  the  teaching-learning 
process  often  takes  the  form,  or  uses  components  of,  independent  study;  (d)  students  do  not  have  to  cease  working  or  interrupt 
their  schedule  while  taking  the  course. 

Our  contemporary  variations  of  distance  education  incorporate  the  newest  technologies  of  computer  networks  and  modem 
telecommunications.  With  the  advent  of  these  new  advancements,  the  potential  for  decoupling  the  traditional  requirements  that 
student  and  teacher  be  present  in  the  same  room  at  the  same  time  has  never  been  greater.  From  an  operational  point  of  view, 
distance  education  can  be  thought  of  as  a set  of  variations  in  the  educational  processes  and  technologies  that  take  place  without 
these  conventional  requirements. 

Today  three  different  types  of  technologies  tend  to  dominate  in  distance  education:  videoconferencing,  interactive 
broadcasting,  and  online  formats.  Videoconferencing  and  broadcasting  generally  require  that  the  participants  meet  at  the  same 
time  although  the  location  of  the  participants  may  vary  considerably — from  different  sites  on  the  same  campus  to  different  cities, 
states,  or  even  countries.  This  variety  is  called  synchronous  distance  education  (SDE).  In  the  latter  technology,  called 
asynchronous  distance  education  (ADE),  the  participants  are  constrained  by  neither  time  nor  location.  The  increasing  trend 
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among  distance  educators  involves  integrating  multiple  platforms  to  maximize  the  benefits  of  each  method  of  delivery.  This 
study  followed  a team  as  they  developed  a largely  ADE  course  for  Internet  delivery. 

While  distance  education  has  existed  in  its  many  forms  for  more  than  a century,  it  has  never  played  a major  role  until  recently 
(Barley,  1999;  Martin,  1999).  This  may  soon  change  since  the  ability  to  reach  students  previously  cut  off  from  the  traditional 
face-to-face,  lecture  style  of  education  has  excited  many  educators  and  politicians  alike. 

The  United  States  Department  of  Education  recently  began  tracking  the  availability  of  distance  education  courses  in  higher 
education  (Lewis,  Alexander,  & Farris,  1998;  Lewis,  Snow,  Farris,  Levin,  & Greene,  2000).  They  reported  that  for  the  1995  fall 
term  approximately  58%  of  2-year  public  higher  education  facilities  and  62%  of  4-year  institutions  offered  distance  education 
classes.  Overall  about  33%  of  institutions  offered  distance  education  courses.  Their  next  survey  covered  data  for  post  secondary 
education  institutions  for  the  1997-1998  school  year  (Lewis,  et  al,  2000).  Among  public  2-year  institutions,  now  62%  offered 
distance  education  courses,  while  public  4-year  institutions  offering  distance  education  courses  rose  to  78%.  As  in  the  previous 
survey,  private  institutions  lagged  behind  public  institutions,  with  only  5%  of  2-ycar  private  schools  and  19%  of  4-year  programs 
offering  distance  education  courses.  The  trend  for  increasing  use  of  distance  education  in  the  near  term  is  clearly  escalating. 

Private  institutions  have  reacted  differently  from  public  ones.  In  2year  private  institutions  only  2%  offered  distance 
education  courses,  and  only  12%  of  4- year  institutions  offered  distance  education  courses.  Many  of  these  elite  campuses  have 
taken  a more  selective  approach  and  focused  on  specialized  degree  programs  and  curricula  that  can  be  exported  to  international 
audiences  without  interfering  their  local,  on-campus  strategies  (Blumenstyk,  1 997).  As  the  United  States  currently  undergoes  its 
massive  build-up  in  the  race  to  embrace  distance  education  technologies  within  its  educational  systems,  questions  regarding  the 
effectiveness  and  best  practices  of  distance  education  remain  unanswered  and  often  even  unasked. 

While  many  different  forms  of  distance  education  varieties  are  being  tried  across  the  country,  using  the  Internet  as  a delivery 
vehicle  has  become  increasingly  popular.  In  the  United  States,  college  courses  have  increased  their  use  of  e-mail  from  8%  in 

1994  to  44%  in  1998,  and  the  use  of  other  Internet  resources  and  World  Wide  Web  (WWW)  pages  for  class  materials  also  show 
dramatic  increases  (Institute  for  Higher  Education,  1999).  In  1998,  the  United  States  Department  of  Education  found  that  1,680 
institutions  offered  54,000  distance  education  courses,  with  1.6  million  students  enrolled  (Camevale,  2000).  This  growth  is  all  the 
more  amazing  when  considering  that,  except  for  a small  number  of  experimental  courses,  the  number  of  such  courses  offered  in 

1995  approached  zero.  A probable  reason  for  this  may  have  been  the  fact  that  browsers  and  other  software  tools  only  existed  in 
primitive  forms  until  recently. 

The  true  number  of  distance  education  courses  offered  is  presently  unknown,  however.  Criteria  have  yet  to  be  established  on 
what  should  be  considered  a distance  education  course.  For  example,  in  some  colleges,  courses  simply  having  a syllabus 
available  on-line  would  qualify  as  a distance  education  course.  In  the  very  near  future,  some  method  of  assigning  an  approximate 
percentage  of  the  course  that  takes  place  via  distance  education  will  have  to  be  devised. 

Many  teachers  will  be  designing  courses  for  distance  education  environments  for  the  first  time,  and  they  will  be  doing  so  for 
a variety  of  reasons.  Some  will  offer  their  courses  in  this  manner  because  it  appeals  to  them  out  of  curiosity  or  because  they  are 
personally  interested  in  the  technological  innovations  and  want  to  experience  them  first  hand.  Others  believe  the  new 
technologies  may  improve  or  enhance  their  current  courses  by  offering  students  new  ways  of  looking  at,  or  contemplating,  a 
problem,  or  a new  way  of  thinking.  Many  will  be  offering  their  courses  for  less  than  optimal  reasons.  Administrators,  desiring  to 
reach  as  many  students  as  possible,  might  pressure  them  to  take  their  courses  online.  To  accomplish  this,  they  may  want  to  add 
some  form  of  distance  education  to  supplement  their  traditional  student  base.  Some  administrators  will  want  their  institutions  to 
be  seen  as  leaders  of  the  field  or  simply  may  not  want  to  be  seen  as  laggards,  out-done  by  their  competitors  (Martin,  1999). 
Whether  these  teachers  will  be  transforming  their  existing,  traditional  courses  or  designing  new  courses  de  integro.  they  often 
will  be  uncertain  as  how  to  proceed. 

To  many  of  the  “early  adopter”  educators,  those  first  to  undertake  distance  education  in  their  classes,  their  efforts  often  focus 
on  transferring  as  much  as  possible  from  what  they  have  already  prepared  for  their  traditional  classes  without  consciously 
adapting  their  materials  to  these  new  media.  While  this  may  be  an  obvious  first  approach,  it  cannot  be  as  effective  as  when 
instructors  completely  rethink  and  reevaluate  the  advantages  and  disadvantages  of  the  new  medium.  The  most  effective  distance 
education  educators  will  focus  on  which  specific  distance  education  options  are  the  most  appropriate  for  their  given  curricula. 

As  part  of  my  case  study,  I met  and  interviewed  Dr.  Gwendolyn  and  other  principals  throughout  the  course  development 
process.  I observed  how  she  approached  the  transformation  process  and  how  she  evaluated  which  changes  needed  to  take  place  in 
the  newly  transformed  course.  I observed  how  and  why  decisions  were  made  regarding  course  content,  student -faculty 
interactions,  and  student-student  interactions.  I also  followed  how  she  interacted  with  the  instructional  technologists  who  helped 
her  give  shape  to  the  on-line  course  in  the  form  of  text,  audio,  graphics,  animation,  and  video  media.  I was  given  access  to  all 
content  meetings  and  was  allowed  to  observe  the  group  decision-making  process  and  how  they  delegated  tasks  and  made 
assignments. 

Research  Overview 

A department  within  the  business  school  of  a large  southern,  urban  university  began  their  experimentation  of  offering  web- 
based  instruction  by  granting  Dr.  Wolf  release  time  to  develop  an  online  course.  The  department  heads  selected  a graduate 
gateway  course  for  the  experiment,  i.e.,  an  introductory  class  required  of  all  incoming  graduate  students  within  that  department. 
The  course  also  acts  as  a pre-requisite  for  many  other  graduate  courses  within  the  entire  business  school. 
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Cast  of  Characters 

The  primary  members  in  this  research  effort  included:  Dr.  Wolf,  Dr.  Gwendolyn,  Dr.  Summerville,  Mr.  Masterson,  and  Ms. 
Veritago.  Each  person  brought  unique  skills  to  the  process  as  tabulated  below.  Dr.  Wolf  assumed  the  chief  responsibility  for  the 
course  content  and  would  teach  the  class  when  completed.  Dr.  Gwendolyn  acted  as  team  coordinator  for  both  phases  of  the 
project  as  well  as  the  instructional  designer  for  the  course.  Dr.  Summerville  contributed  greatly  as  a subject  matter  expert.  Dr. 
Wolf,  Dr.  Gwendolyn,  and  Dr.  Summerville  acted  as  the  principal  content  team  during  the  development  phase  of  the  project.  Mr. 
Masterson  and  Ms.  Veritago  Joined  Drs.  Gwendolyn  and  Wolf  in  the  second  phase  of  the  project  and  contributed  largely  as  web- 
page designers  and  to  web  site  maintenance. 

The  Transformation  Process 

In  the  first  phase  of  the  process,  the  team  examined  the  course  material  for  two  primary  considerations:  the  relevance  of  the 
subject  matter  for  the  current  demands  of  the  field  and  the  suitability  of  the  course  material  for  use  in  a web-delivered 
environment.  Drs.  Wolf,  Gwendolyn,  and  Summerville  presided  over  this  “transformation  phase”  of  the  project. 

Administration 

According  to  Dr.  Wolf,  his  department  cited  several  reasons  for  wanting  to  experiment  with  distance  education  within  their 
programs.  While  the  business  college  boasts  an  excellent  reputation  and  usually  attracts  more  student  applications  than  available 
open  slots,  some  problems  were  beginning  to  emerge.  The  reasons  for  undertaking  the  course  transformation  are  summarized  in 
the  table  below. 

Table  1 Summary  of  Dr.  Wolfs  Department 's  Reasons  for  Developing  DE  Courses 


Stakeholders 

Reasons  for  participating  in  DE  Courses 

Students 

None  directly  solicited 

Faculty 

Increasing  travel  demands 
Potential  for  improving  courses 

Administration 

Perceived  demand  by  students 
Increase  the  number  of  students  per  class 

Perceived  competition  by  other  college  programs  offering  DE  courses 
Reduce  time  and  space  constraints  for  students  attending  college  courses 
Reduce  traffic  and  parking  congestion  on  campus 

Students 

In  preparation  for  overseeing  the  transformation  of  the  course.  Dr.  Gwendolyn  formulated  a 12-step  program  for 
implementing  the  transformation  of  the  course  from  a lecture-based  format  to  an  Internet -based  format.  She  relied  heavily  on  her 
TWIGS  (tools  for  web-based  instruction:  generating  structures)  concepts,  generated  from  her  previous  research  and  experience 
designing  distance  courses,  to  guide  Drs.  Wolf  and  Summerville  in  reviewing  the  course  materials  for  suitability  in  the  new 
educational  medium. 

The  TWIGS  document  guided  the  instructors  to  design  their  distance  education  course  to  the  needs  and  requirements  of  the 
students  by  posing  a series  of  questions  in  a number  of  different  areas.  One  of  the  difficulties  with  the  course  in  question  related 
to  the  very  different  technological  skills  that  students  bring  to  the  course:  some  are  technological  neophytes  and  others  are 
technological  experts.  In  general  the  students  were  pursuing  their  MBAs,  in  their  twenties,  worked  full-time,  and  had  several 
years  of  “real  world”  business  experience.  Some  of  the  students  would  be  of  foreign  birth  and  not  speak  English  as  their  native 
language. 

Instructors 

For  approximately  five  months,  Drs.  Wolf  and  Gwendolyn  accepted  the  challenge  of  designing  a web-based  course  based  on 
a course  that  had  been  traditionally  taught  in  a face-to-face  manner.  Drs.  Wolf  and  Summerville,  acting  in  their  roles  of  subject 
matter  experts,  carefully  reviewed  every  topic  in  past  course  syllabi  to  determine  if  the  material  continued  to  be  necessary  and 
vital  for  students  in  the  field  to  understand  and  master.  They  worked  with  Dr.  Summerville  to  review  the  content  of  the  course 
and  make  certain  each  element  of  the  syllabus  remained  important  and  relevant  given  the  rapid  changes  that  are  taking  place  in 
real-world  business  environments.  Under  Dr.  Gwendolyn's  guidance,  the  team  reviewed  the  course  syllabus;  considered  under 
what  parameters  the  students  would  access  the  Internet  and  how  that  might  affect  their  course  design;  discussed  the  number  and 
types  of  tests  that  would  be  used;  and  reviewed  which  communication  features — synchronous,  asynchronous,  or  both — the  course 
would  incorporate. 

The  principals  agreed  to  incorporate  both  synchronous  and  asynchronous  components  of  distance  education.  Dr.  Gwendolyn 
pointed  out  that  too  much  time  devoted  to  SDE  would  take  away  many  of  the  advantages  of  distance  education,  in  general,  by 
tethering  the  students  to  their  computers  at  certain  times.  This  was  especially  relevant  to  the  given  student  population  since  so 
many  of  them  worked  fill  time. 


In  the  process  the  course  material  underwent  drastic  changes.  During  this  process,  they  did  not  take  into  account  the 
suitability  of  the  material  for  distance  education  delivery,  and  it  fell  upon  Dr.  Gwendolyn  to  accommodate  as  much  of  the  content 
and  style  of  presentation  as  possible  and  to  let  them  know  if  she  believed  changes  needed  to  be  made  to  suit  the  medium. 

Design  Process 

During  the  last  stage  of  the  course  development,  what  I call  the  Design  Process,  the  content  of  the  course  had  already  been 
determined,  and  the  team  now  focused  on  how  what  type  of  course  management  software  would  be  used  and  the  graphic  design 
and  style  of  the  web  pages.  Dr.  Gwendolyn  proposed  three  possible  options  for  web  course  management  software;  use  a pre- 
existing, off-the-shelf  product  such  as  WebCT  or  Blackboard.com,  design  a proprietary,  custom-made  software,  or  a hybrid 
solution  of  modifying  the  commercially  available  software  to  best  suit  Dr.  Wolfs  needs.  Each  option  offered  strengths  and 
weaknesses.  The  commercial  software  would  be  the  easiest  to  use,  but  it  would  also  be  the  most  rigid,  forcing  any  instructor  to 
utilize  preformed  templates.  The  proprietary  software  would  allow  instructors  nearly  unlimited  freedom  in  how  their  course 
materials  would  be  presented  and  how  students  would  interact  with  the  material,  but  it  would  be  the  most  difficult  to  design, 
would  be  difficult  to  maintain,  and  would  not  allow  other  faculty  members  within  the  department  an  easy  template  to  copy  for 
their  courses.  Dr.  Wolf  decided  to  employ  the  hybrid  solution  as  providing  some  flexibility  but  still  allowing  those  within  the 
department  some  guiding  design  prospects. 

Dr.  Gwendolyn  had  three  main  goals  for  her  design  principles  for  the  course:  (a)  accommodate  the  instructor’s  style  of 
teaching  as  much  as  possible;  (b)  provide  a relatively  easy  to  use  model  for  other  instructor  in  Dr.  Wolfs  department  to  emulate 
should  they  decide  to  transform  classes  of  their  own;  (c)  adhere  to  standard  graphic  design  principles  to  maximize  ease  of  reading 
of  text  and  viewing  of  graphics. 

Mr.  Masterson  and  Ms.  Veritago  reviewed  Dr.  Wolfs  class  notes  and  PowerPoint  slides  and  redesigned  them  for  consistency 
of  appearance  and  legibility.  These  style  guides  would  apply  to  all  documents  on  the  web  site.  Dr.  Wolf  was  pleased  with  the 
results. 

The  design  team  incorporated  several  features  within  the  web  page  that  Dr.  Wolf  found  useful  for  this  course  and  for  possible 
improvements  for  future  courses.  They  used  time-sensitive  coding  so  that  answers  to  assignments  and  exercises  only  appeared 
after  a given  date.  “Page  tracking”  offered  the  possibility  of  tracking  which  web  pages  were  utilized  by  students  the  most.  This 
feature  offered  an  indirect  measure  of  which  parts  of  the  web  site  the  students  found  the  most  useful. 

Attempts  to  have  outside  evaluators  review  the  web  site  did  not  produce  as  much  information  as  hoped  for.  Dr.  Summerville’s 
face-to-face  class  looked  at  the  course  prototype  but  only  provided  a small  number  of  comments.  Other  instructors  in  other 
institutions  had  agreed  to  review  the  website  but  because  of  time  conflicts  only  provided  very  basic  comments. 

Discussion 

As  long  as  I’ve  got  an  IP  connection,  I’ve  got  a classroom. 

— Dr.  Summerville 

This  stuff  will  eat  you  alive — ^if  you  let  it. 

— Dr.  Gwendolyn 

A number  of  key  themes  and  issues  manifested  themselves  during  the  course  migration  process,  and  I have  grouped  them  into 
two  categories:  those  concerned  with  unexpected  problems  and  technical  difficulties,  and  those  focused  on  the  pedagogical 
novelty  involved  with  distance  education. 

Technical  Difficulties 

The  course  displayed  far  fewer  technical  problems  than  either  Dr.  Gwendolyn  or  Dr.  Wolf  imagned — they  both  expected  far 
worse  and  doubted  other  instructors  would  be  so  fortunate.  The  one  major  problem  of  students  being  kicked  off  the  chat  room  site 
when  another  student  entered  continued  throughout  the  term  was  never  resolved.  For  the  next  term.  Dr.  Wolf  ported  the  course 
into  WebCT,  and  the  problem  never  recurred. 

Instructor  Evaluations  and  Tenure 

As  with  many  other  universities.  Dr.  Wolfs  department  uses  student  evaluation  of  instructors  as  part  of  their  protocol  for 
determining  academic  tenure.  His  particular  department  makes  use  of  a series  of  questions  that  students  rank  on  a 5-point  scale, 
with  one  being  the  lowest  score  and  five  the  highest  score.  Dr.  Wolf  was  surprised  by  the  relatively  negative  student  evaluations 
he  received  at  the  end  of  the  course.  His  scores  averaged  0.5  points  lower  than  his  usual  scores.  While  many  of  these  comments 
might  be  attributed  to  the  fact  that  he  was  teaching  the  course  for  the  first  time,  it  is  equally  possible  that  many  of  his  low  scores 
might  be  attributed  to  the  fact  that  the  students  were  given  a traditional,  face-to-face  instructor  evaluation  form  to  fill  out.  The 
form  contained  a number  of  inappropriate  questions  that  might  have  confused  the  students  or  possibly  caused  them  to  give  less 
careful  consideration  to  the  process  than  they  might  have  under  different  circumstances.  For  example.  Question  4 asks  if  the 
instructor  “is  accessible  to  students  out  of  class,”  and  Question  6 asks  if  the  instructor  “speaks  in  a manner  that  is  easy  to 
understand.”  While  only  a handful  of  the  37  questions  appearing  on  the  evaluation  form  might  be  considered  totally  irrelevant, 
more  than  one-third  of  the  questions  have  either  little  relevance  to  courses  taught  at-a-distance  or  would  have  to  be  modified  or 
clarified  in  some  manner  to  better  accommodate  the  circumstances  involved  in  the  new  environment. 


Dr.  Gwendolyn’s  department,  at  her  university,  requires  that  she  use  traditional  evaluation  forms  for  her  online  classes,  but 
she  augments  that  data  by  asking  her  students  to  fill  out  an  evaluation  form  of  her  own  design.  This  simple  solution  might  have 
salvaged  a better  review  for  Dr.  Wolf  and  made  the  entire  experience  more  rewarding  for  him. 

As  a by-product  of  the  instructor  evaluation  disappointment.  Dr.  Wolf  raised  the  issue  of  tenure  evaluation.  As  an  assistant 
professor.  Dr.  Wolf  felt  he  must  keenly  focus  on  areas  related  to  tenure.  The  department  establishes  requirements  for  tenure, 
some  of  which  are  explicitly  recorded  and  some  that  are  more  nebulous  and  subjective.  Since  teaching  distance  education  courses 
generally  requires  more  effort  than  traditional  courses,  yet  does  not  generate  extra  rewards  for  that  effort.  Dr.  Wolf  decided  he  no 
longer  wanted  to  teach  them  at  this  stage  in  his  career.  He  felt  the  potential  for  lower  evaluations  could  not  justify  the  extra  effort 
involved.  Dr.  Wolf  thought  the  department  should  offer  bonuses,  in  the  form  of  release  time  or  other  similar  benefits,  to 
instructors  offering  to  teach  online. 

Intellectual  Property  Rights 

While  the  debate  over  who  owns  the  rights  to  distance  education  courses  rages  on  in  a number  of  universities  (Noble,  1998a), 
only  one  of  the  principals.  Dr.  Summerville,  voiced  any  concern  over  the  controversy.  Neither  Dr.  Wolf  nor  Dr.  Gwendolyn 
expressed  the  slightest  concern  over  the  matter,  but  I believe  the  subject  should  be  discussed  and  given  careful  consideration.  At 
the  present  time,  the  issue  may  not  be  that  important  since  most  of  the  instructors  involved  in  distance  education  pursue  distance 
education  for  their  own  interests  and  curiosities.  However,  as  the  push  towards  distance  education  continues,  and  universities 
become  more  insistent  that  their  instructors  offer  distance  education  courses,  the  matter  might  not  be  so  benign. 

Thus,  while  the  issues  of  intellectual  property  rights  and  ownership  have  not  played  a role  in  the  current  course 
transformation,  I believe  it  is  an  issue  of  extreme  importance.  Future  distance  education  instructors  may  wish  to  investigate  this 
matter  when  the  time  comes  for  their  initiation  into  teaching  at-a-distance. 

Redefining  Time  and  Space  Within  a Distance  Education  Environment 

As  stated  earlier,  distance  education  can  be  thought  of  as  a set  of  educational  policies  and  technologies  that  allow  both  the 
student  and  instructor  to  be  separated  in  time,  space,  or  both  time  and  space.  The  process  of  transforming  this  graduate  course 
involved  a number  of  stages,  and  many  of  them  related  to  issues  of  time  and  space,  either  directly  or  indirectly.  While  many 
might  think  the  boundary  between  time  and  space  as  quite  distinct,  the  two  dimensions  can  actually  be  considered  intertwined. 
For  example,  when  a student  complains  of  the  requirement  of  traveling  to  a local  college  campus  to  attend  a class,  seemingly  a 
“space”  demand,  part  of  the  complaint  actually  embeds  the  time  required  for  the  journey:  the  actual  travel  time,  time  spent 
searching  for  parking,  and  time  spent  walking  to  the  classroom  building.  Thus,  just  as  these  two  dimensions  are  mutually  linked 
in  the  concept  of  a space-time  continuum  in  high-level  physics,  these  two  attributes  appear  to  be  conjoined  in  the  everyday  world 
as  well. 

While  the  official  reasons  for  wanting  to  transform  courses  were  largely  administrative  and  financial,  students  and  instructors 
generally  hold  other  motivations  for  being  involved  with  distance  education,  particularly  those  associated  with  convenience  in 
space-time.  As  mentioned  above,  older  students,  or  those  married  with  families,  often  need  flexibility  while  scheduling  classes.  In 
fact,  without  the  option  of  controlling  their  course  scheduling  requirements,  they  may  not  be  able  to  take  part  in  degree  programs. 
But  the  issues  of  space-time  relating  to  distance  education  go  beyond  simply  whether  a student  can  attend  a class  at  a given  time. 
In  addition  to  simply  “showing  up,”  students  have  always  had  to  complete  homework,  exercises,  and  class  projects;  participate  in 
class  discussions;  and  communicate  with  the  instructor  and  with  other  students.  Most  (if  not  all)  of  these  items  will  still  have  to 
be  successfully  navigated  in  a distance  education  environment  and  will  be  complicated  by  the  constraints  of  the  new  medium. 

Instructors  may  have  an  even  more  demanding  reliance  and  dependence  on  space-time  issues  than  do  the  students.  As 
reported  earlier,  instructors  in  Drs.  Wolf  and  Summerville’s  department  must  travel  frequently,  often  disrupting  their  teaching 
schedules.  When  Dr.  Summerville  heard  about  the  amount  of  time  Dr.  Gwendolyn  spent  in  reading  student  communications,  he 
considered  the  continuous  servicing  of  the  online  site  a form  of  “electronic  tether.”  This  seemed  quite  ironic  since  one  of  his  main 
reasons  for  exploring  distance  education  courses  was  the  opportunity  of  breaking  the  tether  of  the  physical  space  of  the 
classroom.  (“As  long  as  I’ve  got  an  IP  connection.  I’ve  got  a classroom.”)  Would  distance  education  simply  substitute  the  time 
tether  for  the  space  tether  rather  than  eliminating  the  tether  concept  altogether? 

The  issues  surrounding  the  novelty  of  distance  education,  for  both  student  and  instructor,  demonstrated  enormous 
complexity.  Pedagogical  matters  on  how  time  and  space  affected  the  course,  on  what  “interacting”  means  in  a distance  education 
environment,  and  the  new  varieties  of  limitations  and  constraints  imposed  by  distance  education  must  be  considered  by  future 
distance  education  instructors. 

Preparing  Students  for  Distance  Education 

While  this  research  project  specifically  focused  on  what  types  of  preparations  an  instructor  might  need  to  take  into  account  in 
transforming  a class,  the  needs  of  the  student  must  also  be  taken  into  account  for  any  distance  education  course  to  be  considered 
successful.  Dr.  Gwendolyn  often  referred  to  the  idea  that  first -time  distance  education  students  were  “learning  how  to  be  different 
types  of  learners.”  Dr.  Wolf  also  commented  on  how  he  had  lost  sight  of  the  students’  sense  of  being  novices  with  taking  distance 
education  courses  during  part  of  his  preparations  and  implementations  for  some  classes. 

While  the  idea  of  preparing  some  type  of  primer  or  manual  for  novice  distance  education  students  surfaced  on  a number  of 
occasions.  Dr.  Wolfs  team  did  not  prepare  or  distribute  such  a manual  for  the  students  before  the  course  began.  The  purpose  of 
such  a document  would  be  to  alert  the  students  to  some  of  the  differences  in  taking  a distance  education  course  versus  traditional 


course,  This  oversight  did  not  emerge  as  an  issue  during  the  “postmortem”  interviews  after  the  course  ended,  and  it  only  occurred 
to  me  during  the  write-up  phase  of  the  research  that  there  were  plans  for  such  a document. 

Departments,  colleges,  or  universities  desiring  to  expand  their  traditional  base  of  students  and  enter  into  the  world  of  distance 
education  might  consider  publishing  and  making  such  documents  readily  available,  even  before  the  class  begins.  As  one  example, 
Pennsylvania  State  University’s  online  program  offers  online  tutorials  and  suggestions  for  students  to  determine  if  they  might  be 
could  candidates  for  distance  education  courses  as  well  as  ways  to  prepare  for  online  instruction.  Students  would  perform  better  if 
explicitly  told  of  the  different  nature  of  the  demands  placed  on  them  in  these  new  environments,  and  it  would  help  any  instructors 
considering  such  course  transformations,  as  well. 

The  new  and  different  learning  styles  required  of  students  taking  distance  education  courses  might  be  thought  of,  in  some 
manner,  as  different  ways  of  conceptualizing  and  managing  time.  Dr.  Gwendolyn  pointed  out  that  people  in  our  culture  possess 
an  almost  innate  understanding  of  what  it  means  to  “go  to  school”  in  the  sense  of  knowing  the  requirements  of  physically  going 
to  the  campus,  attending  class,  taking  notes,  studying  for  and  taking  exams,  etc.  As  with  all  novel  ideas,  concepts  involved  in 
taking  a distance  education  course  will  have  to  become  learned,  and  this  must  occur  over  time.  Many  of  these  concepts  involve 
time,  for  example,  remembering  to  turn  on  the  computer  at  a certain  time  (for  a synchronous  session);  remembering  to  regularly 
check  the  web  site  for  any  changes,  corrections,  or  updates;  remembering  to  regularly  post  to  bulletin  boards;  etc. 

Limitations  and  Constraints  in  Distance  Education  Environments 

To  many  educators,  distance  education  offers  unparalleled  freedom  of  choices,  with  untapped  or  previously  unavailable 
populations  of  students  taking  their  courses  whenever  their  schedules  allow.  However,  in  preparing  this  particular  course  for 
teaching  at-a-distance,  the  opposite  situation,  problems  of  constraints  and  limitations,  often  arose.  The  first  set  of  constraints 
arose  because  the  course  was  being  taught  in  the  summer-shortened  semester.  Instead  of  the  more  luxurious  16-week  term,  which 
might  have  allowed  for  a more  gradual  roll-out  of  the  online  course.  Dr.  Wolf  was  faced  with  an  8- week  course  that  immediately 
required  his  students  to  double  up  on  their  weekly  lectures  and  quickly  adapt  to  the  new  learning  medium.  The  shortened 
semester  also  presented  difficulties  in  setting  up  the  chat  room  sessions  and  forced  Dr.  Wolf  to  hold  both  sessions  on  the  same 
day.  While  undertaking  a course  transformation  would  be  considered  trying  under  even  benign  circumstances,  the  difficulties  in 
scheduling,  planning,  and  preparing  a course  syllabus  were  magnified  by  the  limited  time  available  in  a shortened  semester. 
Simply  doubling  the  course  meeting  time  in  a semester  half  the  usual  length  can  never  guarantee  results  equal  to  that  of  a normal 
semester.  For  example,  some  courses  with  large  reading  requirements  may  not  afford  the  students  ample  opportunity  to  reflect 
over  what  they  have  read.  The  extra  weeks  of  a semester  might  be  critical  for  true  understanding  of  the  materials  presented. 

Dr.  Gwendolyn  spoke  of  the  time  requirement  needed  for  people  to  adjust  to  the  new  expectations  and  procedures  in  distance 
education  courses,  for  both  students  and  instructors.  In  this  case,  a full  semester  might  have  also  given  the  students  more  time  to 
adjust  to  the  new  learning  methods  and  techniques  required  in  distance  education  environments.  If  they  had  been  more 
comfortable  in  their  new  environment,  they  might  have  been  more  generous  in  their  course  evaluations. 

Even  within  the  structure  of  a longer  term,  the  freedom  promised  by  distance  education  must  not  be  regarded  as  a panacea  for 
all  circumstances.  Dr.  Gwendolyn  recounted  an  interaction  with  a student  who  had  been  commenting  that  he  took  the  distance 
education  course  because  he  thought  it  would  be  the  best  choice  to  accommodate  to  his  heavy  business  travel  schedule.  Instead 
he  ended  up  with  a course  that  required  a great  deal  of  student  input  on  a regular  basis.  He  thought  if  was  the  most  restrictive 
class  he  had  ever  taken  rather  than  the  most  liberating.  Perhaps  if  he  had  taken  the  same  distance  education  course  offered  by 
another  professor,  he  would  have  been  able  to  travel  on  business  and  complete  the  course  without  as  much  difficulty. 

As  the  number  and  availability  of  distance  education  courses  increase  in  the  future,  the  full  range  of  offerings  might  one  day 
rival  those  available  with  traditional  classes.  Perhaps  the  description  of  a particular  distance  education  course  might  include  the 
type  of  information  that  would  have  aided  the  student  in  the  example  above  and  that  type  of  unexpected  difficulty  might  be 
avoided. 

Other  constraints  revolved  around  the  discussion  of  technological  access  to  the  Internet.  Drs.  Wolf  and  Summerville  wanted 
to  include  some  types  of  video  clip  formats  for  their  course,  but  using  video  required  high-speed  Internet  access,  which  was  not 
available  to  most  students.  They  had  to  make  certain  that  any  features  incorporated  into  their  web  pages  could  accommodate  the 
lowest  common  denominator  of  access,  which  amounted  to  using  a modem  set  at  30  kps  (kilobytes  per  second).  Using  such  a 
slow  value  of  30  kps  as  a general  standard  might  surprise  some  readers,  since  the  vast  majority  of  people  have  access  to  56  kps 
modems.  A number  of  smaller  calling  areas,  however,  still  limit  access  to  28.8  kps,  and  30  kps  was  taken  as  an  average.  As  forms 
of  high-speed  access  become  more  readily  available,  this  constraint  will  be  lifted. 

Relevance  of  Andragogy  and  Change  Theory  in  Distance  Education 

Early  in  the  research  process,  I proposed  that  Malcolm  Knowles’s  (1977,  1980,  1984)  theory  of  andragogy  might  provide  the 
most  reasonable  theory  to  frame  this  case  study  research.  Dr.  Wolf,  based  on  this  distance  education  class  experience  and  with 
those  of  past  classes,  confirmed  the  majority  of  Knowles’s  description  of  the  adult  learner.  Knowles’s  last  assumption  concerns 
motivation  for  learning,  and  in  this  case.  Dr.  Wolfs  students  did  not  adhere  strictly  to  an  adrago logical  framework.  Knowles 
presumed  that  internal  motivational  forces  of  self-esteem,  self-actualization,  and  recognition  would  provide  greater  incentives 
than  the  external  motivators  of  job  success  and  financial  reward.  This  might  reflect  Knowles’s  work  environment  while  he  was 
developing  his  theories,  in  that  he  was  in  largely  involved  with  what  would  be  called  today  “continuing  education.”  For  the 
course  involved  in  this  research,  while  internal  motivators  played  an  active  role,  most  of  the  students  were  driven  by  external, 
career  factors. 
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While  Knowles’s  theory  of  andragogy  may  not  be  the  best  available  theory  to  study  and  investigate  distance  education, 
another  possible  candidate  might  exist:  change  theory.  In  his  1999  address  at  Northwestern  State  University  of  Louisiana,  Fuller 
addressed  the  need  for  considering  a new  cultural  transformation  and  reeducation  of  adherents  for  distance  education  to  succeed 
(Fuller,  1999).  Relying  on  two  notable  historians  of  science,  Thomas  Kuhn  and  Kurt  Lewin,  Fuller  proposed  ideas  for  effecting 
systemic  change  in  educational  institutions  noted  for  gradualism  and  continuous  change. 

Fuller  believed  that  the  role  of  technology  had  less  to  do  with  the  curriculum  of  a subject  taught  online  than  did  the  actual 
underlying  culture  that  taught  online.  He  did,  however,  acknowledge  the  new  role  technology  played  between  “instructor  and 
instruction,”  and  realized  the  differences  among  all  participants  in  the  new  era  of  distance  education,  including  trust  between 
instructors  and  administrators: 

The  commitment  to  distance  education,  as  a new  paradigm  of  instruction,  requires  a commitment  to  understanding  the  rules  of  instruction 
differently  within  the  new  dispensation.  It  requires  commitment  on  the  part  of  the  faculty  and  institution,  and  implies  a willingness  to  reveal 
much — perhaps  to  risk  much — on  the  part  of  the  faculty  members  participating  in  the  project,  (p.  5) 

Reigeluth  and  Garfmkle  (1994)  also  addressed  the  issue  of  systemic  change  within  educational  systems.  They  stressed  the 
needs  for  seeking  out  new  ways  of  thinking  in  battling  the  problems  facing  education.  They  also  voiced  concern  over  the 
intricacy  that  incorporating  systemic  thinking  involves: 

Systemic  thinking  is  also  difficult.  It  requires  keeping  many  aspects  of  the  problem  set  in  your  head  at  one  time.  It  is  a community 
activity,  not  an  individual  one,  with  all  the  requisite  challenges  of  any  group  task.  Like  design,  system  thinking  demands  persistence,  because 
to  thiiik  systemically  means  to  constantly  reflect  back  to  previous  assumptions,  and  to  be  flexible  enough  to  change  thinking  that  has  been 
agreed  upon  previously,  (p.  v) 

Flexibility  appears  to  be  a common  theme  among  all  researchers.  Fuller  embraced  Lewin ’s  theory  of  reeducation  in  the  sense 
that  basic  human  behaviors  must  change,  and  these  changes  cannot  be  gradual,  whether  in  learning  or  training  settings.  For 
Lewin,  reeducation  involves  a three  step  pattern  by  which  old  behaviors  “unfreeze,”  change  occurs,  and  the  new  behaviors 
“refreeze.”  This  can  only  occur  in  a safe  environment. 

Dr.  Wolfs  experiences  allude  to  the  required  safe  environment  for  novice  distance  education  instructors.  In  some  sense,  he 
trusted  the  departmental  administrators  to  make  allowances  for  his  first  venture  into  teaching  at -a-di stance.  While  he  did  not 
specifically  mention  how  the  department  responded  to  his  lower  student  evaluations,  his  decision  to  opt  out  of  teaching  future 
distance  education  courses  appear  to  indicate  they  did  not  immediately  allay  his  fears  about  how  those  evaluations  might  affect 
his  position  within  the  department. 

While  the  changes  between  student  and  instructor  may  be  the  most  obvious,  perhaps  more  important  are  the  changes  required 
between  the  instructors  and  the  administration  (or  institution,  as  Fuller  wrote).  Fuller  concludes  his  speech  with  the  following 
challenge: 

Distance  education  requires  a thoroughgoing  change  in  the  classroom  and  the  campus.  In  some  ways,  the  computers  are  the 
least  of  the  revolution.  The  fundamental  relationship  among  professor,  the  learner,  and  the  institutions  of  higher  learning  are  all 
on  the  line  today.  New  models  of  classroom  instruction  are  everywhere.  It’s  time  to  test  new  ways  that  scholars  can  prepare 
themselves — to  prepare  each  other  and  their  institutions — to  thrive  in  the  new  world,  (p.  11) 

Concluding  Remarks 

This  Study  presented  the  experiences  of  one  group  of  instructors  as  they  transformed  a traditional,  face-to-face  course  to  an 
Internet -based,  distance  education  course.  I tried  to  present  as  detailed  and  “thick”  a description  of  the  process  as  possible  so  that 
others  interested  in  the  prospect  of  transforming  their  own  courses  might  understand  the  process  as  completely  as  possible.  In  the 
process,  I uncovered  a number  of  themes  and  concepts  that  I believe  transcend  the  issues  of  this  single  case  study  and  would  be 
relevant  to  all  distance  education  instructors. 

By  its  very  nature,  distance  education  distorts  and  amplifies  differences  in  the  concepts  of  time  and  space.  Nearly  all  of  the 
key  themes  and  concepts  examined  during  this  research  reflected  relationships  of  how  people  either  reacted  to  or  planned  to 
account  for  variations  in  these  two  dimensions  when  compared  to  traditional,  lecture-based  education.  Attending  class; 
communicating,  both  in  class  as  well  as  outside  of  class;  compensating  for  the  loss  of  verbal  and  nonverbal  cues  and  body 
language;  and  the  general  issues  relating  to  constraints  and  limitations  within  the  confines  of  distance  education  can  be  attributed 
to  these  key  concepts. 

At  the  outset  of  this  undertaking,  I originally  thought  that  Knowles’s  theory  of  andragogy  would  be  the  most  suitable 
theoretical  framework  to  support  this  research.  I now  do  not  believe  this  original  assessment  to  be  accurate.  I think  an 
investigation  of  change  theory  as  applied  in  educational  environments  might  be  a much  better  framework  for  understanding  the 
different  processes  that  concurrently  take  place  at  the  time  of  transition  from  traditional  modes  of  instruction  to  distance 
educational  practices. 

All  educators  should  be  aware  of  the  changes  taking  place  in  distance  education,  if  not  for  their  immediate  teaching  efforts, 
then  for  the  necessity  of  observing  the  changes  affecting  their  communities.  A case  study,  no  matter  how  thorough,  can  never 
hope  to  capture  the  ultimate  answer  of  any  research  question — it  can  only  hope  to  accurately  portray  what  occurred  in  this  one 
instance.  I hope  that  this  research  effort  will  motivate  and  inform  other  novice  instructors  interested  in  using  distance  education  in 
their  teaching  efforts.  Additional  research  into  the  discovery  of  the  best  methods  for  transforming  and  adapting  to  this  new 
environment  must  be  forthcoming  among  all  stakeholders  in  higher  education. 
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Abstract 

This  Study  examines  the  use  of  question  prompts  and  peer  interactions  as  scaffolding  strategies  to  help  undergraduate 
students  with  their  problem-solving  processes  on  an  ill-structured  task.  The  mixed  research  method,  combining  both 
experimental  and  comparative  multiple-case  studies,  were  used  to  study  the  outcomes  as  well  as  the  processes  of  students' 
problem-solving  activities  in  terms  of  problem  representation,  developing  solutions,  constructing  argumentation,  and  monitoring 
and  evaluation.  The  result  of  the  experimental  study  showed  that  the  students  working  with  peers  and  also  receiving  question 
prompts  (PQ)  significantly  outperformed  the  other  treatment  groups,  that  is,  individuals  with  question  prompts  (IQ),  individuals 
without  question  prompts  (IC),  and  peers  without  question  prompts  (PC).  At  the  same  time,  though  the  students  in  the  IQ  group 
did  less  well  than  the  PQ  group  in  the  process  of  problem  representation,  they  significantly  outperformed  the  PC  and  the  IC 
groups  in  the  processes  of  problem  representation,  justifications,  and  monitoring  and  evaluation.  There  were  no  significant 
differences  between  the  PC  and  the  IC  groups  in  any  of  the  four  problem-solving  processes.  It  appeared  that  question  prompts 
were  a superior  scaffolding  strategy  over  peer  interactions  in  supporting  students’  ill-structured  problem-solving  processes. 
However,  the  comparative,  multiple  case  studies  revealed  the  complexity  of  the  peer  interaction  context  and  the  relationship 
between  question  prompts  and  peer  interactions.  While  this  study  confirms  previous  findings  on  the  effectiveness  of  question 
prompts  in  facilitating  students’  cognition  and  metacognition,  it  also  indicates  the  benefits  of  peer  interactions,  which  were 
contingent  upon  group  members’  active  and  productive  engagement,  that  is,  questioning,  explaining,  elaborating  and  providing 
feedback  among  peers.  The  study  implies  that,  in  order  for  students  to  gain  full  benefits  from  peer  interactions,  the  peer 
interaction  process  itself  need  to  be  scaffolded,  especially  when  students  were  novice  problem  solvers;  and  question  prompts, 
through  expert  modeling,  may  serve  to  facilitate  this  process. 

Problem  Statement 

Complex,  real-world  problem  solving  is  an  essential  component  of  learning.  Based  on  previous  research  (e.  g.,  Bransford, 
Brown,  & Cocking,  2000;  Bransford  & Stein,  1993;  Jonassen,  1997),  engaging  students  in  complex,  ill-structured  problem- 
solving tasks  not  only  helps  them  to  apply  knowledge  in  real-world  situations,  but  also  to  facilitate  knowledge  transfer. 
However,  previous  research  has  also  pointed  to  students'  deficiencies  in  problem-solving  skills,  for  instance,  failing  to  apply 
knowledge  learned  in  one  context  to  another,  especially  when  solving  problems  on  ill-structured  tasks  (Gick  and  Holyoak,  1980; 
Gick,  1986).  While  students’  difficulties  in  problem  solving  are  partly  attributed  to  misconceptions  or  shallow  conceptions  of 
domain  knowledge  (P.  J.  Feltovich,  Spiro,  Coulson,  & J.  Feltovich,  1996),  they  are,  to  a greater  extent,  due  to  a lack  of 
metacognitive  knowledge  (Brown,  1987). 

Therefore,  it  follows  that  supports  should  be  provided  to  students  during  problem  solving  in  cognition  and  metacognition 
through  various  scaffolding  strategies,  such  as  coaching  through  prompts  (Scardamalia,  Bereiter,  & Steinbach,  1984; 
Scardamalia,  Bereiter,  McLean,  Swallow,  & Woodruff,  1989;  Schoenfeld,  1985;  King,  1992),  modeling  through  reciprocal 
teaching  or  peer-regulated  learning  (e.g.,  Palincsar  & Brown,  1984;  Palincsar,  Brown,  & Martin,  1987),  and  guiding  students  to 
self-generate  questions  (King,  1991).  These  strategies  were  found  to  be  effective  in  fostering  comprehension,  monitoring 
cognitive  thinking,  facilitating  general  problem  solving  (e.g.,  Palincsar  & Brown,  1984;  Scardamalia  et  al.,  1989),  and  supporting 
reflective  thinking  (Lin,  Hmelo,  Kinzer,  & Secules,  1 999). 

Purpose  of  the  Study 

The  purpose  of  the  study  was  to  investigate  the  effects  of  question  prompts  and  peer  interactions  in  scaffolding  undergraduate 
students'  problem-solving  processes  on  an  ill-structured  task.  Although  these  two  strategies  had  been  studied  in  previous 
research,  few  studies  had  been  conducted  on  their  use  to  support  students’  ill-structured  problem  solving.  Hence,  this  study  was 
focused  on  the  effects  of  question  prompts  and  peer  interactions  in  scaffolding  students'  problem-solving  processes  on  an  ill- 
structured  task,  especially  in  problem  representation,  solution,  justifications,  and  monitoring  and  evaluation,  which  characterize 
the  major  processes  of  ill-structured  problem  solving  according  to  previous  research  (e.g.,  Sinnott,  1989;  Voss,  1988;  Voss  and 
Post,  1988).  It  is  hoped  that  the  findings  of  this  research  can  be  applied  in  computer-based  and  web-based  instructional  design,  in 
the  context  of  both  distance  education  and  classroom  instruction. 
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This  study  specifically  addressed  the  following  questions: 

Question  1.  Does  the  use  of  question  prompts  have  an  effect  on  students’  problem  solving  on  an  ill-structured  task  in  problem 
representation,  solution  development,  justification,  and  monitoring  and  evaluation  of  solutions? 

Question  2.  Does  the  use  of  peer  interaction  have  an  effect  on  students'  problem  solving  on  an  ill-structured  task  in  problem 
representation  solution  development,  justification,  and  monitoring  and  evaluation  of  solutions? 

Question  3.  Does  the  use  of  question  prompts  combined  with  peer  interaction  have  an  effect  on  students'  problem  solving  on 
an  ill-structured  task  in  problem  representation,  solution  development,  justification,  and  monitoring  and  evaluation  of  solutions? 

Question  4.  How  does  the  use  of  question  prompts  influence  students'  cognition  and  metacognition  in  the  process  of 
developing  solutions  to  ill-structured  problems? 

Question  5.  How  does  the  use  of  peer  interactions  influence  students’  cognition  and  metacognition  in  the  process  of 
developing  solutions  to  ill-structured  problems? 

Operational  Definitions 

In  this  study,  question  prompts  refer  to  a set  of  static  questions,  both  content-specific  and  metacognitive  types,  which  were 
generated  by  the  content  domain  experts  and  were  designed  to  facilitate  cognition  and  metacognition  and  guide  students  through 
problem-solving  processes.  They  were  related  to  a problem-solving  task  which  students  were  engaged  in.  For  example,  “What 
are  the  parts  of  the  problem?”  was  intended  to  provide  cues  to  students,  activate  their  prior-knowledge,  and  lead  them  to  represent 
the  problem.  When  prompted  to  reflect  on  their  solutions,  students  were  asked  “How  do  I justify  this  specific  system  design?  If  I 
develop  a web-based  solution,  for  example,  can  I explain  why  I took  that  approach?”  The  question  prompts  were  delivered 
through  the  web,  which  students  had  access  to  while  working  in  a computer  laboratory. 

Peer  interactions  are  defined  as  verbal  interactions  of  students  working  together  in  small  groups  of  three  or  four  to  engage  in 
a task  of  ill-structured  problem  solving.  Students  were  expected  to  engage  in  a problem-solving  task,  actively  interact  with  each 
other  to  negotiate  meanings,  to  help  each  other  construct  meanings,  and  collaboratively  develop  solutions  to  a problem. 
Although  they  may  have  different  abilities,  skills  and  background  experiences,  they  were  not  assigned  specific  roles. 

Method 

Participants 

1 15  undergraduate  students  were  recruited  from  three  class  sections  of  an  introductory  course  in  Information  Sciences  and 
Technology  (1ST)  at  a major  university  in  the  United  States  to  participate  in  the  experimental  study,  and  19  of  them  also 
participated  in  the  comparative,  multiple-case  studies.  The  course  was  designed  not  only  to  introduce  basic  concepts  and  provide 
an  overview  of  information  sciences  and  technology,  but  also  to  incorporate  collaborative  learning  and  problem-solving 
experiences.  It  consisted  of  two  lectures  and  one  laboratory  session  each  week.  The  75-minute  lecture  session  was  held  by  a 
professor.  The  1 15-minute  laboratory  session  was  conducted  by  a teaching  assistant.  The  primary  purpose  of  the  lab  was  to 
provide  hands-on  experience  in  information  sciences  and  technology  and  develop  technology  skills  as  well  as  problem-solving 
and  collaboration  skills.  There  were  two  teaching  assistants  attached  to  the  three  class  sections,  with  the  principal  investigator 
being  one  of  them.  All  the  three  class  sections  shared  a common  curriculum  and  a core  textbook,  and  the  three  professors  and  the 
two  teaching  assistants  were  considered  equivalent  in  terms  of  their  expertise  and  teaching  experience.  Due  to  the  relative  large 
size  of  the  class  (about  50  students  in  each  class  section),  the  class  web  site  was  used  as  a supplementary  delivery  medium  to 
foster  classroom  instruction  and  monitor  laboratory  activities.  The  students  were  often  required  to  work  in  teams  to  complete  a 
course  project  or  laboratory  tasks. 

Design 

A mixed  study  design,  combining  an  experimental  study  with  comparative,  multiple-case  studies,  was  applied.  According  to 
Greene,  Caracelli,  and  Graham  (1989),  mixed  study  methods  help  a researcher  to  seek  triangulation  of  the  results  from  different 
data  sources,  examine  overlapping  and  different  facets  of  a phenomenon,  discover  paradoxes,  contradictions,  and  fresh 
perspectives,  and  expand  the  scope  and  breadth  of  a study.  The  experimental  study,  designed  to  answer  research  questions  1-3, 
was  conducted  to  measure  the  students'  problem-solving  outcomes  on  an  ill-structured  task  in  the  four  problem-solving  processes: 
problem  representation ^ solutions ^ justifications,  and  monitoring  and  evaluation,  and  in  four  different  treatment  conditions: 
individuals  with  question  prompts  (IQ),  individuals  without  question  prompts  (IC),  peers  with  question  prompts  (PQ),  and  peers 
without  question  prompts  (PC).  The  comparative,  multiple-case  studies,  through  observation,  hterviews,  and  think-aloud 
protocols,  were  carried  out  to  gain  insights  into  students'  problem-solving  processes,  especially  their  cognition  and 
metacognition,  as  influenced  by  question  prompts  or  peer  interactions.  The  case  studies  were  expected  to  seek  explanations  to 
research  questions  4-5. 

The  Quasi-Experimental  Study 

As  the  experimental  study  was  integrated  into  the  curriculum  and  carried  out  in  a natural  classroom  setting,  the  participants 
were  assigned  to  different  treatment  groups  as  intact  groups.  From  Questions  1-3  the  following  hypotheses  were  generated: 

1.  Students  working  individually  and  also  receiving  question  prompts  will  demonstrate  better  problem-solving  skills  on  an 
ill-structured  task  than  their  counterparts  who  did  not  receive  the  question  prompts  in  (a)  problem  representation,  (b) 
developing  solutions,  (c)  making  justifications  and  (d)  monitoring  and  evaluating  solutions. 


2.  Students  working  with  peers,  with  or  without  question  prompts,  will  demonstrate  better  problem-solving  skills  on  an 
ill-structured  task  than  students  working  individually,  with  or  without  question  prompts,  in  (a)  problem  representation, 
(b)  developing  solutions,  (c)  making  justifications,  and  (d)  monitoring  and  evaluating  solutions. 

3.  Students  working  with  peers  but  also  receiving  question  prompts  will  demonstrate  better  problem-solving  skills  on  an 
ill-structured  task  than  all  the  other  treatment  groups  (PC,  IQ,  and  IC)  in  (a)  problem  representation,  (b)  developing 
solutions,  (c)  making  justifications,  and  (d)  monitoring  and  evaluating  solutions. 

The  participants  of  the  four  different  conditions  were  given  the  same  task  to  solve  during  a 1 15-minute  laboratory  session. 
The  task  presented  an  authentic  scenario  (Table  1)  concerning  the  content  domain  of  1ST,  which  required  students  to  develop  a 
solution  report  to  the  problem  presented. 

Table  1.  The  ill-structured  problem-solving  task 

Many  customers  complain  that  they  have  difficulty  finding  items  in  a large  store.  This  problem  especially  affects 
college  students,  who  often  have  very  little  time  for  shopping.  Since  students  are  major  customers  in  this  small 
college  town,  the  manager  of  a local  major  store  has  hired  you  (or  your  team)  as  a consultant  to  propose  IT-based 
solutions  to  the  problem.  Your  task  is  to  make  suggestions  about  the  features  to  be  included  in  a new  information 
system.  As  part  of  this,  you  are  to  develop  a simple  model  illustrating  your  proposed  system.  Based  on  the  findings 
of  a survey,  the  proposed  information  system  should  be  able  to  help  customers  fnd  items  quickly,  to  present  an 
overall  view  of  all  the  items  on  a shelf  and  an  aisle,  and  to  map  out  the  shortest  route  for  getting  all  the  items  a 
customer  needs  to  purchase.  There  may  be  some  other  important  factors  you  may  need  to  consider. 


The  participants  who  were  assigned  to  the  Question-Prompt  conditions  (PQ  and  IQ)  received  question  prompts  at  the  same 
time  as  they  received  the  problem-solving  task.  The  question  prompts  consisted  of  10  question  prompts  categorized  into  four 
types  of  prompts: 

a)  Problem  Representation  Prompts:  JVhat  is  the  problem? 

b)  Problem  Solution  Prompts:  How  do  I solve  the  problem? 

c)  Justification  Prompts:  What  are  the  reasons  for...  ? 

d)  Monitoring  and  Evaluation  Prompts:  Ami  on  the  right  track? 

There  are  a number  of  questions  in  each  category  of  question  prompts,  as  illustrated  in  Table  2: 

Table  2 The  justification  prompts 

What  are  the  reasons  for  mv  proposed  solution? 

• How  would  I justify  this  specific  system  design.  For  example,  if  I develop  a web-based  solution,  can  I 
explain  why  I took  that  approach? 

• Do  I have  evidence  to  support  my  solution  (i.e.,  the  specific  IT  system  I have  proposed)?  What  is  the  chain 

of  my  reasoning  to  support  my  solution? 


The  problem-solving  reports  were  evaluated  based  on  a judgmental  rubric  systau  to  measure  the  four  problem-solving 
processes  by  the  principal  researcher  and  another  two  raters.  The  interrater  consistency  was  reached  to  ensure  the  reliability  of 
the  evaluation.  A multivariate  analysis  of  variance  (MANOVA)  (Stevens,  1986)  was  employed  to  analyze  the  relationships 
between  question  prompts  and  the  four  problem-solving  processes,  and  between  peer  interactions  and  the  four  problem-solving 
processes  across  the  four  different  conditions.  As  the  MANOVA  result  was  statistically  significant,  the  univariate  (ANOVA) 
results  were  examined  for  each  dependent  variable.  For  the  significant  univariate  results,  post  hoc  comparisons  were  performed 
to  identify  where  the  differences  resided. 

The  Comparative,  Multiple-Case  Studies 

Eight  cases  were  studied.  Four  individuals,  with  two  from  the  IQ  and  two  from  the  IC  condition,  were  selected  for  think- 
aloud  protocols,  which  were  conducted  when  each  of  them  was  engaged  in  the  problem-solving  task.  In  addition,  observations 
and  interviews  were  conducted  on  four  selected  groups  to  gather  data  about  their  problem-solving  processes,  two  groups  from  the 
PQ  and  two  groups  from  the  PC  condition.  The  multiple  cases  were  analyzed  for  the  purpose  of  theoretical  replication,  which 
either  (a)  predicts  similar  results  or  (b)  produces  contrasting  results  but  for  predictable  reasons  (Yin,  1989). 

Miles  and  Huberman's  (1994)  data  analysis  model,  which  involves  three  subprocesses— data  reduction,  data  display  and 
conclusion  drawing  and  verification,  was  used  to  guide  the  qualitative  data  analysis.  The  data  analysis  primarily  consisted  of  the 
following  steps:  reading  and  jotting  marginal  notes  on  the  transcripts;  identifying  patterns  and  labeling  concepts;  organizing 
labeled  concepts  into  a data  display  matrix,  identifying  themes  and  drawing  conclusions.  The  focus  of  the  analysis  was  on  cross- 
case comparisons  viewed  from  different  dimensions:  the  four  ill-structured  problem-solving  processes  and  the  effects  of  question 
prompts  and  peer  interactions  on  cognitive  thinking  and  metacognitive  skills. 

Results 

The  Quantitative  Results 

The  results  of  multivariate  analysis  of  variance  showed  overall  differences  for  the  treatment  effect  and  the  four  dependent 
variables  of  problem-solving  processes.  The  MANOVA  results  were  statistically  significant  (F  = 4.025,  p<  .001).  Further,  the 
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results  of  the  univariate  ANOVA  tests  indicated  that  there  were  significant  statistical  differences  in  all  the  four  dependent 
variables,  with  an  F ratio  of  20.43,  8.27,  1 1.26  and  7.21  respectively  (p  < .001).  Table  3 is  a summary  of  post  hoc  Scheffe  mean 
comparison.  It  shows  several  statistical  mean  differences  among  the  four  treatment  conditions  in  the  four  dependent  variables. 
Table  4 summarizes  the  descriptive  statistics  for  the  dependent  variables  by  treatment  groups.  It  displays  means  and  standard 
deviations  of  different  treatment  groups  by  the  four  dependent  variables.  As  each  dependent  variable  has  a different  subtotal  of 
scaled  points,  percentage  was  used  to  create  a common  basis  for  comparison  of  means  among  the  four  dependent  variables. 


Table  3 Summary  of  post  hoc  Scheffe  comparison 


Dependent  Variable 

Reoresentinc 

Problem 

Develooine 

Solutions 

Makine 

Justifications 

Monitoring  and 
Evaluation 

Comparison  Group 

Mean 

Difference  (%) 

Mean 

Difference  (%) 

Mean 

Difference  (%) 

Mean 

Difference  (%) 

Peer  Question  (PQ) 
vs. 

Peer  Control  (PC) 

35.9* 

(PQ  > PC) 

20.3* 

(PQ  > PC) 

33.7* 

(PQ  > PC) 

35.6* 

(PQ  > PC) 

Peer  Question  (PQ) 
vs. 

Individual  Question  (IQ) 

17.6* 

(PQ>IQ) 

11.8 

7.7 

1.5 

Peer  Question  (PQ) 
vs. 

Individual  Control  (IC) 

39.8* 

(PQ  > IC) 

21.3* 

(PQ  > IC) 

27.3* 

(PQ  > IC) 

34.8* 

(PQ  > IC) 

Peer  Control  (PC) 
vs. 

Individual  Question  (IQ) 

-18.3* 

(PC  < IQ) 

-8.5 

-25.9* 

(PC  < IQ) 

-34.0* 

(PC  < IQ) 

Peer  Control  (PC) 
vs. 

Individual  Control  (IC) 

3.9 

1.0 

-6.3 

-0.8 

Individual  Question  (IQ) 
vs. 

Individual  Control  (IC) 

22.2* 

(IQ>IC) 

9.5 

19.6* 

(IQ>IC) 

33.2* 

(IQ>IC) 

Note. 

a)  The  mean  difference  shown  in  this  table  is  the  subtraction  of  the  second  condition  (on  the  lower  line)  from  the  first 
condition  for  example,  35.9  (Mean  Difference  for  Problem  Representation)  = PQ  - PC. 

b)  Mean  difference  (%)  is  calculated  using  the  values  which  appear  in  Table  4. 

c)  (%).  The  mean  difference  is  converted  into  percentage  in  order  to  create  a common  basis  for  mean  comparison,  as  the 
subtotals  for  the  four  dependent  variables  are  different. 

d)  *.  The  mean  difference  is  significant  at  the  .05  level. 


Table  4 Descriptive  statistics  for  each  dependent  variable  by  treatment  group 


Treatment  Group 

Dependent 

Variables 

Peer  Question 

(PQ) 

(N=I3) 

Mean%  SD  % 

Peer  Control 
(PC) 

(N=ll) 

Mean  % SD  % 

Individual  Question 

(IQ) 

(N=I5) 

Mean  % SD  % 

Individual  Control 
(IC) 

(N=I6) 

Mean  % SD  % 

Representing 

Problem 

62.3 

17.4 

26.4 

15.7 

44.7 

15.5 

22.5 

11.3 

Developing 

Solutions 

88.5 

11.9 

68.2 

17.1 

76.7 

10.4 

67.2 

11.1 

Making 

Justifications 

79.1 

18.1 

45.5 

12.5 

71.4 

17.1 

51.8 

18.7 

Monitoring  and 
Evaluation 

61.5 

31.1 

25.9 

24.6 

60.0 

30.2 

26.8 

22.7 
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As  it  can  be  seen  from  Table  3 and  Table  4,  The  statistical  results  generally  confirm  Hypothesis  7,  showing  that  students 
working  individually  and  also  receiving  question  prompts  (IQ)  demonstrated  higher  problem-solving  skills  on  an  ill-structured 
task  than  the  individuals  who  did  not  receive  the  question  prompts  (IC)  in  (a)  problem  representation,  (c)  making  justifications, 
and  (d)  monitoring  and  evaluating  solutions.  However,  the  IQ  group  did  not  perform  significantly  better  than  IC  group  in  (b) 
developing  solutions. 

The  statistical  results  only  partially  support  Hypothesis  2.  Students  working  with  peers  and  also  receiving  question  prompts 
(PQ)  outperformed  those  working  individually  and  without  question  prompts  (IC)  in  all  the  four  problem-solving  processes. 
They  also  outperformed  those  working  individually  and  also  receiving  question  prompts  (IQ)  in  (a)  problem  representation. 
However,  no  significant  differences  were  found  in  any  of  the  four  problem-solving  processes  between  the  PC  and  IC  groups.  On 
the  contrary,  students  working  individually  but  receiving  question  prompts  did  significantly  better  than  students  working  with 
peers  but  without  question  prompts  in  three  processes:  (a)  problem  representation,  (c)  making  justifications,  and  (d)  monitoring 
and  evaluating  solutions. 

In  testing  Hypothesis  i,  students  working  with  peers  and  also  receiving  question  prompts  (PQ)  demonstrated  better  problem- 
solving skills  than  the  students  in  the  other  conditions  (PC,  IQ,  and  IC)  in  (a)  problem  representation.  In  (b)  developing 
solutions,  (c)  making  justifications,  and  (d)  monitoring  and  evaluating  solutions,  students  in  the  PQ  condition  did  significantly 
better  than  those  in  the  PC  and  the  IC  condition,  though  not  than  those  in  the  IQ  group. 

The  Qualitative  Findings 

The  effect  of  question  prompts.  The  cross-case  qualitative  analysis  revealed  that  the  question  prompts  supported  students' 
cognition  and  metacognition  through  directing  attention  to  their  problem-solving  processes,  articulating  thoughts,  and  providing 
guidelines.  First,  question  prompts  served  as  a “reminder”  to  direct  the  students’  attention  to  some  important  information  they 
might  not  have  thought  about.  They  helped  students  to  represent  the  problem,  make  connections  between  different  factors  and 
constraints  and  link  to  the  solutions.  In  addition,  the  question  prompts  also  led  the  students  to  think  about  alternative  solutions 
and  the  viability  of  their  solutions.  A group  of  students  who  failed  to  use  the  question  prompts  and  thus  failed  to  think  about  all 
the  possibilities  and  alternative  solutions  indicated  the  important  functions  of  question  prompts.  Second,  it  was  observed  that 
question  prompts  reminded  the  students  to  state  their  reasons  and  construct  arguments  for  their  proposed  solutions.  Third, 
students  mentioned  that  the  question  prompts  were  useful  to  help  them  organize  their  thinking  and  break  down  the  problem  into 
small  steps.  Therefore,  the  question  prompts  may  have  served  as  expert  modeling  to  support  students'  cognitive  and 
metacognitive  thinking  by  guiding  problem  representation,  metacognition,  and  the  justification  process. 

The  effect  of  peer  interactions.  The  comparative  case  studies  indicated  that  the  greatest  advantages  of  peer  interactions  lie  in 
building  upon  each  other’s  ideas,  questioning  and  providing  feedback,  providing  multiple  perspectives,  and  benefiting  from 
distributed  knowledge.  Those  attributes  influenced  students'  cognitive  thinking  and  metacognitive  knowledge.  It  was  observed 
in  all  the  cases  that  when  peers  worked  together,  they  typically  started  the  problem-solving  processes  by  brainstorming  ideas, 
which  were  presented  in  the  form  of  questions  or  suggestions,  such  as  “How  about...?”  and  “What  do  you  think...?”.  Then,  an 
initial  idea  got  further  developed.  It  was  also  observed  that  during  peer  interactions,  students  asked  questions,  offered 
suggestions,  elaborated  thinking  and  provided  feedback.  Thus,  peer  interactions  created  an  opportunity  to  ask,  clarify,  explain, 
and  elaborate.  The  great  advantages  of  peer  interactions,  as  the  participants  consistently  pointed  out  in  their  interviews,  were  the 
multiple  perspectives  and  different  expertise  different  individuals  brought  to  the  problem-solving  processes. 

Discussion 

Here  is  a summary  of  answers  to  research  questions  1-5: 

1.  Question  prompts  had  a significantly  positive  effect  overall  on  students'  problem-solving  processes  on  an  ill-structured 
task,  specifically  in  (a)  problem  representation,  (c)  making  justifications  and  (d)  monitoring  and  evaluating  solutions. 

2.  The  use  of  peer  interactions  had  a partially  positive  effect  on  students'  problem  solving  processes  on  an  ill-structured 
task  in  that,  the  students  in  the  PQ  condition  significantly  outperformed  those  in  the  IC  condition  in  all  the  problem- 
solving processes  and  the  IQ  condition  in  problem  representation;  whereas  the  students  in  the  PC  condition  did  not 
perform  significantly  better  than  those  in  the  IQ  or  the  IC  condition  in  any  of  the  problem  solving  processes. 

3.  In  comparison  with  the  separate  use  of  question  prompts  and  of  peer  interactions,  the  combination  of  question  prompts 
with  peer  interactions  showed  the  greatest  positive  effect  overall  on  students'  problem-solving  processes  on  an  ill- 
structured  task. 

4.  In  the  process  of  developing  solutions  to  ill-structured  problems,  question  prompts  influenced  students’  cognition  and 
metacognition  by  (a)  directing  attention,  (b)  articulating  thoughts,  and  (c)  providing  guidelines  for  problem  solving. 

5.  In  the  process  of  developing  solutions  to  ill-structured  problems,  peer  interactions  influenced  students'  cognition  and 
metacognition  by  (a)  building  upon  each  other's  ideas,  (b)  questioning  and  providing  feedback,  (c)  providing  multiple 
perspectives  and  (d)  distributing  cognition. 

The  result  of  the  experimental  study  showed  that  the  students  working  with  peers  and  question  prompts  (PQ)  significantly 
outperformed  the  other  treatment  groups,  especially  the  students  without  question  prompts  (either  working  individually  or  with 
peers),  in  all  the  four  problem-solving  processes.  At  the  same  time,  the  students  working  individually  and  with  question  prompts, 
though  they  did  less  well  than  the  PQ  group  in  problem  representation,  significantly  outperformed  the  PC  and  IC  groups  in 
problem  representation,  justifications,  and  monitoring  and  evaluation.  There  were  no  significant  differences  between  the  PC  and 


166 

159 


the  IC  groups  in  any  of  the  four  problem-solving  processes.  It  appeared  that  question  prompts  were  a superior  scaffolding 
strategy  over  peer  interactions  in  supporting  students’  problem  solving  on  an  ill-structured  task.  However,  the  comparative, 
multiple  case  studies  revealed  the  complexity  of  the  peer  interaction  context  and  the  relationship  between  question  prompts  and 
peer  interactions.  While  this  study  confirmed  the  findings  cf  previous  research  on  the  effectiveness  of  question  prompts  in 
facilitating  students’  cognition  and  metacognition,  it  also  showed  the  benefits  of  peer  interactions,  which  were  contingent  upon 
group  members’  active  and  productive  engagement  in  peer  interactions,  that  is,  questioning,  explaining,  elaborating  and 
providing  feedback  among  peers.  The  findings  supported  Webb's  (1989)  research  on  the  learning  conditions  for  group 
collaboration. 

The  study  implies  that,  in  order  for  students  to  gain  full  benefits  from  peer  interactions,  the  peer  interaction  process  itself 
need  to  be  scaffolded,  especially  when  students  were  novice  learners  in  problem  solving;  and  question  prompts,  through  expert 
modeling,  may  serve  to  facilitate  this  process.  Further  research  is  suggested  to  examine  the  transfer  effect  of  question  prompts  on 
students'  self-generated  questions  if  students  are  provided  with  similar  question  prompts  over  a period  of  time  and  if  their 
improved  skills  in  self-generated  questions  during  problem  solving  will  facilitate  them  to  solve  an  ill-structured  problem.  More 
research  efforts  are  also  needed  to  examine  group  dynamics  when  investigating  the  role  of  peer  interactions  in  scaffolding  ill- 
structured  problem  solving.  Group  dynamics  involve  many  aspects,  including  peer  learning  approaches,  peer  interaction  patterns, 
students'  perception  and  motivation  about  peer  learning,  any  of  which  may  have  an  impact  on  students'  problem-solving 
performance. 
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Abstract 

The  Media  Center  at  Eastern  Illinois  University  developed  and  streamed  on  the  Internet  26  short  (1-5  minutes)  instructional 
videos  about  WebCT  that  illustrated  specific  functions,  including  logging- in,  changing  a password,  and  using  chat.  This  study 
observed  trainees  using  and  reacting  to  selections  of  these  videos.  It  set  out  to  assess  attitudes  toward  the  quality  of  online  videos 
and  to  identify  perceptions  trainees  had  about  the  video  *s  impact  on  their  learning.  A secondary  aim  of  the  study  was  to  evaluate 
the  potential  of  a video-split-screen  technique  for  making  observations  of  trainees  during  training. 

Trainees  responded  positively  to  the  video  training  and  the  mode  of  delivery.  They  also  perceived  their  learning  to  have  been 
positively  impacted  as  a result  of  it.  As  an  observation  tool,  the  video-split-screen  technique  was  useful  and  it  yielded  much  data 
in  multiple  media  formats. 

Instructional  video  training  as  presented  in  this  paper  appears  to  be  a viable  WebCT  resource.  An  extensive  development 
effort  was  not  needed  to  produce  a worthwhile  product  that  is  easily  modified  and  updated.  These  facts  coupled  with  trainees’ 
positive  reactions  suggest  that  this  instructional  modality  can  be  an  effective  supplement  to  face-to-face  training. 

Introduction 

In  1997,  the  Media  Center  at  Eastern  Illinois  University  embarked  on  a technology  training  initiative  aimed  at  improving 
faculty  members’  ability  to  effectively  and  appropriately  utilize  and  integrate  computer  technology  into  the  teaching  and  learning 
process.  To  foster  faculty  members’  willingness  to  invest  in  using  technology,  the  initiative  provided  services  in  three  key  areas. 
First,  technical  support  services  addressed  software  and  hardware  problems.  For  instance,  an  instructor  having  difficulty  with 
HTML  coding  or  a specific  software  application  could  call  the  Center  for  assistance.  Second,  the  Center  developed  courseware 
applications  as  well  as  materials  to  support  classes,  including  Web  sites  and  CD-ROM  materials.  Third,  the  Center  offered  a 
number  of  faculty  development  activities,  such  as  hands-on  training  workshops,  informational  sessions/demonstrations,  and 
computer-based  tutorials.  In  this  paper,  the  authors  limit  the  discussion  of  the  initiative  to  the  area  of  faculty  (technology) 
development. 

Subsequent  to  the  implementation  of  the  initiative  the  University  adopted  WebCT  as  its  Web-based  course  development 
platform.  WebCT  is  a development  tool  that  enables  instructors  to  create  and  distribute  on-line  class  materials,  or  entire  online 
courses.  The  University  administration  charged  the  Media  Center  with  formulating  a WebCT  training  program,  as  part  of  it 
existing  training  initiative.  The  program  was  to  introduce  the  software  to  faculty  and  teach  them  how  to  use  it  for  instructional 
and  learning  purposes. 

The  Center’s  staff  delivered  WebCT  training  primarily  through  face-to-face  workshops  that  were  2-3  hours  in  length. 
Throughout  each  academic  semester,  they  offered  over  40  workshops  on  the  following  topics: 

• WebCT  basics:  An  Introduction  to  WebCT 

• Using  WebCT  Communication  Tools 

• Using  WebCT  Quiz  Modules 

• Using  WebCT  Student  Management 

• Using  WebCT  File  Manager,  Course  Content,  Calendar 

Several  open  sessions  were  also  offered  in  which  the  staff  presented  no  formal  instruction  but  assisted  faculty  with  developing 
content  in  WebCT. 

Streaming  Training  Content 

While  the  WebCT  trainers  successfully  offered  numerous  workshops  at  varying  times  throughout  the  semester  to 
accommodate  faculty  schedules,  they  observed  three  shortcomings  with  the  workshop  approach.  First,  despite  the  availability  of 
sessions,  class  scheduling  conflicts  prevented  some  faculty  from  attending.  Second,  faculty  indicated  that  the  timing  of  the 
workshops  did  not  coincide  with  the  time  they  allocated  for  development.  For  example,  one  individual  commented  that  while  he 
attended  a workshop  in  October,  he  was  unable  to  do  any  WebCT  development  until  the  end  of  the  semester,  at  which  time,  he 
thought  his  familiarity  with  the  program  would  wane.  Third,  in  many  ways,  the  workshops  did  not  support  individual  work  habits 
or  needs.  Trainers  offered  all  sessions  during  normal  work  hours  and  training  was  general  in  nature  so  that  it  accommodated  the 
greatest  number  of  participants.  As  a result,  an  individual  working  outside  normal  business  hours  could  not  call  on  the  expertise 
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of  a trainer  and  was  therefore  left  to  his/her  own  resources  to  solve  a problem.  Moreover,  individuals  with  specific  WebCT  needs 
or  questions  were  difficult  to  accommodate  during  workshops,  and  the  trainers  observed  that  many  of  these  unique  needs  were 
unmet. 

The  aforementioned  issues  associated  with  the  workshop  format  prompted  the  Media  Center  staff  to  explore  alternative 
training  delivery  modalities,  one  of  which  being  streaming  video  training.  Two  staff  instructional  technologists  developed  26 
short  (1-5  minutes)  instructional  videos  about  WebCT  that  illustrated  specific  functions,  including  logging-in,  changing  a 
password,  and  using  chat.  While  the  videos  were  general  to  WebCT,  they  contained  information  specific  to  Eastern  Illinois 
University,  such  as  how  to  access  a WebCT  course  from  the  university  home  page  or  how  to  obtain  a WebCT  account.  The 
technologists  used  a screen  capturing  utility  (Snagit)  to  record  the  WebCT  screens,  and  they  narrated  while  performing  program 
functions  (see  Figure  1). 
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Figure  J.  WebCT  Video  Played  in  a Web  Browser 

The  approach  did  not  require  many  hours  of  development,  an  important  criterion  of  the  technologists.  Time  constraints  prohibited 
these  individuals  from  spending  exceedingly  long  periods  of  time  developing  the  videos.  Moreover,  the  changing  nature  of  the 
WebCT  software  with  updated  versions,  etc.  necessitated  a short  development  cycle. 

Developing  the  WebCT  Training  Videos 

As  mentioned,  the  instructional  technologists  used  the  Snagit  software  utility  to  capture  computer  video  screens  and  audio 
narration  simultaneously.  The  program  records  the  video  and  audio  into  a single  Avi  file,  which  is  commonly  used  in  Windows- 
based  PC’s. 
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The  recordings  produced  large  file  sizes  that  could  not  be  streamed  on  the  Internet  and  so  the  Avi  files  were  converted  to  a 
QuickTime  format.  QuickTime  was  chosen  because  the  Media  Center  owned  a QuickTime  streaming  server.  The  accepted 
industry  standard  for  converting  audio  and  video  files  with  the  least  possible  quality  loss  (or  generation  loss)  is  Terran 
Corporation’s  Media  Cleaner  Pro.  Version  5 of  the  software  is  called  Cleaner.  Cleaner  and  QuickTime  Pro  both  encode 
miscellaneous  QuickTime  and  streaming  QuickTime  files  with  constant  bit-rate  encoding  using  several  different  types  of 
compression  schemes. 

Compressing  video  causes  the  video  to  shrink  in  size.  Instead  of  encoding  each  frame  of  video  with  the  entire  contents  of  that 
frame,  only  the  changes  from  frame  to  frame  are  encoded.  In  addition,  if  colors  in  an  area  of  the  video  are  similar,  a single  byte  of 
information  representing  the  entire  area  is  used  resulting  in  smaller  file  size.  Smaller  video  files  and  lower  data  rates  enable  the 
video  to  be  streamed  more  easily  over  the  Internet  and  to  be  viewed  on  dated  computers. 

The  Media  Center  also  purchased  a plug-in  for  Media  Cleaner  Pro  called  Sorenson  Developer  edition.  In  order  to  encode  a 
QuickTime  movie  with  variable  bit -rate  encoding  the  plug-in  is  required.  Variable  bit -rate  encoding  allows  the  bit-rate  to  vary  as 
the  file  requires  bytes  resulting  in  higher  quality  video  at  a low  average  bit -rate.  Constant  bit -rate  provides  predictable  data  rates, 
but  produces  video  of  poorer  quality. 

Typically,  the  largest  video  frame  size  that  can  be  streamed  over  the  Internet  or  a very  fast  connection  (e.g.,  T-1  or  faster)  is 
320  X 240  with  10  frames  per-second.  Since  the  size  of  the  WebCT  videos  was  approximately  640  X 400,  a compromise  was 
achieved  by  decreasing  the  frame  rate.  To  obtain  the  same  data  rate,  one  can  increase  frame  size  while  decreasing  frame  rate  or 
vice  versa.  Since  the  motion  on  the  screen  was  mainly  mouse  movement  and  Web  page  scrolling,  the  file  frame  rate  was 
decreased  to  4 frames  per-second. 

After  the  files  were  processed  by  Cleaner  5,  they  were  copied  to  the  QuickTime  server.  The  developers  used  a program  called 
Make  Reference  Movie  to  create  small  QuickTime  files  (approximately  4K)  that  were  placed  on  the  university’s  web  server 
along  with  HTML  documents  that  linked  to  the  videos. 

Purpose 

In  the  researchers’  view,  delivering  the  videos  in  a Web-based  hypermedia  environment  was  unique,  and  it  presented  a 
number  of  issues  that  needed  careful  examination.  As  a result,  they  set  out  to  examine  how  trainees  would  use  and  react  to  the 
instructional  videos.  Specifically,  the  researchers  set  out  to: 

1 . assess  trainees’  attitudes  about  the  quality  of  the  online  instructional  videos  used  in  the  project  and  the  method  by 
which  they  were  delivered; 

2.  identify  trainees’  perceptions  about  the  effectiveness  of  the  videos  to  fostering  their  learning. 

A secondary  aim  of  the  study  was  to  evaluate  the  potential  of  a video  split-screen  technique  for  observing  the  processes  (e.g., 
performance  tasks/work  behaviors)  trainees  engage  in  while  using  the  instructional  videos. 


Method 

Hypermedia,  characterized  by  an  arrangement  of  nodes  and  links,  provides  non-sequential  access  to  mediated  content 
(Kumar,  Helgeson  & White,  1994).  These  informationally  rich  and  flexible  user- centered  designs  add  complexity  to  the  study  of 
how  users  interact  with  a system  (Gay  & Mazur,  1993).  Some  evaluation  approaches  promote  a holistic  orientation  (Winograd  & 
Flores,  1986)  using  qualitative  methods  (Card,  Moran  & Newell,  1983)  and  multiple  data  collection  instruments  (Marchionini, 
1990).  Instruments  for  monitoring  human-computer  interactions  often  allow  for  the  compilation  of  data  in  visual  (video, 
photographic),  textual,  and  auditory  form. 

In  accordance  with  a more  qualitative  approach  to  evaluation,  the  researchers  used  a video-split -screen  recording  technique  to 
monitor  how  trainees  used  the  instructional  videos.  A video  camera  recorded  trainees  as  they  used  the  software.  Simultaneously, 
a scan  converter  converted  the  computer  screen  output  to  a video/NTSC  signal.  Both  signals  (trainee  and  computer  screen)  were 
sent  to  a video  effects  generator  and  combined  into  one  image,  which  allowed  the  researchers  to  observe  the  computer  screen, 
including  mouse  movements,  object  (e.g.,  buttons,  links,  etc.)  selections,  and  web  page  changes  simultaneously  with  trainees’ 
behavior  (see  Figure  2). 
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Figure  2.  Video  Split-screen  Recording  Process 


Materials 

The  researchers  created  a web  site  that  provided  an  overview  of  the  study  and  instructions  for  completing  it.  They  chose  1 1 
of  the  26  instructional  videos  for  review  because  the  selected  videos  represented  fundamental  WebCT  components.  Video  topics 
included: 

• Logging  into  WebCT 

• Modifying  headers  and  footers 

• Communication  Tools  Overview  Part  1 

• Communication  Tools  Overview  Part  2 

• WebCT  Mail  Overview 

• Sending  Mail  through  WebCT 

• Using  Bulletin  Boards  Part  1 

• Using  Bulletin  Boards  Part  2 

• Using  Bulletin  Boards  Part  3 

• Starting  Chat 

• Using  Chat 

Prior  to  beginning  the  study,  trainees  completed  a pre-usability  survey  that  assessed  their  familiarity  with  various  WebCT 
components.  Trainees  rated  their  familiarity  on  a 7-point  scale  from  I (Completely  Unfamiliar)  to  7 (Very  Familiar).  They 
viewed  each  of  the  11  videos,  in  any  order  of  preference.  After  viewing  each  video,  trainees  practiced  the  WebCT  function  or 
component  they  just  viewed.  When  practicing,  the  researchers  asked  them  to  think  aloud  or  verbalize  what  was  going  through 
their  minds.  After  practicing,  trainees  completed  a short  WebCT  Function  survey,  which  presented  7 attitudinal  statements  to  be 
rated  from  1 (Strongly  Disagree)  to  7 (Strongly  Agree).  Once  trainees  watched  all  1 1 videos,  practiced  them,  and  completed  their 


,173 


BEST  COPY  AVAILABLE 


respective  function  surveys,  they  received  a post-usability  survey  that  presented  identical  questions  to  those  of  the  pre-usability 
survey.  It  took  each  trainee  approximately  two  hours  to  complete  the  study. 

The  five  trainees  who  completed  all  evaluation  tasks  were  undergraduate  students  employed  at  the  Media  Center.  Most 
indicated  that  they  were  computer  familiar  but  had  little  or  no  experience  with  WebCT. 

Results 

Table  1 shows  how  familiar  the  trainees  were  of  the  WebCT  components  before  and  after  the  video  training.  The  higher  the 
average  rating,  the  more  familiar  the  trainees  were  with  the  specific  component  (the  scale  ranges  from  1 to  7).  Separate  t-tests  for 
dependent  or  paired  means  were  conducted  for  each  WebCT  component.  At  a significance  level  of  .05,  results  indicate  that  the 
trainees  became  significantly  more  familiar  with  the  components  after  the  video  training.  Table  1 also  shows  that  the  trainees 
were  slightly  familiar  with  the  WebCT  in  general  and  with  logging  into  the  WebCT  before  the  training,  but  were  completely 
unfamiliar  with  the  specific  WebCT  components. 


WebCT  Component 

Before  Video  Training 

After  Video  Training 

WebCT  in  general 

2.60 

5.60 

Logging  into  WebCT 

3.60 

6.80 

Changing  WebCT  password 

1.80 

5.20 

Modifying  header/footer  on  the  WebCT 
homepage 

1.40 

7.00 

Reading  a message  in  a WebCT  discussion 
forum 

1.40 

5.80 

Posting  a message  to  a WebCT  discussion 
forum 

1.40 

5.80 

Reading  a message  with  WebCT  email 

1.40 

6.40 

Posting  a message  with  WebCT  email 

1.40 

6.40 

Accessing  WebCT  chat  rooms 

1.40 

6.40 

Using  WebCT  chat  rooms 

1.40 

6.40 

Table  L Familiarity  with  WebCT’s  Components  Before  and  After  the  Video  Training 

The  trainees’  evaluations  of  the  eleven  videos  were  averaged  per  evaluation  item  and  are  shown  in  Table  2 below.  For  each  item, 
the  trainees  gave  very  positive  evaluations  of  the  videos.  The  higher  the  average  rating,  the  more  positive  the  evaluation  (on  a 
scale  of  1 to  7). 


Evaluation  Item 


Average  Rating 


No  problems  accessing  the  video  6.76 

No  problems  using  and  viewing  the  video  5.95 

The  video  was  easy  to  understand  6.33 

The  video  was  well-paced  6.44 

The  video  helped  in  learning  about  the  specific  WebCT  function  6.59 

The  video  format  was  the  best  way  of  learning  about  the  specific  WebCT  function  6.62 

Would  recommend  that  others  use  the  specific  video 6.40 


Table  2.  Trainees  ’ Evaluations  Across  the  Training  Videos 

The  trainees’  evaluations  of  each  instructional  video  were  obtained  by  averaging  their  ratings  across  the  seven  evaluation 
items.  The  average  ratings  per  video  are  shown  in  Table  3 below.  The  trainees  gave  very  positive  evaluations  for  each  video. 
The  higher  the  average  rating,  the  more  positive  the  evaluation  (on  a scale  of  1 to  7). 
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Training  Video  Title 


Average  Rating 


Logging  into  WebCT 

6.11 

Modifying  headers  and  footers 

6.17 

Communication  tools  overview  (part  1) 

6.29 

Communication  tools  overview  (part  2) 

6.54 

WebCT  mail  overview 

6.69 

Sending  mail  through  WebCT 

6.69 

Using  bulletin  boards  (part  1) 

6.37 

Using  bulletin  boards  (part  2) 

6.57 

Using  bulletin  boards  (part  3) 

6.40 

Starting  chat 

6.51 

Using  chat 

6.49 

Table  3.  Trainees ' Evaluations  of  the  Specific  Training  Videos 

Trainees  Commentary 

Overall,  trainees  had  few  general  comments  about  the  videos.  Two  individuals  wrote  the  following  comments  on  the  pre- 
usability survey: 

• I did  a little  with  WebCT  questions  over  in  the  Media  Center, 

• I don't  know  anything  about  WebCT  so  it  will  be  nice  to  learn. 

The  same  two  individuals  wrote  the  following  comments  on  the  post-usability  survey: 

• / don't  feel  I learned  much, 

• This  was  fairly  easy  to  understand.  Because  the  video  was  sometimes  fuzzy  it  was  hard  to  see  everything  that  was  going 
on,  I feel  like  I have  a good  grasp  of  the  beginnings  of  WebCT  and  could  probably  figure  out  more  things  on  my  own. 

It  is  difficult  to  ascertain  whether  the  trainee  who  wrote,  I did  not  learn  muchy  was  commenting  about  the  inability  of  the  video 
training  to  facilitate  learning  about  WebCT  or  that  her  learning  did  not  increase  because  her  skills  equaled  the  level  of  mastery 
the  videos  sought  to  promote.  Her  ratings  of  post -usability  survey  items  were  high  (M  = 6.4)  compared  to  her  pre-usability  survey 
ratings  (M  = 1.7),  which  suggests  that  she  perceived  her  learning  to  have  increased. 

Trainees  made  15  independent  comments  about  the  videos  (see  Table  4).  Generally,  the  commentary  reflected  their  concerns 
about  the  video  images  appearing  fuzzy.  Internet  congestion  periodically  caused  the  video  image  to  distort.  Comments  also 
point  out  that  trainees  preferred  the  videos  to  be  short,  helping  them  to  avoid  feeling  overwhelmed.  Lastly,  commentary  reflected 
that  some  trainees  saw  similarities  between  specific  WebCT  functions  and  third-party  email  software  and,  for  at  least  one 
participant,  this  association  fostered  understanding  of  WebCT  functions. 
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Training  Video  Title 

Comments 

Logging  into  WebCT 

Improve  the  picture  quality  of  the  video.  It  was  choppy  and  messy  as  it  ran. 
It  was  difficult  at  time  to  see  what  was  actually  on  the  screen.  It  was  ftizzy  a 
few  times  but  because  the  voice  told  me  what  was  there,  I knew  what  to 
expect.  If  possible  it  may  be  quicker  to  one-on-one  demonstrations. 

Modifying  headers  and  footers 

Good  timing  of  the  videos.  They  are  not  too  long  and  don't  make  me  feel 
overwhelmed. 

Because  the  screen  is  fuzzy  it  was  hard  to  catch  all  of  the  information. 
There  were  several  screens  that  were  similar,  but  it  didn't  take  too  long  to 
figure  out. 

Communication  Tools  Overview 
Part  1 

This  was  less  information  to  soak  in.  This  was  easier  than  the  previous  one. 

Communication  Tools  Overview 
Part  2 

This  was  easy  to  understand  because  it  was  just  telling  about  the  links  and 
how  they  are  similar  to  other  things  we  use  everyday. 

Web  CT  Mail  Overview 

I think  this  is  very  basic,  and  I would  expect  most  people  to  know  it,  but  if 
they  didn't  this  would  help  them. 

Again,  this  was  easy  to  understand  because  it  was  an  overview  and  because 
it  works  basically  the  same  way  email  does. 

Sending  Mail  through  Web  CT 

This  was  easy  to  understand  because  it  works  Just  like  regular  email. 
Anyone  who  has  used  email  will  be  able  to  understand  it. 

Using  Bulletin  Boards  Part  1 

Good  timing  on  the  videos.  They  are  not  long  and  don't  make  me  feel 
overwhelmed. 

Because  this  is  similar  to  email  it  is  very  easy  to  understand.  It's  still  fuzzy 
at  times. 

Using  Bulletin  Boards  Part  2 

This  was  short  and  easy  to  understand. 

Using  Bulletin  Boards  Part  3 

This  worked  well  because  it  was  explained  twice.  I could  see  what  I was 
doing  and  it  was  immediately  reflected  on  the  screen. 

Starting  Chat 

I have  never  been  in  or  used  a chat  room  so  I don't  know  if  this  is  similar  but 
it  was  easy  for  me  to  understand. 

Using  Chat 

This  was  easy  to  understand  as  well.  It  was  similar  to  email  and  other 
functions  on  WebCT. 

Table  4.  Trainee  Commentary  About  Instructional  Videos 

Discussion 

Trainees’  Use  of  the  Videos 

The  researchers  were  in  interested  in  examining  how  trainees  used  the  instructional  videos.  They  speculated  that  trainees 
might  watch  them  while  working  in  WebCT.  For  example,  a trainee  developing  content  in  WebCT  could  open  two  browser 
windows,  one  of  which  contained  a video  and  the  other  the  WebCT  workspace.  He  could  then  watch  a video  as  needed  while 
working. 

Overall,  trainees  did  not  use  the  videos  when  working.  They  watched  a video  and,  after  it  ended,  opened  the  WebCT 
workspace  to  perform  tasks  without  returning  or  reviewing  the  video.  In  a few  isolated  instances,  they  left  the  WebCT  workspace 
to  review  a video  only  when  an  alternative  solution  to  a task  could  not  be  found.  When  encountering  a problem,  trainees 
attempted  to  find  a solution  within  the  WebCT  workspace  (e.g.,  clicking  WebCT  options,  etc.)  prior  to  reviewing  a video. 

Trainees’  Reactions  to  the  Videos 

Trainees  perceived  that  their  learning  increased  as  a result  of  the  instructional  videos.  From  the  video  recordings,  the 
researchers  made  two  observations  that  appear  to  support  this  perception.  First,  trainees,  when  performing  WebCT  tasks,  did  so 
in  the  same  sequence  and  using  the  same  input  information  as  presented  in  videos.  Second,  there  are  several  functions  in  WebCT 
that,  in  the  researchers’  view,  are  not  obvious  to  uscts.  When  the  videos  presented  such  tasks,  trainees  performed  them.  For 
example,  the  chat  room  video  demonstrated  how  to  send  a private  chat  message,  a feature  not  always  apparent  to  users.  When 
trainees  performed  tasks  associated  with  the  chat  room,  they  each  sent  a private  message.  Had  trainees  not  viewed  the  video, 
they  would  not  have  known  about  this  task  and  how  to  perform  it. 

Trainees’  perception  that  they  learned  from  the  videos,  their  high  ratings  of  them,  and  the  researchers’  observation  pertaining 
to  the  potential  of  the  videos  to  support  learning  suggests  that  this  mode  of  WebCT  training  is  beneficial.  Moreover,  the  videos 
can  be  created  in  a short  development  cycle  without  excessive  time  demands  being  placed  on  the  developers. 
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The  Video  SjJit-Screen  Evaluation  Method 

The  researchers  observed  that  the  video-spit -screen  evaluation  technique  provided  several  advantages.  It  enabled  them  to 
trace  (visually  and  aurally)  the  actions  taken  by  trainees  and  to  monitor  their  use  of  the  Web-based  videos,  including  time  on  task, 
navigation  of  landscape,  and  options  selected.  At  one  point,  a trainee  could  not  get  a WebCT  chat  room  to  open.  Using  the  video, 
the  researchers  retraced  the  trainee’s  actions.  There  were  able  to  view  his  verbal  and  non-verbal  response  and  frustration  with 
being  unable  to  open  the  room.  They  reconstructed  the  actions  he  took  attempting  to  open  the  room  and  the  length  of  time  he 
spent  on  this  task  before  asking  for  help.  They  were  also  able  to  observe  distractions  or  the  series  of  actions  that  led  to 
unsuccessful  results  and  to  make  informed  judgments  (based  on  verbalizations)  about  the  trainee’s  decision  making. 

The  researchers  felt  that  the  video-split -screen  technique  was  a useful  data  collection  method  but  they  noted  three 
disadvantages.  First,  it  requires  evaluators  to  invest  much  time  in  data  analysis.  The  split-screen  method  collects  a large  amount 
of  data  in  multiple  media  formats.  Proper  analysis  of  them  requires  time,  which  may  be  prohibitive  for  some  Web  developers. 
Second,  the  approach  requires  specialized  video  and  computing  hardware  that  some  developers  do  not  own.  Third,  untrained 
evaluators  need  guidance  or  training  in  how  to  conduct  evaluations.  Only  after  a review  of  the  video  recordings  did  the 
researchers  realized  the  extent  to  which  their  comments  served  as  prompts  to  trainees  and,  in  some  cases,  directed  their  actions. 
Untrained  evaluators  must  ensure  their  questioning  and  prompting  do  not  lead  users. 

Summary  and  Further  Research 

As  described  in  this  paper,  the  instructional  video  training  appears  to  be  a viable  WebCT  resource.  An  extensive 
development  effort 

was  not  needed  to  produce  a worthwhile  product  that  is  easily  modified  and  updated.  These  facts  coupled  with  trainees’  positive 
reactions  to  them  suggest  that  the  videos  hold  potential  for  training. 

All  the  videos  used  in  this  study  were  streamed,  which  worked  well  since  trainees  had  fast  Internet  connections.  However,  as 
training  audiences  extend  to  off-campus  locations,  slow  Internet  connections  will,  to  a great  extent,  prohibit  video  training. 
Alternative  delivery  formats  such  as  DVD  and  CD-ROM  will  be  needed  to  reach  geographically  dispersed  trainees. 

This  study  was  an  initial  attempt  to  observe  trainees  using  and  reacting  to  selected  instructional  videos  and,  in  this  regard,  the 
video-split-screen  technique  yielded  much  data.  The  work  presented  here  needs  to  be  extended  to  more  fully  analyze  the 
information  collected  and  its  implications  for  the  design  and  delivery  of  video  training.  The  video-split-screen  technique  presents 
the  possibility  of  making  more  in-depth  analysis  of  not  only  software  evaluations  but  trainees’  information  processing  and 
decision  making  as  they  are  engaged  with  software. 
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Abstract 

Eighteen  students  were  equally  divided:  Group  A had  textbooks  while  Group  B did  not  have  textbooks.  “A  " learned  the 
course  material  via  the  text  and  lectures.  “5"  accessed  instructor  notes,  lecture  slides,  and  future  probable  test  items  via  the 
instructors  ’ Intranet  program  called  ^'Blackboard.  ” Prior  to  each  examination,  the  instructor  presented  key  words  that  had  been 
thoroughly  discussed.  The  students  had  to  explain  each  item  in  one  sentence  demonstrating  understanding.  The  midterm 
examination  had  sixty-two  items  valued  at  three  points  apiece,  totaling  186.  The  average  score  for  Group  A was  132  while 
Group  B averaged  164.  On  the  seventy-item,  210-point  final  examination.  Group  A averaged  156  while  Group  B averaged  188 
points.  Hence,  using  different  approaches  for  student  learning  does  make  a difference.  Providing  student  access  via  an  Intranet 
to  the  instructors  material  and  allowing  pre-lecture  knowledge  of  probable  examination  items  significantly  improves 
examination  results  and  student  learning. 

Introduction 

Normally,  all  students  are  taught  and  treated  equally  to  avoid  any  hint  of  discrimination  in  the  classroom.  Those  students  in 
grammar  school,  however,  are  often  subdivided  into  groups  or  categories  based  upon  their  level  of  knowledge,  skills  or  ability  to 
learn.  Then  the  teacher  endeavors  to  teach  the  students  according  to  their  level  or  group.  Evidence*’^  has  shown  this  method  to 
be  quite  successful  even  though  the  students  in  the  lower  level  groups  may  only  be  expected  to  learn  a small  percentage  of  those 
in  the  highest  group.  The  students  are  graded  based  upon  the  teacher’s  expectations  for  those  within  each  distinctive  group. 

Other  studies  concerning  group  subdivisions  in  the  classroom  deal  with  what  is  called  the  “self-fulfilling  prophecy.”^  In  these 
cases  the  teacher  does  not  actually  subdivide  the  students  and  usually  believes  he/she  actually  treats  all  the  students  equally.  But, 
mentally,  those  students  believed  to  be  superior  are  actually  dealt  with  differently  and  their  work  performance  is  measurably 
better. 

No  previous  studies  can  be  found  where  a class  of  “equally  qualified”  students  is  taught  identically  but  pre-class  information 
differs  between  groups  of  students.  Such  a study  was  undertaken  and  the  results  are  here  in  reported.  Eighteen  senior  or  graduate 
students  were  enrolled  in  the  “Organizational  Effectiveness”  course.  Spring  2000,  at  Arizona  State  University.  This  course 
investigated  all  aspects  of  human  behavior  in  organizations,  relevant  theories  to  help  interpret  and  understand  such  behavior,  and 
probable  effectiveness  of  both  the  organization  and  individuals.  The  author,  a visiting  professor  at  Arizona  State  University, 
discovered  on  the  first  night  of  class  that  the  students  were  equally  divided:  nine  students  already  had  their  textbooks  by  the  same 
name  as  the  course  and  nine  students  had  not  purchased  their  texts  yet.  This  was  an  ideal  situation  for  this  research  project 
although  it  may  be  classified  as  controversial.  Another  commonality  was  that  both  groups  were  equally  comprised  of  one  mid- 
twenties full-time  graduate  student  and  eight  adult  learners  averaging  thirty-two  years  of  age  and  employed. 

The  class  was  informed  of  the  overall  objectives  of  the  course  and  the  andragogy  method  to  be  used  throughout  the  semester. 
The  instructor  would  use  extensive  PowerPoint  slides  that  directly  corresponded  to  the  textbook  content.  Rather  than  using  the 
typical  lecture  mode,  an  open  discussion  methodology  would  be  followed.  Through  this  discussion  each  student  should  master 
the  true  meaning  and  possible  applications  of  each  topic  area.  In  fact,  they  should  be  able  to  thoroughly  explain  each  topic  using 
simple  and  easily  understood  language.  This  technique  supports  the  instructor’s  belief  “If  the  students  have  not  learned,  the 
instructor  has  not  taught,”  and  his  definition  of  communication  that  is  “The  equal  responsibility  of  both  the  sender  and  listener  for 
the  guarantee  of  mutual  understanding.”  The  students  were  also  informed  that  their  mid-term  and  final  examinations  would 
consist  of  key  topical  words  or  names  of  individuals  discussed  in  the  class.  They  would  have  to  thoroughly  define  or  explain 
each  item  in  one  sentence  without  using  a typical  textbook  “What”  answer.  In  their  responses  they  would  have  to  convey 
complete  understanding  of  the  word,  topic  or  person.  Further,  during  the  week  prior  to  each  examination,  the  list  of  words  and 
names  would  be  presented.  The  class  would  approve  of  each  item  for  inclusion  in  the  examination.  The  basis  for  inclusion  or 
omission  would  be  the  thoroughness  of  discussion  during  the  class  sessions.  If  a majority  of  the  students  voted  in  favor  of  an 
item,  it  was  included  in  the  examination. 

Dividing  The  Groups 

The  nine  students  who  already  had  their  textbooks  were  not  told  anything  unusual  about  their  expectations.  The  course 
would  follow  the  thirteen  chapters  in  the  600-plus-page  textbook.  They  were  to  read  the  material  prior  to  each  class  so  they 
would  be  prepared  for  discussion.  They  would  notice  that  all  the  PowerPoint  slides  used  in  the  class  corresponded  to  similar 
material  in  the  textbooks;  however,  not  all  the  tables,  figures,  charts,  etcetera  would  be  used.  The  nine  students  who  did  not  have 
their  textbooks  were  given  two  options:  (1)  Buy  the  $121.00  book  and  learn  from  it  as  usual  or  (2)  choose  not  to  buy  a textbook 
and  obtain  the  course  material  from  the  instructor’s  Intranet  “Blackboard”  site.  They  would  be  able  to  access  the  instructor’s 
presentation  and  discussion  notes,  copies  of  all  the  PowerPoint  slides  and  a listing  of  the  key  topical  words  and  people  prior  to 
each  class  session.  All  nine  students  opted  for  using  the  Intranet.  They  were  then  provided  with  confidential  access  codes  and 
information  sheets  explaining  how  to  use  the  “Blackboard”  system. 


The  Blackboard  System 

Blackboard  Courselnfo^'”'^  is  an  online  teaching  and  learning  server  software  product.  Its  purpose  is  to  provide  distance 
learning  as  an  enhancement  to  traditional  classroom  instruction.  Figure  1 shows  Blackboard  Courseinfo’s  four  core  features. 


Content  Management : 

Posting  course  documents  and  assignments, 
staff  information,  and  incorporating  files 
from  Microsoft  Word,  Excel  or  PowerPoint. 

Communication  and  Collaboration: 

Instructor  and  student  interaction. 

Mutual  learning  via  discussion  boards, 
real-time  chat  opportunities,  and 
virtual  office  hours. 

Assessment: 

Create  tests,  quizzes  and  surveys. 
Password-protected  examinations,  and 
Performance  feedback. 

Administration: 

Instructor  Control  Panel  access. 

Control  of  course  content,  communication, 
assessment,  and  user  management  via 
security  permissions,  and  tracking  student 
usage 


Figure  L Blackboard  Courseinfo  Core  Features 

The  instructor  used  all  of  these  options  other  than  Assessment  since  only  one-half  of  the  students  had  access  to  this  Intranet 
information.  All  course  information  was  provided  using  this  site  including  the  syllabus,  course  purpose  and  objectives,  schedule, 
PowerPoint  slides,  discussion  notes,  key  words  and  people  as  probable  examination  items  and  other  pertinent  student 
information.  Grades  and  class  ranking  was  included,  A major  feature  of  BlackboardCourseInfo  that  was  a great  benefit  to  the 
instructor  was  the  virtual  office  hours.  Eight  of  the  nine  students  using  this  system  were  full-time  employed  and  were  often 
unable  to  schedule  meetings  with  the  instructor.  They  regularly  used  the  virtual  meeting  aspect  of  the  system,  that  was  a 
confidential  threaded  discussion  board  one-on-one  “chat,” 

The  Mid-Term 

During  the  week  prior  to  the  mid-term  examination  the  instructor  presented  all  the  key  topical  words  and  people  that  had  been 
discussed  since  the  beginning  of  the  course.  Each  item  was  again  considered,  questions  pertaining  to  each  were  answered,  and 
the  students  voted  on  each  items’  inclusion  or  omission  to  the  test.  The  majority  ruled.  If  one-half  voted  to  omit  an  item,  it  was 
not  included  in  the  examination.  The  remainder  of  the  items  constituted  the  test.  The  students  knew  the  procedure  and  scoring 
for  this  examination.  They  had  to  demonstrate  their  degree  of  “understanding”’  of  each  item  in  one  sentence,  A textbook 
“What”  answer  would  not  gamer  any  points  since  it  would  not  show  understanding. 

The  mid-term  test  consisted  of  sixty-two  items  each  having  a value  of  three  points.  Therefore,  the  total  points  for  the  mid- 
term examination  was  1 86,  The  two  groups  were  labeled  by  the  instructor  Group  A,  the  textbook  group,  and  Group  B,  the 
Intranet  group.  Table  1 shows  the  mid-term  test  results. 


Total  Possible 

Average  Score 

% 

GROUP  A 

186 

132 

71 

GROUP  B 

186 

164 

88 

Table  I.  Mid-term  Examination  Scores 

Group  B did  significantly  better  than  Group  A by  an  average  of  thirty-two  points.  Also,  the  range  of  scores  differed  greatly 
for  the  two  groups.  Group  B ranged  from  a low  of  153  to  a high  of  183;  a difference  of  thirty  points.  Group  A had  a low  of  96 
and  high  of  154  for  a range  of  fifty-eight  points.  The  instructor’s  initial  prediction  based  upon  andragogy  principles  was 
accurate.  Students  who  can  organize  and  plan  their  own  learning,  with  guidance,  will  do  better  than  those  who  are  required  to 
follow  a prescribed  pedagogy  method:  read,  take  notes,  memorize,  and  then  forget  following  the  examination. 

At  the  next  class  session  the  mid-term  examination  results  were  told  to  the  whole  class.  The  instructor  informed  the  students 
in  Group  A about  Group  B and  how  they  were  learning.  The  instructor  avoided  uproar,  or  charges  of  discrimination,  by  assuring 
all  Group  A students  that  their  examination  scores  would  be  adjusted  at  the  end  of  the  semester  following  the  final  examination. 
However,  they  were  encouraged  to  excel  because  they  had  a lot  of  points  between  themselves  and  those  students  in  Group  B. 
None  of  the  students  complained  because  of  the  assured  adjustment  with  a guarantee  of  fairness. 


Final  Examination 

The  results  of  the  final  examination  were  almost  identical  to  the  mid-term  examination.  There  were  seventy  items  approved 
by  the  class.  Again,  each  item  was  valued  at  three  points  for  a total  of  210.  Table  2 shows  the  results  for  the  two  Groups. 


Total  Possible 

Average  Score 

% 

GROUP  A 

210 

156 

74 

GROUP  B 

210 

188 

90 

Table  2.  Final  Examination  Results 

The  average  difference  between  the  two  groups  was  identical  - thirty-two  points.  The  range  for  Group  A was  much  less  than 
in  the  mid-term  examination.  Group  A scores  ranged  from  a low  of  142  to  a high  of  178  for  an  overall  range  of  thirty-sbc.  Yes, 
the  lowest  and  highest  Group  A students  were  the  same  on  both  the  mid-term  and  final  examinations.  The  range  for  the  Group  B 
students  was  much  narrower:  a low  of  178  and  a high  of  200. 

Table  3 shows  the  combined  results  of  the  mid-term  and  the  final  examinations.  Also  shown  are  the  adjusted  Group  A scores. 
The  instructor  added  thirty  points  to  each  student’s  mid-term  and  final  examination  scores. 

Total  Possible  Average  Score  % 


GROUP  A 

396 

288 

73 

GROUP  B 

396 

352 

89 

GROUP  A 

396 

348 

88 

(Adjusted) 

Table  3.  Overall  Examination  Results 

The  instructor  equalized  the  scores  as  closely  as  possible.  The  Group  A student  who  had  the  high -test  score  in  that  group  was 
within  thirty -two  points  of  a perfect  score  on  both  the  mid-term  and  final  examinations.  That  student,  incidentally,  completed  the 
course  with  the  highest  test  score  average.  The  applied  adjustment  and  exactly  how  it  was  reached  was  thoroughly  explained  to 
the  students.  They  all  expressed  their  satisfaction. 


Conclusion 

The  results  of  this  study  are  highly  conclusive  even  though  the  sample  size  was  quite  small.  Different  approaches  for  student 
learning  does  make  a difference.  Those  students  who  had  access  via  the  instructor’s  Intranet  site  and  could  utilize  actual  course 
material  scored  significantly  better  than  those  students  who  followed  the  typical  ‘‘learn  from  the  textbook”  approach.  All 
students  participated  in  the  discussions  and  helped  equally  to  determine  examination  content.  But,  those  students  who  had  access 


to  the  instructor’s  discussion  notes  and  prior  knowledge  of  probable  examination  items  before  each  class  could  isolate  their 
concentration  on  the  most  pertinent  and  valuable  content.  Their  learning  was  under  their  individual  control. 

Although  the  sample  size  of  this  individual  study  was  quite  small,  the  author  feels  quite  strongly  about  these  results  and  his 
personal  experiences  such  that  he  strongly  recommends  instructors  reevaluate  how  they  are  providing  learning  opportunities  for 
their  students.  With  an  Intranet  system  similar  to  Blackboard,  the  burden  of  learning  is  transferred  to  its  rightful  source  with  the 
students.  Another  benefit  is  the  virtual  office  hours  utilizing  on-line  one-on-one  chat  room  capabilities.  This  student  preparation 
process  can  be  used  for  virtually  every  course  regardless  of  its  content.  The  ultimate  objective  is  for  the  students  to  learn  at  their 
maximum  potential. 

A word  of  caution  must  be  given,  however.  Any  instructor  wanting  to  a similar  study  must  guard  against  any  possible 
discriminatory  practices.  Any  and  all  groups  must  be  treated  equally  in  the  final  grading. 
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Abstract 

Combining  assessment  and  research  components  on  a large  development  and  research  project  is  a complex  task.  There  are 
many  descriptions  of  how  either  assessment  or  research  should  be  conducted,  but  detailed  examples  illustrating  integration  of 
such  strategies  in  complex  projects  are  scarce.  This  paper  provides  definitions  of  assessment,  evaluation,  and  research  and 
illumination  on  how  to  strategically  integrate  and  manage  assessment  and  research  activities  by  providing  examples  of  tools 
used  to  develop  and  manage  a comprehensive  assessment  and  research  plan  for  Kids  as  Airborne  Mission  Scientists  (KaAMS),  a 
large  instructional  development  and  research  project. 


Introduction 

Kids  as  Airborne  Mission  Scientists  (KaAMS)  is  a large  development  and  research  project  funded  by  the  National 
Aeronautics  and  Space  Administration’s  (NASA)  Leading  Educators  to  Applications,  Research,  and  NASA -Related  Educational 
Resources  in  Science  (LEARNERS)  project.  The  purpose  of  the  KaAMS  project  was  to  develop  a series  of  teacher  resources, 
framed  in  interdisciplinary  problem-based  learning  approaches  integrating  authentic  and  ill-structured  problem  situations, 
inspiring  teachers  to  inspire  students  to  pursue  the  sciences.  The  goal  of  the  project  was  to  develop  informed  instructional 
materials  and  assess  their  impact  on  the  target  audiences,  middle  school  teachers  and  students.  Thus,  the  project  proposal 
included  a strong  commitment  to  formative  and  summative  evaluation  as  well  as  a research  component. 

Definitions 

Assessment  is  an  umbrella  term  that  refers  to  the  systematic  study  of  a problem  or  innovation  to  make  effective  decisions 
about  what  should  happen  (Rossett,  1987).  In  the  case  of  the  KaAMS  project,  the  assessment  plan  defined  the  objectives, 
strategies,  tools,  and  protocols  used  to  collected  formative  and  summative  feedback. 

The  formative  evaluation  component  of  the  assessment  plan  answered  the  question  liow  are  we  doing?’  as  instructional 
materials  were  being  developed  (Morrison,  Ross  & Kemp,  2001).  The  focus  of  formative  evaluation  was  on  measuring  the 
effectiveness,  efficiency,  usability,  and  acceptability  of  the  material  produced  so  that  sound  instructional  design  decisions  could 
be  made  as  the  materials  were  developed  and  finalized.  Feedback  collected  during  formative  evaluation  served  the  function  of 
informing  the  development  team  how  well  each  of  the  instructional  materials  were  serving  the  instructional  objectives  and  what 
should  be  done  to  enhance  them. 

The  summative  evaluation  procedures  were  designed  to  answer  the  question  ‘how  did  we  do?’  after  instructional  materials 
had  been  implemented  with  the  target  audience  (Morrison,  Ross  & Kemp,  2001).  Summative  feedback  focused  specifically  on 
the  degree  to  which  the  major  instructional  outcomes  were  attained  as  a result  of  using  KaAMS  materials.  Summative  evaluation 
addressed  (1)  reaction,  (2)  learning  gains,  (3)  performance  changes,  (4)  education  system  changes,  and  (5)  impact  on  the  greater 
society.  This  feedback  was  useful  in  developing  implementation  plans  and  supporting  materials  to  train  new  teachers  in  the  use  of 
KaAMS  materials. 

Research,  on  the  other  hand,  is  a process  of  identifying  something  unknown  and  then  collecting  data  to  make  it  known  (Gall, 
Borg  & Gall,  1996).  In  the  case  of  KaAMS,  the  research  components  focused  on  analyzing  the  effect  that  using  KaAMS  had  on 
the  stakeholders  in  the  middle  school  learning  environment.  For  example,  it  was  unknown  how  teaching  strategies  would  be 
affected  by  introducing  KaAMS  into  the  classroom  or  whether  the  use  of  these  materials  would  affect  student  interest  in  pursuing 
science.  Answering  such  questions  can  help  researchers  develop  a better  understanding  of  the  types  of  materials  and  activities 
that  affect  instructional  and  learning  processes. 

Thus,  the  purpose  of  assessment  is  to  gather  feedback  that  will  inform  the  instructional  design  process.  The  dangers  of  having 
incomplete  and  poorly  thought  out  assessment  strategies  are  that  either  required  feedback  is  not  collected  or  time  is  wasted 
collecting  feedback  that  is  not  important  to  design -enhancement  decisions.  Thus,  the  focus  of  a good  assessment  strategy  must 
include  plans  and  instruments  for  collecting  feedback  that  is  essential  to  drawing  conclusions  about  what  is  needed  to  develop 
great  instruction.  The  purpose  of  research  is  to  gather  data  on  the  impact  that  instructional  materials  have  on  stakeholders  and  the 
instructional  and  learning  environment.  Poor  planning  of  the  research  data  collection  process  can  result  in  obsolete  data,  lost 
opportunities  to  collect  needed  data,  or  data  collection  conditions  that  interfere  with  gathering  reliable  data.  Thus,  strategic 
collection  of  research  data  must  also  be  tied  to  the  development  process,  based  on  the  key  research  questions,  provide  flexibility 
for  adjusting  to  new  learnings  during  product  development  phases,  and  seek  data  that  can  test  theoretical  assumptions. 


Although  assessment  and  research  literature  provides  guidelines  to  develop,  manage,  and  conduct  assessment  or  research, 
comprehensive  examples  illustrating  how  to  integrate  assessment  and  research  strategies,  methods,  and  tools  could  not  be  found 
in  the  literature.  Published  literature  on  large-scale  development  and  research  projects  usually  contained  only  brief  descriptions  of 
certain  components  of  assessment  and  research  often  neglecting  to  provide  detailed  descriptions  of  the  relationship  among 
development  cycles,  feedback  and  data  collection  procedures,  and  assessment  and  research  tools.  Therefore,  decision-making 
about  design  and  implementation  issues  can  be  problematic  if  the  right  amount  or  right  kinds  of  feedback  and  data  are  not 
collected.  Examples  that  illustrate  the  integration  of  assessment  and  research  strategies  and  tools  are  scarce.  Sharing  such 
examples  is  therefore  beneficial  to  others  developing  comprehensive  and  targeted  assessment  and  research  plans.  The  remainder 
of  this  paper  will  describe  and  provide  examples  of  the  tools  and  procedures  developed  to  manage  and  conduct  assessment  and 
research  on  the  KaAMS  project. 

Strategic  Planning  of  Assessment  and  Research  Cycles 

The  challenge  in  creating  a strategic  approach  to  assessment  and  research  began  with  identifying  the  formative  and  research 
needs  of  the  project.  The  literature  on  instructional  development  formative  and  summative  evaluation  provided  guidelines  for 
determining  the  types  of  feedback  needed  to  develop  sound  instructional  materials  (Dick  and  Cary,  1985).  The  scope  of  the 
KaAMS  project  helped  to  determine  who,  e.g.,  teachers,  administrators,  students,  curriculum  specialists,  would  best  be  able  to 
provide  such  feedback.  Thus,  during  the  development  of  the  KaAMS  lesson  plans  feedback  was  sough  from  middle  school 
administrators,  teachers,  and  students  on  the  layout  of  the  material,  background  resources,  terminology,  appropriateness  of 
activities,  time  dedicated  to  preparation,  impressions  after  using  the  materials,  and  thoughts  on  what  worked  and  what  did  not 
during  the  use  of  the  materials.  This  feedback  was  gathered  during  and  immediately  after  the  initial  classroom  testing  of  KaAMS. 

The  research  questions  were  drawn  in-part  from  the  purpose  of  the  grant,  a literature  review  on  problem-based  learning, 
technology  integration  in  the  schools,  and  career  development  in  adolescents  and  in-part  based  on  the  researchers  interests.  The 
research  questions  probed  for  understanding  on  the  effects  of  KaAMS  on:  (1)  teachers  use  of  KaAMS  and  NASA  resources  in 
their  classrooms,  (2)  changes  in  teaching  practices  during  and  after  using  KaAMS  materials,  (3)  changes  in  students’  success 
rates  in  science,  (4)  changes  in  students’  interests  in  pursuing  science,  and  (5)  diffusion  of  KaAMS  and  NASA  materials  to  the 
surrounding  educational  environment. 

Three  cycles  of  development  and  data  collection 

Being  a large  development  project,  lasting  three  years  and  consisting  of  collaborations  among  two  major  universities,  several 
NASA  scientists,  and  several  schools,  the  KaAMS  products  were  developed  in  three  cycles:  alpha,  beta,  and  final  versions.  The 
frameworks  for  the  PBL  units  and  major  lesson  plan  components  were  developed  during  the  alpha  cycle.  The  alpha  version  was 
developed  based  on  collaboration  among  the  KaAMS  project  team  that  included  a group  of  advising  middle  school  teachers  who 
reviewed  and  classroom  tested  the  KaAMS  materials.  Assessment  of  the  alpha  materials  focused  on  the  appropriateness  of  the 
level  of  content  and  activity  for  the  students,  background  information  and  instruction  for  teachers,  and  general  organization  of 
lesson  plans  and  associated  lesson  components. 

The  beta  cycle  focuses  on  using  developing  a beta  version  of  KaAMS  that  incorporates  learnings  from  alpha  testing, 
including  modifications  needed  to  activities  and  development  of  support  materials  to  meet  the  needs  of  teachers.  Beta  testing  is 
conducted  with  a new  set  of  teachers  from  a broader  group  of  teachers  and  students  spread  across  three  states.  Beta  testing 
focuses  on  the  usefulness  of  the  completed  product  and  how  teachers  integrate  the  lesson  plans  or  lesson  components  into  their 
curriculum.  This  feedback  helps  to  finalize  the  KaAMS  products  and  teacher  support  resources  and  develop  national  distribution 
plans.  Simultaneously,  research  data  were,  or  will  be,  collected  to  establish  measurement  baselines  and  gather  further  data  on  the 
effects  of  KaAMS  on  teachers,  students,  and  the  surrounding  community.  During  Alpha  testing  the  research  instruments, 
protocols,  and  logistics  were  be  tested. 

The  assessment  and  research  matrix 

The  first  step  in  developing  the  overall  assessment  plan  was  to  document  and  examine  the  assessment  and  research  needs. 
The  assessment  and  research  questions  were  placed  in  a matrix  to  develop  a strategic  view  of  the  feedback  and  data  requirements 
for  the  project.  The  formative  and  summative  feedback  needs  were  cross-referenced  to  the  data  collection  instruments  and 
questions  by  subject,  e.g.,  teacher,  student.  See  Table  1 . 
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Table  1.  Example  from  KaAMS  Research  and  Assessment  Matrix:  Lesson  Plan  Development  - Formative  Evaluation 


Lesson  Plan  Development  - 

Formative  Evaluation 

Stakeholders/  Instruments  / 
questions 

impressions  of  t / , ^ 

1 . value  / needs  or  connections  to  the 

^ ^ , background  curriculum, 

terminology,  etc.  ^ , 

r*  t t resources  terminology 

of  lesson  plans 

value  / usefulness  / 
appropriateness  of 
activities  and 
resources 

TEACHERS 

\ 

l ! 

TAP  A initial  interview 

Teacher  background 
Curriculum  connections 

Q4-5 

Ql-3 

Lesson  plan  impressions 

Q6-7 

Q8 

Background  content 

QlO-13 

Classroom  use 
Discuss  KaAMS  w/others 

Q15 

LP  Follow-up  survey 

lesson  used 

Q4 

Q5 

resources  used 

Q8-9 

classroom  description 
during  lesson 

Q18 

Q17,  Q23-24 

Q 19-22 

student  success 

STUDENTS  ! i 

Student  focus  groups 
Student  classroom 

X X 

X 

X 

documents 

X 

X 

Student  journals 

X 

X 

* B-baseline,  A=immediately  after  use,  L=  long-term  follow-up. 

1 to  6 months  after  use  P-During  prep,  Q=question 

The  research  questions  were  also  cross-referenced  to  the  data  collection  instruments  and  questions  by  subject,  e.g.,  teacher, 
student.  See  Table  2. 

After  cross-referencing  all  of  the  feedback  and  data  needs  by  instrument  and  stakeholder  the  matrix  was  examined  for  gaps 
and  overlap.  Instruments  and  protocols  were  adjusted  to  collected  feedback  and  data  that  were  not  accounted  for  in  the  matrix. 
For  example,  it  was  found  that  student  opinion  and  artifacts  were  gathered  to  assess  student  success  rates,  but  teacher  feedback 
was  not  gathered  on  student  success.  Questions  were  added  to  interview  protocols  to  gather  such  feedback. 

Analysis  of  the  overlaps  indicated  that  there  were  several  instruments  and  questions  gathering  feedback  on  the  materials. 
Since  this  goal  of  the  project  was  to  develop  new  instructional  materials  it  was  agreed  that  the  level  of  overlap,  especially  since 
feedback  was  from  multiple  perspectives  of  the  teachers  and  students,  was  appropriate.  To  ensure  that  the  data  collection  was  not 
too  obtrusive  to  the  learning  environment,  the  next  step  was  to  plan  the  timing  for  administration  cf  the  data  collection 
instruments  and  protocols. 

The  assessment  strategies  protocols  called  for  continual  gathering  of  feedback  from  teachers  and  students  through  multiple 
strategies,  e.g.  surveys,  interviews,  and  observations.  The  KaAMS  product  development  plan  laid  out  the  calendar  timing  for  the 
development  and  testing  of  each  component  of  the  instructional  materials.  Examining  each  instrument  and  protocol  provided  an 
estimate  of  how  long  each  stakeholder  would  be  required  to  provide  solicited  feedback  and  data.  Thus,  the  timing,  by  calendar 
date,  and  timing  in  terms  of  stakeholder  commitment  in  providing  feedback  through  the  various  instruments  were  examined. 
Time  frames  were  established  for  administering  data  collection  instruments  based  on  the  project  development  timelines  and 
collection  patterns  for  the  most  appropriate  data  that  would  be  least  invasive  to  those  involved.  These  estimated  timelines  were 
added  to  the  KaAMS  Research  and  Assessment  matrix.  See  Table  2,  freq  columns. 


Table  2.  Example  from  KaAMS  Research  and  Assessment  Matrix:  Research 


KaAMS  Research 


How  are 

How  are  student 

How  are 

How  are 

student 

levels  of  interest  in 

teachers  using 

teachers 

success  rates 

pursuing  science- 

Stakeholders/ 

KaAMS  and 

* 

changing  their 

in  science 

* 

related  career 

* 

Instruments  / 

NASA 

fre 

teaching 

* changing  over 

fre 

changing  over 

fre 

questions 

resources? 

q 

practices? 

freq  time? 

q 

time? 

q 

TEACHERS 


Electronic  survey 

Q103,Q114- 

115,Q122- 

BA 

demographics 

136 

L 

teacher  attitude 

Ql-31 

BA 

L 

percept,  of  env. 

BA 

for  web 

Q32-52 

L 

methods  of 

Q53-78, 

BA 

teaching 

Q112-113 

L 

Q79-102, 

Q104-110, 

BA 

Resource  use 

Qlll 

L 

Importance  of 
NASA,  Aero,RS 

BA 

to  curric. 

Q116-121 

L 

LP  Follow-up 
survey 
lesson  used 

resources  used  Q8-9 

classroom 
description  during 
lesson 

student  success 

Classroom 

Observations  X D X D_ 

* B=baseline,  A=immediately  after  use,  L=  long-term  follow-up,  1 to  6 months  after  use  P-During  prep,  Q=question 

One  additional  benefit  of  the  strategic  matrix  was  in  providing  an  overall  picture  of  the  project  development,  assessment,  and 
research  components.  Thorough  review  of  the  matrix  provided  insights  into  the  complexity  of  the  research  agenda.  This 
prompted  the  development  of  a strategy  to  use  the  alpha  testing  cycle  to  test  the  research  protocols.  This  would  allow  the  team  to 
refine  logistics  for  data  collection  and  identify  initial  findings  related  to  the  effects  of  using  the  KaAMS  materials  with  middle 
school  populations.  However,  it  is  important  to  report  such  findings  with  caution,  since  effects  may  be  related  to  additional 
attention  from  product  development  teams. 

Additional  Planning  and  Managing  Tools 

The  strategic  planning  matrix  provided  the  framework  for  a comprehensive  Assessment  Plan  to  be  used  by  the  project  team 
to  manage  the  assessment  and  research  processes.  The  components  of  the  Assessment  Plan  included  text  and  graphic 
representations  of  the  overall  assessment  strategy,  teacher  advisory  panels  for  alpha  and  beta  testing,  roles  and  responsibilities  of 
assessment  team  members,  components  of  each  evaluation  and  research  procedure,  development  and  data  collection  timelines, 
selection  criteria  for  participants,  forms  for  tracking  procedural  issues,  copies  of  all  instruments  and  protocols,  and  human 
subjects  approval  forms.  The  Assessment  Plan  was  developed  so  that  each  component  was  tied  to  the  research  matrix  supporting 
the  data  collection  needs  while  keeping  the  amount  of  disruption  to  subjects  at  a minimum. 

The  strategic  planning  matrix  also  provided  the  framework  for  creating  a database  for  all  data  collected  during  the  assessment 
and  research  phases.  In  this  case,  a spreadsheet  was  developed  with  multiple  worksheets.  Each  worksheet  was  titled  based  on  the 
instrument,  stakeholder  and  type  of  data  it  collected.  .This  naming  structure  ultimately  helped  in  the  data  analysis  processes. 

The  final  assessment  and  research  report  was  also  based  on  the  strategic  planning  matrix.  The  matrix  provided  an  overview  of 
the  data  requirements  in  response  to  the  evaluation  and  research  questions.  This  overview  was  used  as  the  planning  structure  to 
analyze  the  data  collected  and  to  report  on  findings  based  on  the  project  questions.  The  final  assessment  and  research  report 


Q10,Q16, 

Q27  A Q19-22  A 

Q13,Q15  A 


included  text  and  graphic  representations  for  an  executive  summary  of  the  assessment  and  research  findings,  a short  summary  of 
the  project  goals  and  assessment  aid  research  objectives,  summary  of  the  formative  and  summative  evaluation  feedback  and 
recommendations  for  each  component  of  the  KaAMS  material,  tables  and  figures  describing  results,  summaries  of  the 
observations  and  data  collected  from  each  classroom,  baseline  and  subsequent  description  of  each  stakeholder  in  the  classroom 
trials,  and  appendices  that  described  each  of  the  lesson  plans  tested,  detailed  notes  from  observations,  interviews,  and  focus 
groups,  and  selected  printouts  of  data  analysis  on  stakeholders.  Thus,  the  information  in  the  summary  report  was  linked  to  the 
other  assessment  and  research  tools  developed  for  the  project,  based  on  the  framework  established  in  the  strategic  planning 
matrix. 

Conclusions 

Combining  assessment  and  research  components  on  a large  development  and  research  project  is  a complex  task.  Creating 
strategic  planning  tools  can  help  visualize  the  feedback  and  data  needs  and  provide  a framework  for  creating  overall  assessment 
and  research  plans,  data  collection  schedules  and  instruments,  and  summary  reports. 

During  KaAMS,  several  levels  of  assessment  and  research  were  conducted  and  data  were  collected  from  several  stakeholders. 
A planning  matrix  was  created  to  avoid  over-  and  under-collection  of  data  and  assure  that  the  appropriate  data  were  collected. 
The  matrix  provided  a strategic  tool  to  help  cross-reference  the  collection  of  different  data,  at  different  times,  from  multiple 
stakeholders.  It  helped  to  identify  weaknesses  and  strengt  hs  of  the  assessment  strategies  and  guided  further  development  of 
instruments  and  protocols.  The  key  benefit  of  such  as  matrix  was  in  its  value  to  provide  a framework  for  creating  an  overall 
Assessment  Plan  that  helped  coordinate  assessment  and  research  team  members  during  data  collection,  data  analysis,  and 
reporting.  This  tool,  thus,  helped  to  plan  and  manage  the  complex  task  of  gathering  the  right  type  of  feedback  and  data,  at  the 
right  time,  from  the  right  stakeholders  during  a large,  multi-year  development  and  research  project. 
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Abstract 

This  research  was  recognized  with  the  2000  Robert  M.  Gagne  Dissertation  Award  from  the  AECT  Division  of  Instructional 
Development.  The  study,  both  qualitative  and  quantitative,  focused  on  gender  differences  in  the  use  of  a computer-based 
mathematics  game.  High  school  level  males  and  females  were  compared  in  their  game-playing  and  learning  strategies, 
motivation  to  use  a computer-based  mathematics  game,  beliefs  about  mathematics  and  computers,  and  mathematics 
performance.  The  evident  gender  differences  and  instructional  design  implications  are  discussed. 


Introduction 

When  computer-based  games  are  developed  for  learning,  consideration  must  be  given  to  gender.  Males  and  females  report 
different  reasons  for  motivation  to  play  computer-based  games  (Dempsey,  Lucassen,  Haynes,  & Casey,  1997;  Malone,  1981). 
Along  with  differing  views  for  motivation,  males  and  females  report  differences  in  the  use  of  metacognitive  and  cognitive 
strategies.  Metacognitive  strategies  are  used  to  monitor  and  regulate  progress  toward  a goal  (Flavell,  1981),  Metacognitive 
strategies  are  also  known  as  comprehension  monitoring  strategies  (Smith  & Ragan,  1999).  Cognitive  strategies  are  strategies 
selected  by  learners  to  process  information  and  solve  problems  (Gagne,  Briggs,  & Wager,  1992),  Metacognitive  strategies 
govern  the  use  of  cognitive  strategies  (Gagne,  et  al.,  1992).  Learners  can  examine  their  use  of  cognitive  and  metacognitive 
strategies  to  improve  their  performance  in  computer-based  instructional  games  (Jacobs  & Dempsey,  1 993), 

The  now  classic  ARCS  Model  of  Motivation  (Keller,  1983;  1987a;  1987b)  is  relevant  to  the  study  of  motivation  and  the  use 
of  computer  games  to  motivate  learners.  The  ARCS  Model  of  Motivation  includes  four  categories  of  conditions  that  affect  the 
motivation  to  learn.  The  four  categories  are  attention,  relevance,  confidence,  and  satisfaction. 

In  a frequently  cited  study  of  computer-based  games,  Malone  (1981)  proposed  a theory  of  intrinsically  motivating  instruction. 
Malone  developed  his  theory  to  include  three  categories:  challenge,  fantasy,  and  curiosity . 

In  this  study,  evidence  emerged  that  fit  with  Keller’s  (1983;  1987a;  1987b)  ARCS  Model  of  Motivation  and  Malone’s  (1981) 
theory  of  intrinsically  motivating  instruction.  Males  and  females  reported  different  game-playing  strategies  and  different  reasons 
for  motivation.  Consequently,  guidelines  are  needed  for  the  design  of  computer-based  instructional  games  that  are  appealing  and 
motivating  to  learners  regardless  of  gender.  Finding  exceptional  ways  to  design  and  develop  computer-based  instructional  games 
has  the  potential  to  assist  all  learners  in  improving  their  confidence  in  mathematics  ability  and  beliefs  about  mathematics  and 
computers. 

Purpose  and  Scope  of  the  Study 

The  purpose  of  this  study  was  to  compare  high  school  level  males  and  females  in  their 
game-playing  and  learning  strategies  and  their  motivation  to  use  a computer-based  mathematics  game.  In  addition,  males  and 
females  were  compared  in  their  beliefs  about  mathematics  and  the  use  of  computers.  Comparisons  were  made  also  for 
mathematics  performance. 

Gender  differences  in  learning  have  been  researched  in  various  areas  such  as  mathematics  performance  (Friedman,  1989; 
Hyde,  Fennema,  & Lamon,  1990),  attitude  toward  mathematics  (Hyde,  Fennema,  Ryan,  Frost,  & Hopp,  1990;  Kaiser-Messmer, 
1993),  and  attitude  toward  the  use  of  computers  (Shashaani,  1994;  Taylor  & Mounfield,  1994),  Gender  differences  in  the  use  of 
computer-based  games,  however,  have  not  been  examined  extensively  (Dempsey,  Lucassen,  Gilley,  & Rasmussen,  1 993-94).  In  a 
review  of  51  instructional  gaming  articles  (Dempsey,  et  al,,  1993-94),  the  authors  note  that  the  articles  were  unclear  in  reporting 
learner  characteristics  such  as  cognitive  style  or  demographics  such  as  gender,  race,  and  age.  Along  a similar  line  of  thought,  the 
American  Association  of  University  Women  [AAUW]  (1992)  points  out  that  research  reports  frequently  refer  to  students  without 
specifying  gender  and  fail  to  explore  links  between  gender  and  academic  performance. 

Method 

Participants 

The  participant  population  included  five  ninth-grade  classes  of  students  (58  females  and  52  males)  enrolled  in  Algebra  I at  a 
private  parochial  school  in  the  Southeast  region  of  the  United  States.  Entry  to  the  school  was  arranged  through  a fellow  graduate 
student  who  is  an  administrator  at  the  school.  The  classes  each  had  19  to  23  students  (N  = 1 10).  To  assure  confidentiality,  an 
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identification  code  number  was  assigned  to  all  participants.  Because  the  number  of  computers  was  limited  to  17,  participants 
were  randomly  assigned  to  work  in  pairs  with  a student  of  the  same  sex. 

Facilities 

Students  played  the  game  in  the  school  library.  The  library  had  a separate  room  with  17  IBM -compatible  PC  computers.  The 
computers  were  arranged  in  a rectangular  shape  facing  the  walls. 

Instructional  Materials 

A commercially  developed  software  program  was  used  in  the  study.  The  software,  produced  by  The  Learning  Company 
(1997),  is  called  Grade  Builder  Algebra  J (1997).  It  includes  a tutorial  and  two  games  related  to  the  algebra  topics.  One  of  the 
games  (Soak  Your  Sibling)  was  used  in  this  study  to  review  previously  learned  algebra  topics.  The  Learning  Company  donated 
the  software  for  this  study.  The  software  remained  with  the  school  upon  completion  of  the  study. 

The  treatment  was  the  use  of  a computer-based  mathematics  game  to  review  algebra  topics.  The  participants  learned  the 
algebra  topics  two  to  three  months  before  this  study  took  place.  The  game  questions  included,  for  example,  linear  equation 
problems.  For  the  linear  equation  problems,  the  participants  identified  where  an  animated  character  was  hiding  by  typing  the 
missing  values  of  the  line’s  equation. 

Research  Questions 

Six  major  questions  were  considered.  The  first  question  on  strategies  was  studied  with  the  use  of  qualitative  data.  The 
second  question  on  motivation  was  examined  with  qualitative  data  supplemented  by  quantitative  data.  The  remaining  questions 
were  answered  primarily  through  quantitative  data  with  supplemental  qualitative  data. 

Research  Question  1:  What  strategies  will  be  reported  by  males  and  females  following  the  use  of  a computer-based 
mathematics  game? 

Research  Question  2:  What  will  males  and  females  find  motivating  about  a computer-based  mathematics  game? 

Research  Question  3:  Will  there  be  a difference  between  males  and  females  in  their  beliefs  about  mathematics,  controlling 
for  prior  beliefs,  following  the  use  of  a computer-based  mathematics  game? 

Research  Question  4:  Will  there  be  a difference  between  males  and  females  in  their  beliefs  about  the  use  of  computers, 
controlling  for  prior  beliefs  and  spatial  visualization,  following  the  use  of  a computer-based 
mathematics  game? 

Research  Question  5:  Will  there  be  a difference  in  the  scores  of  males  and  females  on  an  immediate  posttest,  controlling  for 
pretest  scores,  following  the  use  of  a computer-based  mathematics  game? 

Research  Question  6:  Will  there  be  a difference  in  the  scores  of  males  and  females  on  a delayed  posttest,  controlling  for 
pretest  scores,  following  the  use  of  a computer-based  mathematics  game? 


Research  Design 

This  study  was  qualitative  and  quantitative.  The  qualitative  data  included  researcher  observations  and  focus  group 
interviews.  Interviews  are  important  because  thoughts  and  feelings  cannot  be  observed  (Patton,  1990).  In  addition  to  the 
observations  and  focus  group  interviews,  daily  Journals  and  personal  letters  written  by  students  provided  insight  into  the  students’ 
learning  and  game-playing  strategies,  as  well  as  their  motivation  to  play  the  game. 

The  quantitative  data  included  results  from  the  Visualization  in  Two-Dimensions  Test  (Flanagan,  Davis,  Dailey,  Shaycoft, 
Orr,  Goldberg,  & Neyman,  1964),  the  Fennema-Sherman  Mathematics  Attitudes  Scales  (Fennema  & Sherman,  1976),  the 
Computer  Attitude  Scale  (Shashaani,  1993),  and  a Motivational  Gaming  Scale  (Dempsey,  Lucassen,  Haynes,  & Casey,  1997). 
The  Motivational  Gaming  Scale  is  based  on  Keller’s  ARCS  Model  of  Motivation  (Keller,  1983;  1987a;  1987b).  The  scale  is  a 
modification  of  Keller’s  Instructional  Motivational  Scale  (Keller,  1987c).  The  Likert-type  scale  includes  statements  related  to 
attention,  relevance,  confidence,  and  satisfaction.  Pretest  scores,  immediate  posttest  scores,  and  delayed  posttest  scores  provided 
reports  of  mathematics  performance. 

Combining  methodologies  and  using  multiple  data  sources  is  known  as  triangulation  (Creswell,  1994).  Triangulation  helps 
the  researcher  to  “cross-check”  information  and  conclusions  by  using  multiple  procedures  or  data  sources  (Johnson,  1997).  A 
graphic  organizer  for  the  instruments  is  displayed  in  Figure  1 . 
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Independent  and  Dependent  Variables 

For  the  quantitative  analysis  in  this  study,  the  independent  variable  was  gender.  The  dependent  variables  included  beliefs  about 
mathematics,  beliefs  about  the  use  of  computers,  gaming  motivation,  mathematics  performance  on  an  algebra  immediate  posttest, 
and  mathematics  retention  measured  by  an  algebra  delayed  posttest.  Spatial  visualization  was  a covariate  for  beliefs  about 
computers.  Prior  beliefs  were  covariates  for  beliefs  about  mathematics  and  computers. 


I QuaHtiktlve  uuxi  Quaiitliatiye 


Procedures 

On  the  first  day  of  the  study,  the  participants  remained  in  their  regular  classroom  and  completed  an  algebra  pretest  and  the 
Visualization  in  Two-Dimensions  Test  (Flanagan,  et  al.,  1964),  The  next  day,  the  participants  ©mpleted  a demographic 
questionnaire,  the  Fennema-Sherman  Mathematics  Attitudes  Scales  (Fennema  & Sherman,  1976),  and  the  Computer  Attitude 
Scale  (Shashaani,  1993).  The  participants  were  randomly  assigned  to  work  in  pairs  with  a student  of  the  same  sex  in  the  computer 
laboratory.  Four  class  sessions  were  devoted  to  playing  the  game.  Near  the  end  of  the  class  session,  the  students  completed  a 
three-page  Daily  Journal  form  to  describe  their  experience.  The  students  described  the  strategies  they  employed,  » well  as 
elements  they  liked  or  disliked  about  the  game.  In  addition,  students  described  successful  and  unsuccessful  situations,  along  with 
their  attributions  for  success.  A sample  of  the  journal  questions  is  shown  in  Figure  2.  After  the  four  days,  participants  again 
completed  the  Fennema-Sherman  Mathematics  Attitudes  Scales  (Fennema  & Sherman,  1976),  the  Computer  Attitude  Scale 
(Shashaani,  1993),  and  the  Motivational  Gaming  Scale  (Dempsev.  et  al.,  1997).  The  next  day,  the  participants  completed  an 
immediate  posttest  on  the  algebra  topics.  Two  weeks  later,  the  participants  completed  a retention  test  on  the  algebra  topics. 

Three  days  after  the  immediate  posttest,  the  students  participated  in  focus  group 
interviews.  For  each  of  the  five  algebra  classes,  Iwo  separate  focus  group 
sessions  were  conducted.  One  focus  group  consisted  of  males  only.  The  other 
focus  group  consisted  of  females  only.  Each  focus  group  session  lasted 
approximately  20  minutes.  Upon  arrival  at  the  focus  group  session,  the 
participant  s submitted  a personal  letter  to  a friend  that  described  their  experience 
with  the  game  and  the  strategies  they  used. 

Data  Analysis 

The  statistical  software  SPSS  6.1  (1994)  served  as  the  primary  tool  for  the 
analysis  of  the  quantitative  data.  Chi-square  tests  were  used  to  analyze  gender 
differences  in  the  responses  to  the  Demographic  Survey  and  Game  Survey  Part  I. 
Game  Survey  Part  I was  developed  primarily  to  determine  if  students  had  played 
the  game  before  and  to  see  if  the  participants  felt  in  control  cf  the  technical 
aspects  of  the  game,  such  as  using  the  mouse  and  keyboard.  Descriptive  statistics 
were  used  with  the  Fennema-Sherman  Mathematics  Attitudes  Scales  (Fennema  & 
Sherman,  1976)  and  the  Computer  Attitude  Scale  (Shashaani.  1993).  Analysis  of 
covariance  (ANCOVA)  was  used  to  examine  differences  between  males  and 
females  in  their  beliefs  about  mathematics,  controlling  for  prior  beliefs. 
ANCOVA  was  also  used  to  examine  differences  between  males  and  females  in 
their  beliefs  about  the  use  of  computers,  controlling  for  prior  beliefs  and  spatial 
visualization.  ANCOVA  was  used  to  examine  the  algebra  posttests.  The  algebra 


Figure  2.  Sample  Questions 
from  the  Daily  Journal 

What  steps  did  you  take  to  play  the 
game  today?  In  other  words,  what 
was  your  Plan  of  Action? 

Describe  a situation  where  you  were 
successful  today.  Explain  why 
you  were  successful. 

What  do  you  remember  enjoying  most 
today? 

What  do  you  remember  enjoying  least 
today? 


o 
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pretest  was  the  covariate.  The  Motivational  Gaming  Scale  (labeled  as  Game  Survey  Part  II)  was  examined  with  factor  analysis. 
T tests  were  conducted  for  gender  differences  on  the  factor  scores.  Chi-square  tests  were  used  for  a follow-up  analysis  on 
individual  Motivational  Gaming  Scale  items. 

For  this  study,  some  of  the  statements  were  modified  to  be  more  specific  to  reviewing  algebra  with  a computer-based  game.  The 
estimated  reliability  of  the  scale  was  .89  (Cronbach’s  alpha).  In  an  earlier  study  (Dempsey  & Johnson,  1998),  the  estimated 
reliability  was  .64  (Cronbach’s  alpha). 

A qualitative  software  program,  QSR  NUD*IST  4(1997),  was  used  in  the  analysis  of  the  qualitative  data.  The  acronym 
NUD*IST  stands  for  Non-numerical  Unstructured  Data  * Indexing  Searching  and  Theorizing.  The  software  can  be  used  for 
coding  data  in  an  index  system.  When  all  the  qualitative  data  were  coded  for  this  study,  the  resulting  index  system  was  exported 
to  the  software  program.  Inspiration  4.0  (1992)  in  the  form  of  an  outline. 

The  coded  qualitative  data  were  submitted  to  a peer  reviewer.  As  a result  of  the  comments  and  suggestions  from  the  peer 
reviewer,  modiftcations  were  made  to  the  coding.  Peer  review  is  a strategy  used  to  promote  validity  in  qualitative  research 
(Johnson,  1997). 

Results  and  Discussion 

Strategies 

Evidence  emerged  that  showed  differences  in  males  and  females  and  their  reports  of  strategies.  Females  gave  more  evidence 
of  using  metacognitive,  cognitive  and  cooperative  strategies.  Males  gave  more  reports  that  they  did  not  have  a specified  strategy. 
Both  males  and  females  showed  evidence  that  their  strategy  was  to  challenge  themselves . 

Malone  (1981)  suggests  that  learners  can  be  challenged  by  uncertain  outcome.  Malone  recommends  four  strategies  for 
achieving  uncertain  outcome:  varying  the  level  of  difficulty,  designing  multiple-level  goals,  hiding  information,  and  introducing 
randomness.  The  students  discovered  ways  to  vary  the  level  of  difficulty  and  make  the  game  suitable  for  their  skill  level  by 
selecting  algebra  problems  accordingly.  Multiple-level  goals  were  reported  by  students  whose  strategies  included  working 
quickly,  beating  the  timer,  and  solving  the  problem  mentally  or  working  the  problem  in  their  head. 

The  students  demonstrated  a connection  between  motivation  and  the  use  of  strategies.  Malone  (1981)  recommends 
motivating  students  to  learn  by  presenting  just  enough  information  to  make  the  learner’s  existing  knowledge  seem  incomplete, 
inconsistent,  or  unparsimonious.  The  students  demonstrated  their  ability  to  recognize  when  their  existing  knowledge  was 
incomplete  by  reporting  the  use  of  metacognitive  strategies.  For  example,  one  male  wrote  in  his  Letter  to  a Friend,  “Every  day 
before  I played,  I learned  the  formulas  to  the  problems  I was  doing.  This  helped  me  understand  the  problem  better  when  I played 
the  game.”  A female  student  wrote  in  her  Daily  Journal,  “Review  the  material  since  we  have  not  looked  at  it  in  a while.” 

With  respect  to  cognitive  strategies,  females  (n  = 34)  gave  twice  as  many  reports  as  males 
(n  = 17)  for  solving  problems  on  paper.  As  a cooperative  strategy,  females  (n  = 16)  showed  a much  greater  use  of  comparing 
individual  answers  than  males  (n  = 4).  The  cognitive  and  cooperative  strategies  appear  to  have  a linkage.  The  linkage  appears  to 
be  where  females  compared  individual  answers  (a  cooperative  strategy)  that  they  solved  on  paper  (a  cognitive  strategy). 

Motivation 

In  the  analysis  of  motivation,  categories  emerged  that  fit  with  Keller’s  (1983,  1987a,  1987b)  ARCS  Model  of  Motivation  and 
Malone’s  (1981)  theory  of  intrinsically  motivating  instruction.  Females  showed  higher  motivation  with  respect  to  relevance  of 
the  game  as  a learning  tool  (e.g.,  finding  the  game  helpful).  In  addition,  females  indicated  higher  motivation  through  satisfaction 
(e.g.,  finding  benefit  of  examination  review).  Males  were  more  highly  motivated  by  challenge  in  terms  of  building  self-esteem 
(e.g.,  achieving  a high  score).  Both  males  and  females  showed  evidence  of  gaining  satisfaction  from  positive  consequences  (e.g., 
reporting  the  game  as  fun,  enjoyable,  good,  or  great).  Both  males  and  females  were  motivated  by  confidence  generated  by 
success  opportunities  (e.g.,  solving  problems,  learning  new  formulas). 

Keller  (1987a)  points  out  that  success  opportunities  promote  confidence.  Keller  emphasizes  the  importance  of  allowing 
learners  to  have  meaningful  success  experiences  as  soon  as  possible  to  stimulate  continued  motivation.  Numerous  citations  in  the 
Daily  Journals  illustrate  the  success  of  both  males  and  females.  The  following  quotes  were  written  by  female  students  in 
response  to  the  question,  “What  did  you  enjoy  most  today?”  “Figuring  them  out,  finally!”  “Finally  figuring  them  out.” 
“Figuring  out  slope  and  distance.”  “Knowing  how  to  solve  the  problem.”  “Finding  out  we  were  right.”  “Learning  how  to  do  the 
problems.”  “Remembering  how  to  do  the  problems.”  “Finally  understanding  slope-intercept.”  (The  student  drew  a smiley  face 
next  to  her  response.) 

Male  students  wrote  similar  responses  in  the  Daily  Journals  to  illustrate  their  success.  The  following  quotes  are  examples  of 
what  they  wrote  in  response  to  the  question,  “What  did  you  enjoy  most  today?”  “When  I got  this  long  problem.”  “Learning  and 
solving  problems.”  “Having  more  of  a challenge  by  working  all  of  the  problems.”  “Figuring  out  the  problems.”  “Answering 
problems  correctly.”  “Learning  a new  formula.”  “Learning  distance  formula.”  “Remembering.” 

Both  males  and  females  indicated  repetition  as  a problem  with  sustaining  attention  to  the  game.  Males  indicated  a desire  for 
instructional  games  to  include  elements  of  entertainment  games,  such  as  3-D  images  and  violent  fantasies,  to  attract  their 
attention.  Although  the  issue  of  violence  was  raised  only  in  the  focus  group  interviews,  ignoring  the  implications  is  difficult.  The 
issue  arose  primarily  in  discussions  of  why  more  males  play  computer  games  and  whether  the  game  used  in  this  study  was 
appropriate  for  both  males  and  females.  The  researcher  presented  no  violence-related  questions.  The  following  dialogue 
illustrates  the  students’  ideas  about  violent  themes  in  games. 

“Tell  me  what  you  think  about  the  Soak  Your  Sibling  game  for  males  and  females.  Is  it  more  interesting  to  males  or  females 
or  both?  ” A female  student  said,  “If  they  pulled  out  a Tommy  gun,  guys  would  like  it  more.” 


Similarly,  in  another  focus  group  interview,  one  male  said,  “It  needs  to  add  some  violence.”  The  researcher  asked,  “Violence? 
You  think  it  needs  more  violence?”  The  student  responded,  “Yeah.  Like  a guillotine  to  chop  off  your  head,”  Another  male 
offered  a less  violent  solution  by  saying,  “Males  would  like  it  more  if  it  had  sports.” 

Beliefs  About  Mathematics  and  the  Use  of  Computers 

Before  and  after  student  use  of  the  game,  t-test  results  of  the  attitude  scales  showed  that  males  were  more  likely  than  females 
to  stereotype  both  mathematics  and  computers  as  male  domains.  Although  the  qualitative  data  from  the  focus  group  interviews 
gave  some  indication  of  gender-based  stereotyping,  both  males  and  females  gave  evidence  during  the  interviews  that  they  did  not 
hold  stereotypical  views  about  mathematics  and  the  use  of  computers.  Females  also  recognized  that  stereotypical  views  stem 
from  societal  expectations  and  are  more  often  views  that  are  held  by  males. 

Mathematics  Performance 

The  results  did  not  show  a statistically  significant  difference  between  males  and  females  in  their  immediate  posttest  or 
delayed  posttest  scores.  The  results  did  not  support  the  research  hypotheses  that  males  will  score  higher  than  females  on  an 
immediate  or  delayed  posttest  following  the  use  of  a computer-based  mathematics  game.  The  results  on  the  delayed  posttest, 
however,  indicated  a tendency  toward  statistical  significance  (p.  = ,070)  with  females  performing  better  than  males. 

Implications 

Both  males  and  females  demonstrate  the  development  of  cooperative,  cognitive,  and  metacognitive  strategies  when  playing  a 
computer-based  mathematics  game.  The  use  of  games,  such  as  the  one  used  in  this  study,  may  be  disadvantageous  to  males, 
however.  If  males  have  more  experience  with  computer-based  games,  especially  violent  games,  they  may  be  less  likely  to  see  the 
benefits  of  games  to  their  learning.  Females,  on  the  other  hand,  may  benefit  from  the  use  of  this  type  of  computer-based  game 
because  they  are  more  likely  to  see  the  value  of  the  game  as  a learning  tool.  The  evident  gender  differences  indicate  the  need  to 
design  games  that  are  appealing  to  both  males  and  females.  The  development  of  various  game  fantasies  will  allow  students  to 
make  choices  that  are  more  relevant  to  the  student.  In  addition,  allowing  students  to  choose  the  level  of  difficulty  promotes 
challenge  and  success  opportunities. 

Conclusion 

Both  males  and  females  are  interested  in  the  use  of  a computer-based  mathematics  game,  although  they  take  a different 
approach  and  have  different  reasons  to  engage  in  the  game.  Designing  computer-based  mathematics  games  that  appeal  to  both 
males  and  females  supports  the  learners’  development  of  cooperative,  cognitive  and  metacognitive  strategies.  Games  that  are 
designed  with  varying  levels  of  difficulty  challenge  learners  to  succeed  without  damaging  their  self-esteem.  Designing  games 
that  challenge  learners  to  succeed  assists  the  learners  in  building  confidence  in  their  mathematics  ability. 
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Sister  Mary  Theresa  Brentano,  OSB’s  Innovative  Use 
of  Magnetic  Audio  Tapes: 

An  Overlooked  Story  in  the  History  of  Educational  Technology 

Linda  Herndon  O.S.B. 

''The  keenest  disappointment  of  our  trail  blazing  would  be 
that  we  could  not  fit  into  the  picture  of  new  developments.  " 
from  Brentano‘s  daily  journal  calendar,  January  5,  1959 


Introduction 

In  this  paper,  I tell  the  story  of  Sister  Mary  Theresa  Brentano,  O.S.B. ’s  (1902-1987)  innovative  use  of  magnetic  audio  tapes 
to  provide  instruction  for  students  in  grades  K-12.  From  1952  to  approximately  1968,  Brentano  implemented,  refined,  and  tested 
her  tape  teaching  methods  with  special  emphasis  on  individualizing  instruction  in  the  elementary  school.  Brentano ’s  innovative 
tape  teaching  ideas  are  not  mentioned  in  Saettler’s  The  Evolution  of  American  Educational 
Technology  (1990),  DeVaney’s  Voices  of  the  Founders:  Early  Discourses  in  Educational 
Technology  (1996),  or  Butler’s  Women  in  Audiovisual  Education.  1920-1957:  A Discourse 
Analysis  (1995).  In  this  paper  I provide  interpretive  biographical  look  at  Brentano’s  tape 
teaching  innovation.  I discuss  her  rationale  for  and  implementation  of  tape  teaching  and  share 
some  of  the  successes  and  struggles  of  tape  teaching.  I conclude  by  discussing  two  ways  my 
research  into  Brentano’s  individualized  tape  teaching  innovation  benefits  the  field  of 
educational  technology. 

Sister  Mary  Theresa  Brentano,  O.S.B.  was  a Benedictine  Sister  of  Mount  St.  Scholastica 
in  Atchison,  Kansas.’  In  June  1933,  Brentano,  a member  of  the  Phi  Beta  Kappa  honor 
society,  received  the  first  Ph.D.  in  English  awarded  by  the  University  of  Kansas  (“City 
News,”  1933).  After  receiving  her  Ph.D.,  Brentano  returned  to  the  college  run  by  her  religious 
community.  Mount  St.  Scholastica  College^  in  Atchison,  Kansas,  as  head  of  the  English 
Department. 

Sometime  during  the  1947-48  school  year.  Mother  Walburga  (Anna)  Franz,  Prioress  of 
St.  Scholastica  Priory,  Covington,  Louisiana,  came  to  Atchison,  Kansas,  to  confer  with 
Mother  Lucy  Dooley,  Prioress  of  Mount  St.  Scholastica,  about  the  possibility  of  sending  a few  Sisters  to  Covington.  Franz 
wanted  some  sisters  from  Atchison  who  had  their  Ph.D.  to  go  to  Covington  to  help  the  sisters  maintain  state  certification  of  their 
school,  St.  Scholastica  Academy,  by  teaching  some  classes  to  their  sisters.  Brentano  was  one  of  two  sisters  from  Atchison  who 
volunteered  to  go  to  Louisiana  in  the  fall  of  1 948. 

The  Inspiration  for  Tape  Teaching 

Sometime  in  1952  while  she  was  principal  of  St.  Scholastica  Academy  in  Covington,  Brentano  and  Franz  went  on  a trip  to 
New  Orleans.  While  riding  in  a taxicab  there,  Brentano  noticed  that  the  cab  driver  could  call  his  headquarters  to  get  his  orders  to 
find  out  who  to  wait  for  and  where.  Brentano  got  the  idea  that  if  the  taxi  driver  could  talk  over  the  intercom  and  communicate 
with  headquarters,  then  why  couldn't  this  same  technology  be  used  in  a single  classroom  with  the  different  students  hearing  the 
teacher  whenever  she  talked  to  them.  (M.  P.  Ege,  OSB,  personal  communication,  November  6,  1 999) 

With  the  full  support  of  Franz,  Brentano  set  about  to  make  her  germ  of  an  idea  for  individualized  tape  teaching  a reality. 
Having  previously  observed  the  use  of  tape  recorders  and  earphones  to  teach  graduate  students  foreign  languages  (“Classroom 
electron,”  unknown;  Stoma  1 957),  she  decided  to  use  this  technology  to  provide  the  individualized  instruction  that  parents  were 
requesting  for  their  daughters  and  sons.  Although  we  now  think  of  individualized  instruction  as  meaning  each  child  is  taught 
according  to  his  or  her  needs,  for  Brentano  individualized  instruction  meant  that  a child's  individual  needs  were  important,  but  for 
instructional  purposes,  the  child  is  grouped  with  other  children  who  have  similar  reeds.  She  referred  this  method  as 
“individualization  within  the  group”  (1959).  Brentano  described  individualized  tape  teaching  as  “probably  the  greatest  aid  now 
available  for  taking  care  of  individual  differences  in  the  classroom”  (“Sisters  Prepare  Tape  Recorded  Lessons,”  1954?).  The 
huge  increase  in  the  student  population  as  the  Baby  Boomers  started  school  created  a teacher  shortage  and  individualized  tape 
teaching  was  touted  as  a possible  way  to  help  to  alleviate  this  problem. 

Tape  Teaching— What  It  Is  and  How  It  Developed 

A class  period  taught  using  Brentano's  individualized  tape  teaching  methods  was  divided  into  three  sections  of  about  20 
minutes  each  (Keating,  1961).  During  the  first  section  of  the  class  period,  the  teacher  introduced  the  class's  topic.  The  teacher 
then  divided  the  students  into  four  ability  groups.  While  three  groups  used  earphones  to  listen  to  tapes  prepared  by  their  teacher 
on  the  topic,  the  teacher  worked  with  the  fourth  group  that  needed  extra  help.  The  brightest  students  heard  on  tape  how  to  apply 
what  the  teacher  had  presented.  The  average  students  got  some  clues  or  reminders  about  how  to  apply  what  was  presented  while 
the  slow  learners  had  the  entire  lesson  repeated,  possibly  more  than  once  (Stoma,  1 957).  The  last  section  of  the  class  period  was  a 
total-class  discussion  where  students  had  a chance  to  share  what  they  learned  from  listening  to  the  tapes.  A former  tape  teacher 
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shared  with  me  that  Brentano  encouraged  her  to  make  sure  that  the  low  ability  students  always  had  some  information  on  their 
tape  that  the  other  groups  did  not  have  (T,  Balot,  personal  communication,  March  24,  2001).  In  this  way,  the  lower  ability 
students  could  always  participate  and  contribute  something  special  to  the  class  discussion. 

Ideally,  only  the  teacher  knew  to  which  ability  group  she  or  he  assigned  a given  student.  The  reason  for  this  was  to  remove 
the  stigmas  that  can  go  with  the  grouping  of  students  (Stengren,  1958).  These  ability  groups  did  not  need  to  be  the  same  for 
every  subject  (Dickerson,  1958).  For  instance,  a student  could  be  in  one  group  for  arithmetic  and  a different  one  for  English.  The 
teacher  could  change  the  student  from  one  group  to  another  during  the  school  year  and  the  student  could  also  request  to  change 
groups  (S.  Ross  & Kiester,  1958,  and  Brentano  notes).  From  interviews  with  tape  teachers,  all  shared  with  me  that  in  practice 
students  did  know  to  which  of  the  four  groups  they  were  assigned. 

In  the  fall  semester  of  1952,  a class  of  primary  student  first 
used  earphones  and  tapes  in  a room  with  30  listening  booths 
and  six  channels  of  tape  (S.  M.  T.  Brentano,  O.S.B.,  1962,  p. 
368)  (See  Figure  2).  Brentano  believed  that  providing  primary 
grade  children  with  the  best  education  possible  was  important 
since  that  was  where  their  foundation  in  education  began  (M. 
K.  Taylor,  OSB,  personal  communication,  January  12,  2000) 
and  she  believed  that  “small  pupils  were  being  slighted  in  their 
American  technological  heritage”  (Laube,  1958).  Figure  2,  the 
oldest  dated  photograph  of  tape  teaching,  requires  some 
explanation.  This  photograph  shows  a six-tape  console  being 
used  by  a teacher.  When  Brentano  originally  started  tape 
teaching,  she  used  six  different  tapes.  It  seems  that  experience 
proved  that  three  tapes  were  as  effective  as  six  and  required 
much  less  preparation  for  the  teacher.  The  photograph  shows 
high  school  girls,  not  elementary  school  children,  in  the  carrel- 
style  listening  booths.  Several  newspaper  articles  in  the  fall  of 
1954  (“Electronic  Classroom,”  1954;  “Sisters  Prepare  Tape 
Recorded  Lessons,”  1954?;  “Tape-recorders,  books  team  up  in 
ademy.  1952.  first  U.S.  ‘Electronic  Classroom’,”  1954)  have  photographs  that  show  this 

classroom  being  used  to  teach  primary  grade  girls  and  boys.  Brentano  coined  the 
term  Electronic  Classroom  to  refer  to  this  room  since  she  said  it  did  not  seem  appropriate  to  call  it  a laboratory  with  first  and 
second  graders  using  it  (date  unknown). 

Brentano’s  individualized  tape  teaching  innovation  evolved  with  experience.  One  of  the  first  additions  to  tape  teaching  was 
the  use  of  a worksheet  to  accompany  the  tape.  As  Brentano  explained  (1962), 

On  tape  the  child  never  merely  listens.  He  will  listen,  learn,  and  on  his  worksheet  record  the  fruits  of  his 
mental  activities  both  for  his  own  help  in  review  and  his  teacher's  evaluation  of  this  achievement.. ..On  his 
worksheet  he  will  likely  be  asked  to  write  answers  that  relate  the  lesson  to  his  past  experience,  to  give  the 
result  of  his  observations,  and  to  carry  his  observations  forward  until  he  can  infer  from  facts  some 
generalization  or  principle  . . . and  finally  correct  his  paper  by  noting  the  answers  which  the  tape  will  give 
him.  (p.  369) 

Since  the  six-track  individualized  tape  instruction  electronic  classroom  was  popular  with  both  teachers  and  parents,  there  was 
a need  to  create  another  tape  teaching  classroom.  The  second  iteration  of  the  tape  classroom  was  built  in  1953  and  it  provided  a 
less  expensive  alternative  to  the  original  electronic  classroom  (See  Figure  3).  In  this  classroom,  conduits  that  held  the  lines  from 
the  tape  recorders  extended  between  the  rows  of  desks.  While  seated  at  their  desks,  the  children  plugged  their  earphones  into  a 
jack  box  in  the  conduit  and  could  listen  to  the  tape  without  having  to  be  in  special  listening  booths  (date  unknown). 
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Figure  3.  Second  iteration  of  tape  teaching  classroom. 
Sister  Maris  Stella  Pravata  (Maris  Leitz)  is  the  teacher. 
Date  unnknown. 


Figure  4.  The  third  iteration  of  the  tape  teaching  classroom. 
On  Pravata 's  desk  is  a microphone  and  a switch  box.  Photo 
by  Petit ’s  Studio.  Date  unknown. 


The  third  iteration  of  the  tape  classroom  (See  Figure  4)  developed  later  involved  the  addition  of  an  intercom  system  by  which  the 
teacher  could  contact  an  individual  student  without  disturbing  the  rest  of  the  class.  Brentano  described  needing  an  intercom  as  a 
way  to  get  the  attention  of  an  inattentive  student  as  well  as  to  determine  if  a student  could  properly  hear  the  tape.  The  first 
intercom  system  allowed  only  one-way  communication-from  teacher  to  a student  (Laube  & Brentano,  1957).  Figure  4 shows  a 
tape  teaching  classroom  complete  with  the  microphone  by  which  the  teacher  could  contact  individual  students.  To  contact  an 
individual  student,  the  teacher  had  a set  of  switches  that  allowed  her  tap  into  the  same  line  as  an  individual  student. 

Our  Lady  of  Wisdom  Hall-The  First  Electronic  School 

The  success  of  tape  teaching  resulted  in  an  increased  enrollment  at  St.  Scholastica  Academy.  In  1957  to  provide  more 
classroom  space  for  the  larger  enrollment  and  to  meet  the  needs  of  Brentano's  developing  tape  teaching  innovation,  the 
Benedictine  Sisters  built  Our  Lady  of  Wisdom  Hall.  Newsweek  (“Electronic  classrooms,”  1957,  p.  99)  referred  to  Our  Lady  of 
Wisdom  Hall  as  “the  first  structure  in  the  history  of  American  education  to  be  specially  designed  and  pre-wired  for  electronic 
teaching.”  Our  Lady  of  Wisdom  Hall,  a one-story,  air-conditioned,  three-classroom  building,  cost  $75,000  of  which  $15,000  was 
for  the  electrical  features  (“Classroom  electron,”  unknown).  Two  classrooms,  that  Brentano  called  Classroom  Electron  I,  were 
somewhat  traditional  looking  with  individual  desks  and  the  other  classroom.  Classroom  Electron  II,  had  32  individual  recitation 
booths  as  well  as  desks.  There  was  a fourth  and  much  smaller  room  called  the  individual  measurement  room. 

Classroom  Electron  I 

At  first  glance,  each  of  the  two  identical  Classroom  Electron  Ts  looked  like  traditional  classrooms  (see  Figure  5).  One 
noticeable  difference  is  that  on  the  floor  beside  each  desk  is  a recessed  receptacle  into  which  a student  plugged  in  a set  of 
earphones.  This  innovation,  designed  by  Brentano,  eliminated  the  raised  conduits  with  wires  in  the  aisles  between  the  students* 
desks.  The  second  special  feature  was  that  the  teacher's  desk  was  replaced  by  a special  six-foot  long  console  that  includes  four 
heavy-duty  Viking-75  tape  recorders  (“Electronic  age  school,”  1959),  a set  of  control  switches  for  each  student's  desk,  and  a 
headset  with  a microphone  for  the  teacher  (see  Figure  6).  The  four  reel-to-reel  audio  tape  players  allowed  the  teacher  to  present 
up  to  four  different  taped  lessons  at  any  given  time.  Besides  the  three  tapes  used  for  instructing  three  different  ability  groups,  the 
fourth  tape  recorder  was  available  for  a variety  of  purposes:  a special  lesson  for  highly  accelerated  students,  a remedial  lesson,  a 
make-up  lesson  replaying  a tape  missed  a previous  day,  or  as  a replacement  tape  recorder  if  one  of  the  other  three  recorders  was 
not  working  correctly.  The  toggle  switches  shown  in  Figure  6 allowed  the  teacher  to  select  which  of  the  four  tapes  each 
individual  student  heard. 
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Figure  5.  Classroom  Electron  I.  While  Pravata  instructs 
some  students  others  are  using  earphones  for  taped 
instruction.  Photo  by  Petit's  Studio.  1957. 


Figure  6.  Classroom  Electron  I teacher's  console.  Sister 
Anthony  Balot  (Toni)  adjusts  the  switches  to  set  which  tape 
a student  hears.  Photo  by  Petit's  Studio.  1957. 


Clasroomsroom  Electron  II  (or  the  Booth  Room) 

The  single  Classroom  Electron  II  (or  Booth  Room)  in  Our  Lady  of  Wisdom  Hall  was  equipped  with  desks  in  the  middle  of 
the  room  and  a total  of  32  soundproof  individual  recitation  booths  along  twosides  of  the  room  (see  Figure  1).  The  desks  permit 
the  teacher  to  work  with  a small  group  of  students  while  the  other  students  are  working  in  their  own  recitation  booths.  Each  booth 
had  a writing  shelf,  one  or  more  chairs,  a set  of  lightweight,  removable  earphones,  a two-way  intercom  speaker,  and  a light.  In 
each  booth  there  are  four  possible  openings  into  which  to  plug  the  earphones.  This  allowed  the  teacher  to  select  audio  from  one 
of  the  three  tapes,  a record  on  the  phonograph,  or  a radio  program  depending  on  what  she  or  he  wanted  the  student  to  hear,  A 
switch  in  each  booth  allowed  the  student  to  contact  the  teacher  by  turning  on  a light  on  the  annunciator  panel  on  the  teacher's 
console  (see  Figure  8).  The  teacher,  through  the  use  of  her  headset  at  the  console,  could  answer  the  student's  question  privately 
without  disturbing  anyone  else.  The  use  of  the  headset  also  allowed  the  teacher  to  listen  in  on  what  was  going  on  in  any  of  the 
booths  without  disturbing  the  student  (“$40,000  Ford  grant  enables  college  to  test  new  electronic  teaching  system,”  1958; 
Sherman,  1958), 


Figure  7.  Classroom  electron  II.  Photo  by  Petit 's 
Studio.  1957 


Figure  8.  Classroom  Electron  II  teacher 's  console  complete 
with  3 tape  recorders,  record  player,  annunciator  lights 
Switches  for  each  of  the  32  booths  and  teacher’s  headset. 
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Stoma  (1957)  reports  that  there  was  an  individual  measurement  room  in  Our  Lady  of  Wisdom  Hall  where  the  teachers  posted 
records  of  each  student’s  progress  and  of  each  class's  achievements.  The  original  plan  was  that  this  small  room  would  be  used  as 
a research  laboratory  in  which  the  teachers  could  study  how  a child  actually  learns.  From  what  I could  ascertain  from 
conversations  with  former  tape  teachers  and  from  Brentano's  notes,  the  individual  measurement  room  was  never  used  for 
anything  other  than  a place  to  post  test  results.  It  is  likely  that  the  primary  reason  for  this  is  lack  of  development  of  the  individual 
measurement  room  is  that  Brentano  returned  to  Atchison,  Kansas,  at  then  end  of  August  1957,  only  months  after  Our  Lady  of 
Wisdom  Hall  opened. 

1957  Summer  Tape  Workshop 

Late  in  May  of  1957,  the  Ford  Foundation  Fund  for  the  Advancement  of  Education  awarded  Brentano  $15,000  to  conduct  a 
six-week  Tape  Workshop  in  Our  Lady  of  Wisdom  Hall  during  the  summer  of  1957  (Fine,  1957).  At  the  workshop,  Brentano 
taught  teachers  how  to  implement  her  tape  teaching  methods  in  their  own  classrooms  including  how  to  write  scripts,  record  tapes, 
and  make  worksheets.  A total  of  36  teachers,  all  members  of  Catholic  religious  communities,  attended  the  workshop  and  made 
tapes  in  geography. 

The  Tape  Institute  at  Mount  St.  Scholastica 

In  August  1957,  Brentano  returned  to  her  home  religious  community  of  Mount  St.  Scholastica  in  Atchison,  Kansas,  to  set  up 
and  run  the  Tape  Institute.  With  more  than  $100,000  in  grants  from  the  Fund  for  the  Advancement  of  Education  over  a three-year 
period,  $23,000  from  the  Raskob  Foundation  for  Catholic  Activities,  Inc.,  and  with  space  and  financial  assistance  from  Mount  St. 
Scholastica  College,  the  Tape  Institute  was  the  center  of  Brentano's  tape  teaching  innovation  until  the  mid  1960s.  The  Tape 
Institute  was  responsible  for  creating  more  than  twelve  thousand  tapes,  scripts,  and  worksheets  in  all  different  subjects  for  all 
grades  and  distributing  them  to  schools  using  Brentano's  methods  around  the  country.  Most  of  these  tapes  were  created  by  the 
teachers  who  attended  the  four  summer  tape  workshops  from  1958  to  1961.  The  Tape  Institute  was  also  responsible  for 
coordinating  the  standardized  testing  done  in  tape  schools  and  their  counterpart  non-tape  schools  to  provide  for  statistical 
measures  of  the  success  of  tape  teaching. 

By  1957  the  following  schools  had  adopted  Brentano's  individualized  tape  teaching  methods: 

• St.  Scholastica  Academy,  Covington,  LA:  5^  through  8^**  grades 

• Mater  Dolorosa  School,  New  Orleans,  LA:  2nd  grade,  remedial  reading 

• Cathedral  School,  Lafayette,  LA:  8^  grade 

• De  La  Salle  Normal  School  and  Junior  Novitiate  of  Brothers  of  the  Christian  Schools,  Lafayette,  LA:  social  studies  and 
chemistry 

• Our  Lady  of  Fatima  School,  Lafayette,  LA:  6th  grade 

• Immaculate  Conception  School,  Grand  Prairie,  TX:  4th  grade,  reading  clinic 

• Incarnate  Word  School,  San  Antonio,  TX:  I st  grade 

• St.  Joseph  Cathedral  School,  St.  Joseph,  MO:  3rd  and  6**^  grades 

In  1958  there  were  45  rooms  in  16  schools  using  Brentano's  tape  teaching  methods  and  by  1960  there  were  electronic  classrooms 
in  80  schools  (Keating,  1961 , p.  20). 


The  Benefits  of  Tape  Teaching 

According  to  Brentano  (1962,  p.  369),  there  were  four  main  benefits  for  students  taught  using  individualized  tape  teaching. 
The  first  benefit  was  that  tape  teaching  fits  student's  needs  because  the  students  listened  to  tapes  especially  made  for  their  ability 
level.  If  a student  needed  special  attention,  then  the  teacher  provided  that  while  the  rest  of  the  class  listened  to  tapes. 

A second  benefit  was  that  tape  teaching  presented  superior  instruction  since  it  was  the  goal  of  the  teacher  making  the  tape  “to 
present  the  best  material  in  the  best  way,  so  that  the  pupils  who  learn  it  will  be  equipped  to  live  successfully  in  their  world” 
(Knoedel,  1958).  Tapes  were  not  merely  to  repeat  a textbook,  but  were  to  be  creative,  interesting,  up-to-date  works  of  art  worthy 
of  keeping  and  using  again  and  for  sharing  with  other  teachers. 
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The  third  benefit  of  tape  teaching  was  that  students  learned  to  concentrate  better.  According  to  Brentano,  one  reason  for 
improved  concentration  is  that  the  use  of  earphones  created  a feeling  of  a more  personal  contract  with  the  teacher.  ‘The  hearing 
of  a lesson  on  earphones  is  a much  more  intimate  experience  than  hearing  a tape  played  aloud.  With  this  information  going  into 
the  student’s  ears,  and  his  eyes  occupied  with  the  worksheet,  the  attention  is  truly  phenomenal”  (personal  notes  of  Brentano). 
Directions  were  given  at  the  beginning  of  the  tape.  Since  the  tape  did  not  repeat  directions,  students  soon  learned  that  if  they 
missed  the  directions,  they  were  out  of  luck  since  the  teacher  would  be  busy  instructing  another  group  and  could  not  be  ■ 
interrupted.  Throughout  the  tape,  the  student  had  to  listen  carefully  in  order  to  answer  questions  on  the  worksheet.  At  the  end  of 
the  tape,  answers  to  the  worksheet  were  given  and  the  student  had  to  continue  to  pay  careful  attention  to  make  sure  s/he  graded 
the  worksheet  properly. 

The  fourth  benefit  Brentano  (1962,  p.  369)  listed  for  tape  teaching  is  that  it  allowed  the  student  to  have  more  personal  contact 
with  the  teacher.  A former  tape  student  told  me  that  when  she  listened  to  a tape,  she  felt  like  the  teacher  was  talking  directly  to 
her.  Brentano  claimed  that  tape  teaching  multiplied  a teacher's  voice  and  personality  since  she  or  he  was  really  teaching  four 
classes  at  a time-not  just  one.  This  allowed  teachers  “wore  time  for  discussion,  more  time  for  questions,  more  time  for  individual 
help”  (emphasis  original)  (S.  Ross  & Kiester,  1958).  Brentano  strongly  believed  that  tapes  were  not  meant  to  replace  a teacher, 
but  were  meant  to  supplement  her  by  multiplying  her  presence  in  the  classroom  (M.  T.  Brentano,  O.S.B.,  1959;  Keating,  1961; 
“Tape-recorders,  books  team  up  in  first  U.  S.  ‘Electronic  Classroom’,”  1954;  “What  is  a tape-teaching  workshop?,”  unknown). 

Tape  teaching  often  produced  impressive  test  results.  For  instance,  of  the  100  elementary  students  at  St.  Scholastica  Academy 
who  were  taught  using  Brentano's  tape  teaching  methods  during  the  1956-57  school  year,  only  one  student  failed  to  average  the 
ten-month  expected  gain  on  the  Metropolitan  Test.  “Twenty -five  averaged  a gain  between  one  and  two  years;  fifty -five  averaged 
a gain  between  two  and  three  years;  eighteen  a gain  between  three  and  four  years,  while  one  surpassed  the  equivalent  of  four 
years’  gain  as  measured  by  standard  norms”  (Laube,  1958,  p.  13). 

Some  Problems  and  Challenges  of  Tape  Teaching 

As  with  all  innovations  in  educational  technology,  tape  teaching  had  its  share  of  challenges  and  problems.  In  this  section  I 
briefly  describe  many  of  these  challenges.  One  problem  with  implementing  tape  teaching  was  the  cost  of  setting  up  an  electronic 
classroom,  the  ongoing  cost  of  maintaining  it,  and  the  cost  of  paper  for  worksheets.  In  1958,  Brentano  estimated  that  a school 
could  equip  a classroom  for  tape  teaching  for  $3000  (B.  Ross  & Ross,  1959).  If  the  funds  were  available,  a major  problem  was 
finding  an  electrician  who  could  build  and  wire  the  specialized  console  since  these  were  not  available  commercially.  Tape 
teachers  often  shared  with  me  that  equipment  did  not  work  properly,  i.e.,  tape  recorders  broke  frequently,  students  couldn’t  hear 
with  their  earphones,  headsets  picked  up  radio  programs  rather  than  the  tapes,  etc.  The  quality  of  the  recordings  often  made  it 
hard  to  hear  and  understand  a lesson. 

Although  tape  teaching  was  supposed  to  give  the  teacher  more  time,  tape  teachers  in  the  first  years  of  the  innovation  found 
this  to  be  anything  but  true.  Since  there  were  no  scripts,  tapes,  or  worksheets,  the  classroom  teacher  had  to  do  all  this 
herself/himself  besides  the  usual  classroom  responsibilities.  To  research  and  write  the  kind  of  creative  script  that  Brentano 
expected  could  take  20  or  more  hours.  Another  problem  was  that  when  a trained  tape  teacher  left  a school  there  was  often  no  one 
trained  to  use  tapes  to  replace  her/him.  The  loss  of  the  trained  tape  teacher  often  meant  that  tapes  were  no  longer  used  or  were 
not  used  as  Brentano  intended. 

Brentano’s  intent  was  to  have  the  Tape  Institute  create  an  entire  tape  curriculum  for  all  elementary  subjects,  but  this  never 
came  to  pass.  From  what  I can  tell  from  my  research,  it  seems  that  the  Tape  Institute  never  made  an  entire  year’s  set  of  tapes  for 
any  subject  for  any  grade.  Most  of  the  creative,  artistic  tapes  produced  by  teachers  for  the  Tape  Institute  were  not  correlated  to 
specific  textbooks.  Teachers  using  tapes  obtained  from  the  Tape  Institute  had  to  figure  out  where  in  their  own  curriculum  to  use 
a tape  or  set  of  tapes.  For  instance,  tapes  that  a teacher  created  for  her/his  fifth  grade  social  studies  class  in  Kansas  may  not  fit 
with  a similar  class  in  Kentucky.  Tapes  also  became  outdated  rapidly  and  required  a teacher  to  update  and  re-record  a new  tape. 

Since  tape  teaching  was  first  implemented  in  Catholic  schools,  the  press  and  others  raised  questions  as  to  whether  tape 
teaching  could  be  as  effective  in  public  schools  as  it  was  in  Catholic  schools.  An  article  in  The  New  York  Times  (Fine,  1957,  p. 
47)  asked,  “Can  this  program  operate  as  effectively  in  a typical  public  school,  with  large  classes  and  overworked  teachers?” 
Another  problem  that  kept  tape  teaching  from  becoming  more  widespread  in  public  schools  was  the  Catholic  content  of  many 
tapes.  Most  of  the  participants  at  the  summer  tape  workshops  were  members  of  Catholic  religious  communities  who  taught  in 
Catholic  schools.  As  they  would  when  teaching  in  their  own  classroom,  the  scripts  they  wrote  contained  numerous  references  to 
things  Catholic  although  these  were  not  scripts  for  religion  class  tapes.  For  instance,  a script  for  a tape  for  first  grade  science 
began  with,  “Today,  boys  and  girls,  we  are  going  to  talk  about  this  big,  wide,  wonderful  world  that  God  made  for  us.”  The 
Catholic  content  of  tapes  was  problematic  when  Brentano  sold  the  tapes. 

Brentano  had  little  assistance  in  running  the  Tape  Institute.  It  seems  that  she  had  someone  available  to  multilith  the 
worksheets  and  at  times  had  part  time  secretarial  help.  From  several  letters  written  to  Brentano  from  tape  teachers,  it  seems  that 
she  sometimes  had  a hard  time  getting  the  correct  tapes,  scripts,  and  worksheets  mailed  to  teachers  in  a timely  manner. 

In  some  ways  Brentano  may  have  been  counter-productive  in  carrying  on  her  own  tape  teaching  innovation.  For  instance,  her 
attitude  was  that  all  tapes  were  always  unfinished.  This  attitude  meant  that  Brentano  was  always  in  the  process  of  revising  and 
updating  tapes.  Brentano  was  not  a practical,  detail  person,  but  a dreamer  and  a visionary.  Once  she  had  experimented  with  tape 
teaching  and  saw  that  it  could  be  successful,  Brentano  moved  on  to  explore  her  other  creative  ideas.  As  early  as  1958,  she  started 
working  on  a Thinking  Curriculum  that  used  audio  tapes  and  worksheets  to  teach  thinking  skills  to  children  in  elementary  school. 
The  Thinking  Curriculum  grew  to  involve  the  study  of  the  importance  of  color  in  teaching  and  how  to  teach  logic  to  elementary 
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school  children. 


The  End  of  Tape  Teaching 

With  the  increasing  availability  and  popularity  of  video  technologies  in  education  and  computer-aided  instruction  plus 
Brentano’s  shift  to  other  interests,  tape  teaching  soon  became  history.  By  1962,  there  were  no  longer  articles  in  newspapers  or 
magazines  about  her  tape  teaching  methods  and  by  the  middle  1960s,  there  was  very  little  correspondence  to  Brentano  regarding 
tape  teaching.  In  1970  Brentano  entered  into  a contract  with  Scott  Scientific,  Inc.  of  Fort  Collins,  Colorado,  to  sell  them  the  tapes, 
scripts,  worksheets,  reel-to-reel  tape  recorders,  and  other  equipment  with  the  hope  that  they  could  successfully  market  them. 
Despite  the  best  efforts  of  Scott  Scientific,  including  removing  the  references  to  God  and  things  Catholic,  re-voicing  the  tapes, 
copying  them  onto  cassettes,  and  creating  ditto  masters  of  the  worksheets,  by  1971  Scott  was  no  longer  able  to  sell  any  of  her 
tapes  and  Brentano ’s  tapes  were  eventually  returned  to  her  (A.  Spring,  personal  communication,  April  29,  2000). 

Importance  of  This  Research  for  the  Field  of  Educational  Technology 

My  research  provides  several  interesting  results  for  the  field  of  educational  technology.  Due  to  space  considerations,  I discuss 
two  of  these  in  this  paper.  Besides  making  known  an  overlooked  innovation  in  the  field,  my  research  provides  a comprehensive 
look  at  an  innovation  in  educational  technology  from  its  initial  idea  to  its  conclusion.  I have  extensive  documents  from  Brentano 
including  personal  correspondence  and  even  some  reel-to-reel  tapes  that  she  made  describing  tape  teaching  and  its  history. 
Interviews  with  the  first  two  tape  teachers  from  St.  Scholastica  Academy,  with  several  tape  teachers  involved  during  the  height  of 
tape  teaching,  and  with  some  who  used  tapes  during  its  waning  years  of  use  provide  a fascinating  look  at  the  life  cycle  of  an 
innovation.  Personal  communication  with  friends  and  associates  of  Brentano  during  the  tape  teaching  years  supply  a wealth  of 
knowledge  and  insights  about  what  it  was  like  for  Brentano  as  an  innovator  in  educational  technology  during  the  1950s  and 
1960s.  My  dissertation.  Sister  Mary  Theresa  Brentano,  Innovator  in  the  Use  of  Magnetic  Audio  Tapes— An  Overlooked 

Story  in  the  History  of  Educational  Technology  (Herndon,  O.S.B.,  2002  (anticipated)),  provides  a more  thorough  documentation 
of  Brentano’s  innovation  than  is  possible  here. 

The  second  implication  of  my  research  concerns  why  official  histories  of  educational  technology  have  overlooked  Brentano's 
individualized  tape  teaching  innovation.  I believe  there  are  several  possible  reasons  for  this  oversight.  Brentano's  work  was 
featured  in  more  than  thirty  newspaper  and  magazine  articles  including  The  New  York  Times  (Fine,  1957)  and  Newsweek 
(“Electronic  classrooms,”  1957).  The  United  States  Information  Agency  (“USIA  television  service  films  electronic  classroom,” 
1959)  produced  a five-minute  movie  clip  illustrating  Brentano's  tape  teaching  innovation.  Upon  her  death.  The  New  York  Times 
(“Nun  Who  Pioneered  the  Use  of  Electronic  Teaching  Dies,”  1987)  printed  her  obituary.  One  might  guess  that  her  place  in  the 
history  of  educational  technology  was  assured.  However,  the  popular  press  is  not  usually  a place  where  academics  look  when 
writing  the  history  of  the  field.  For  instance,  De  Vaney  and  Butler  in  “Voices  of  the  Founders:  Early  Discourses  in  Educational 
Technology”  (1996)  used  information  from  audiovisual  textbooks  and  oral  history  audiotapes  from  the  Archives  of  the 
Department  of  Audio-Visual  Instruction.  Brentano  did  not  write  a textbook  about  individualized  tape  teaching  and  the  reel-to- 
reel  oral  tapes  that  Brentano  herself  made  have  been  stored  in  the  Mount  St.  Scholastica  Archives  unknown  to  anyone  at  all  until 
this  research. 

The  data  that  I have  does  not  suggest  that  Brentano  belonged  to  any  professional  organizations.  One  can  only  guess  that 
Brentano  may  have  believed  that  she  did  not  have  time  or  that  she  had  no  need  for  any  type  of  professional  affiliation.  She  did 
speak  at  many  regional  professional  meetings  and  also  two  national  conferences:  the  Catholic  Audio-Visual  Association 
Conference  in  1954  and  the  Department  of  Audiovisual  Instruction  Convention  in  1962.  The  only  professional  publication  of  her 
work  is  the  text  of  her  presentation  from  the  latter  convention  published  in  Audiovisual  Instruction  (S.  M.  T.  Brentano,  O.S.B., 
1962).  One  might  guess  that  women  religious  were  not  permitted  or  encouraged  to  hold  membership  in  professional 
organizations  or  to  publish  scholarly  papers  during  the  1950s  and  early  1960s,  but  that  was  not  so  at  Mount  St.  Scholastica. 
During  this  time  period,  several  sisters  were  actively  involved  in  professional  organizations  and  in  publishing  scholarly  research.^ 

Other  reasons  could  also  have  contributed  to  Brentano's  work  being  overlooked  in  official  histories  of  the  field  of  educational 
technology.  Very  little  is  written  in  any  book  regarding  the  use  of  audio  tapes  in  the  classroom,  including  Saettler's  (1990) 
definitive  history  of  the  field.  The  Evolution  of  American  Educational  Technology.  Brentano's  original  work  with  audio  tape 
teaching  was  not  based  at  a major  research  university,  but  at  a small  Catholic  K-12  school.  When  she  did  move  her  base  of 
operation  to  be  associated  with  a college,  it  was  to  Mount  St.  Scholastica  College,  a very  small  Catholic  women’s  college  in 
Atchison,  Kansas.  Although  this  does  not  outwardly  appear  to  be  a concern  to  Brentano,  this  move  was  not  helpful  in  making  her 
work  known  in  academic  circles.  De  Vaney  and  Butler  (1996)  state  that,  “Historically,  although  women  in  audiovisual  education 
played  major  roles,  they  were  often  overlooked  and/or  assigned  minor  ones”  (p.  39).  One  can  speculate  that  the  fact  that 
Brentano  was  a woman  and  was  a member  of  a Catholic  monastery  were  also  reasons  in  their  own  way  for  her  being  overlooked 
in  official  histories  of  the  field. 
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Conclusion 

This  paper  presented  an  overview  of  Sister  Mary  Theresa  Brentano's  innovation  in  educational  technology -the  use  of 
magnetic  audio  tapes  to  provide  individualized  instruction  to  children  in  K-12  schools.  I described  Our  Lady  of  Wisdom  Hall  at 
St.  Scholastica  Academy,  Covington,  Louisiana,  the  first  school  designed  and  built  for  electronic  teaching  with  its  two  different 
types  of  Classroom  Electron.  Benefits  of  tape  teaching  were  recounted  along  with  some  challenges  and  problems  involved  with 
the  innovation.  Lastly,  I presented  two  contributions  this  research  contributes  to  the  field  of  educational  technology:  a complete 
history  of  an  innovation  in  educational  technology  from  its  beginning  to  its  end  and  some  reasons  how  an  innovation  such  as  this 
can  be  overlooked  in  the  official  histories  of  the  field. 
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Abstract 

The  purpose  of  this  study  was  to  explore  best  practices  for  community  building  in  Web-based  learning  environments.  The 
study  took  place  in  two  Web-based  courses  at  two  universities.  An  embedded  case  study  design  was  used,  and  multiple  sources  of 
evidence  (e.g.,  chat  and  bulletin  board  transcripts,  interviews,  and  surveys)  were  gathered  to  inform  the  results.  Overall, 
participants  in  both  courses  indicated  some  sense  of  community,  albeit  limited  in  scope  (i.e.,  the  learners  indicated  a stronger 
connection  with  their  team  members  than  with  the  larger  class  group).  While  more  research  is  needed,  our  study  indicates  that 
enabling  and  encouraging  the  use  of  these  strategies  and  techniques  may  contribute  to  the  long-term  viability  and  use  of  WBI  in 
institutions  of  higher  education. 

Introduction 

The  use  of  electronic  technologies  for  the  delivery  of  instruction  has  grown  at  an  exponential  rate  over  the  last  five  years. 
Institutions  of  higher  education,  corporations  and  K-12  environments  continue  to  seek  ways  to  use  on-line  tools  to  deliver 
instruction.  Concurrently,  the  technological  infrastructure  is  expanding  in  terms  of  its  capabilities  and  power  (Daniel,  1998; 
Katz,  1999),  increasing  learner  access  to  the  technologies  needed  to  acquire  and  share  information  with  other  participants.  This 
convergence  in  interest  by  educators  and  learners  in  learning  via  distance  technologies  has  enabled  an  exponential  increase  in  the 
quantity  of  distance  courses  and  programs  across  disciplines  (e.g.,  art,  history,  information  systems,  education,  science)  and 
contexts  (Otchet,  1998). 

While  the  increased  interest  and  need  is  an  exciting  development,  several  challenges  associated  with  the  successful 
implementation  of  Web-based  instruction  (WBI)  remain  unresolved,  including  retention  and  high  dropout  rates  (see  Barley, 
1999;  and  Hill,  in  press,  for  an  overview  of  several  issues).  Retention  has  historically  presented  challenges  for  distance 
educators.  According  to  Moore  & Kearsley  (1996),  dropout  rates  have  ranged  from  30  - 50%.  While  this  figure  is  inclusive  of  a 
variety  of  distance  learning  technologies  (video,  print,  etc.)  and  the  authors  point  out  that  the  dropout  rate  would  now  be  at  the 
lower  end,  30%  is  still  a considerable  percentage  to  lose  in  a learning  experience.  WBI,  with  its  high  demands  psychologically 
and  technically,  makes  this  challenge  even  more  significant. 

Several  factors  may  contribute  to  retention  challenges  in  distance  education.  Factors  mentioned  in  the  literature  include:  lack 
of  prior  experience  with  distance  learning,  external  demands,  and  conflicts  with  motivation  demands  (external  vs.  internal)  (see, 
for  example,  Carr,  2000).  Another  explanation  for  high  dropout  rates  and  dissatisfaction  with  distance  delivered  courses  may 
relate  to  a lack  of  a perception  of  community  in  courses  that  are  not  face-to-face.  Students  may  feel  like  they  are  isolated, 
creating  an  experience  of  lack  of  presence  from  others  involved  in  the  course. 

Perception  of  a community  may  assist  learners  with  feeling  connected  or  belonging  (Halaby,  2000;  Joyce  & Weil,  1996). 
Research  in  on-line  environments  indicates  that  community  building  can  occur  in  distance  delivered  courses  (Hill,  1999a;  Palloff 
& Pratt,  1999),  much  like  community  building  can  occur  in  virtual  teams  in  the  business  sector  (Lave  & Wenger,  1991;  Raven, 
1999).  Given  that  a sense  of  a community  has  been  demonstrated  to  contribute  to  group  performance  within  a corporate  context 
(Lave  & Wenger,  1991),  it  may  prove  to  be  a benefit  in  a learning  context.  Discovering  the  best  strategies  and  techniques  for 
community  building  may  lead  to  enhanced  course  outcomes  (e.g.,  retention,  satisfaction,  learning  outcomes)  by  participants  in 
WBI. 

Purpose 

The  purpose  of  this  study  was  to  explore  best  practices  for  community  building  in  WBI.  In  doing  so,  the  study  sought  to 
examine  specific  strategies  and  techniques  designed  to  facilitate  the  establishment  of  an  on-line  community.  The  study  was 
guided  by  the  following  general  research  question: 

What  are  the  best  techniques/strategies  to  enable  community  building  in  WBIl 

This  question  was  addressed  through  a number  of  sub-questions,  two  of  which  will  be  focused  upon  in  this  paper: 

• What  can  we  do,  as  designers  of,  and  instructors  in,  a WBLE  to  assist  the  learner  in  the  effective  building  of 
community  while  learning  in  a Web-based  environment? 
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• What  strategies  can  learners  use  to  assist  themselves  (individually  and  with  each  other)  in  community  building  while 
engaged  in  learning  in  Web-based  environments? 

Significance  of  the  Study 

While  considerable  research  has  been  conducted  in  the  general  area  of  distance  learning,  research  specific  to  Web-based 
environments  for  learning  has  only  recently  been  published  (see,  for  example.  Dehoney  & Reeves,  1999;  Khan,  1997;  Hill, 
1997a;  Hill,  1999a;  Owston,  1997;  Pritchard,  1998),  and  most  is  being  presented  at  a theoretical  rather  than  an  empirical  level. 
As  the  Web  and  Internet -based  technologies  (e.g.,  bulletin  boards,  email,  CUSeeMe,  streaming  video,  instant  messaging) 
continue  to  grow  in  popularity  and  use  in  higher  education,  we  felt  that  institutions  would  benefit  greatly  from  investigation  of 
best  practices  related  to  WBI,  in  our  case,  specifically  examining  best  practices  for  community  building. 

Interest  in  building  community  is  certainly  not  new,  nor  is  it  something  isolated  to  study  in  the  context  of  higher  education. 
Lave  and  Wenger  (1991)  have  spent  considerable  time  examining  the  issues  related  to  forming  community  in  a business  and 
industry  setting.  Joyce  and  Weil  (1996)  called  for  the  creation  of  a communities  of  professional  educators  within  a school 
setting.  Halaby  (2000)  brings  the  notion  of  belonging  into  the  classroom  setting,  emphasizing  a need  to  help  students  belong. 
More  recently,  Palloff  and  Pratt  (2001)  extended  their  work  on  community  building  within  higher  education  settings,  focusing  on 
providing  hints  and  tips  for  the  online  teacher. 

Certainly,  this  work  is  useful  and  adds  to  our  literature  base.  However,  much  of  the  work  completed  to  date  is  primarily 
theoretical,  and  while  based  in  experience,  is  not  primarily  driven  by  empirical  research.  Further,  the  current  work  does  not 
define  specific  models  for  how  to  enable  community-building  in  a Web-based  environment  --  both  from  the  teacher  and  student 
perspective.  We  need  datardriven  strategies  and  models,  presenting  techniques  on  both  sides  of  the  desk,  so  that  others  can  test 
the  robustness  of  the  models  in  a variety  of  environments. 

Research  Plan 

Research  Design 

An  embedded  case  study  design  was  used  for  this  study,  involving  the  use  of  multiple  cases,  or  embedded  units,  within  a 
larger  context.  The  unit  of  study  in  the  case  was  the  individual  faculty  member  or  student  involved  in  the  WBI  implementation. 
Multiple  sources  of  evidence  were  used  to  triangulate  the  data,  thus  addressing  possible  concerns  with  internal  validity  (Yin, 
1994).  This  approach  has  been  used  by  one  of  the  researchers  in  previous  research  (Hill  & Hannafin,  1997;  Hill,  1997b),  and  has 
proven  successful  when  looking  to  describe  rich  contexts,  and  for  model  development  (Hill,  1999b). 

Selection  and  Description  of  the  Participants 

Two  groups  of  participants  were  engaged  in  this  study.  One  consisted  of  a an  instructor  and  students  involved  in  the  Master's 
level  course  Information  Technology  Infrastructures  in  a college  of  business  at  a university  in  a large  metropolitan  area.  The 
other  consisted  of  an  instructor  and  Master's  level  students  involved  in  the  course  Instructional  Design  in  a college  of  education  at 
a university  in  a rural  area.  The  population  included  university  instructors,  instructional  design  experts  and  working 
professionals  returning  to  school  from  various  sectors  of  business  and  industry  (e.g.,  information  technology  management, 
technical  support,  Web  development)  and  education  (K-12  and  higher  education). 

The  courses  were  selected  for  two  primary  reasons: 

• Involvement  in  the  courses  was  voluntary.  Although  for  many  students  the  courses  were  required  for  completion  of  the 
degrees,  they  decided  when  and  how  to  take  the  course.  Most  students  enter  with  a high  level  of  interest  and 
motivation. 

• Learners  begin  the  course  with  a variety  of  backgrounds,  as  well  as  differences  in  their  technology  experience.  This 
variety  is  essential  for  examining  strategies  and  techniques  across  potential  students. 

Two  groups  comprised  the  sample  of  this  study: 

• university  faculty  as  subject  matter  experts  to  help  inform  the  design,  development,  and  implementation  of  the  courses; 
and 

• students  enrolled  in  the  courses  during  summer  term. 

WBI  Development 

Measures  and  Instrumentation 

A combination  of  positivistic  and  interpretivist  techniques  were  used  in  gathering  evidence  for  the  study.  Various 
instruments  were  used  to  facilitate  data  collection  for  the  study:  surveys,  interviews,  observations,  and  content  analysis  of 
discussion  transcripts.  Positivistic  techniques  were  used  to  generate  individual  difference  measures  for  each  case.  Interpretivist 
techniques  were  used  to  monitor  the  use  of  community-building  strategies  and  techniques. 

Settings  and  Procedures 

Implementation  of  the  courses  took  place  over  a 7-8  week  period  during  the  summer  of  2000.  Data  were  collected  in  a 
variety  of  environments.  Pilot  testing  with  learners  in  the  spring  and  data  gathering  with  learners  in  the  summer  took  place  in  the 
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environment  in  which  the  WBI  was  used,  including  campus  computer  labs  and  the  learners'  homes/places  of  employment 
(depending  on  where  they  have  access  to  the  Web).  A combination  of  questionnaires,  observations,  interviews,  and  content 
analysis  of  transcripts  from  on-line  discussions  were  used  to  gather  data  from  students.  The  facilities  and  necessary  equipment 
for  data  gathering  were  fully  established  at  each  institution. 

Analysis 

To  the  extent  possible,  the  collection,  organization,  and  initial  analysis  of  data  occurred  concurrently.  Previous  research 
indicates  that  this  assists  with  indicating  gaps  in  data  as  they  are  gathered  and  allow  for  adaptations  in  the  process  (e.g.,  need  for 
additional  information)  (Glaser  & Strauss,  1967;  Hert,  1992;  Hill  & Hannafin,  1997).  One  "gap"  that  did  occur  related  to  the 
number  of  participants.  We  did  experience  a reduction  in  participants  in  both  courses,  with  the  final  number  of  participants  being 
21  in  the  Information  Technology  Infrastructures  course  and  22  participants  in  the  Instructional  Design  course. 

In-depth  data  analysis  took  place  throughout  the  academic  year  following  the  offering  of  the  courses.  One  level  of  in-depth 
analysis  involved  reading  through  and  coding  the  transcripts  from  the  online  chat  and  bulletin  board  discussions.  As  the 
researchers  read  the  data,  pre-established  codes  were  used  to  mark-up  the  data  (Ericsson  & Simon,  1984;  Hill  & Hannafin, 
1997).  Additional  codes  were  established  as  themes  and  patterns  not  readily  applicable  to  the  established  categories  emerged. 

Another  level  of  in-depth  analysis  involved  chunking  sections  of  the  data  related  to  specific  research  questions  according  to 
pre-established  strategies  and  techniques  for  community  building  (Hill,  1999b).  Pattern  matching  was  used  to  inform  the 
generation  of  an  overall  list  of  strategies  and  techniques  — instructor  and  student  — for  community  building  in  WBI,  enabling  the 
creation  of  a theoretical  model  for  community-building  in  WBI.  These  coding  and  analysis  techniques  have  been  documented  in 
the  literature  (Bogdan  & Biklen,  1992;  Krathwohl,  1998;  Yin,  1994)  and  were  used  by  the  researchers  in  previous  studies  (Hill, 
1997b;  Hill  & Hannafin,  1997). 

Data  Presentation 

The  content  of  each  student’s  and  instructor’s  posting  was  analyzed  to  determine  the  number  and  type  of  constructs  was 
examined.  Thirteen  constructs  in  participants’  posting  were  identified:  active  interaction,  socially  constructed  meaning, 
expressions  of  support  and  encouragement,  collaborative  learning,  sharing  information,  acknowledgement  of  others,  chitchat, 
teacher  initiative,  student  initiative,  teacher  response,  student  response,  student  evaluation,  and  teacher  evaluation. 

To  help  inform  the  results,  the  data  were  first  divided  into  two  main  categories:  infrastructure  strategies  and  interaction 
strategies.  Codes  and  definitions  for  strategies  included  in  these  categories  are  displayed  in  Table  1 . 


Codes 

Constructs 

Definition 

AI 

Active  Interaction 

Involving  both  course  content  and  personal  communication; 
purposeful,  engaged,  energetic 

SCM 

Socially  Constructed  Meaning 

Agreement  or  questioning  with  the  intent  to  achieve  agreement  on 
issues  of  meaning 

ESE 

Expressions  of  Support  and  Encouragement 

Encouraging  comments  to  their  classmates.  Exchanged  between 
students. 

CL 

Collaborative  Learning 

Sharing  ideas  and  knowledge  among  students. 

Comments  directed  primarily  student  to  student  rather  than  student 
to  instructor 

SI 

Sharing  Information/Resources 

Among  students  (resources  = container)  Integration  of  people, 
resources 

ACK 

Acknowledgement  of  others 

Noting  presence  of  person 

CC 

Chit  Chat 

Social  interactions  not  related  to  class 

Table  L Codes  for  infrastructure  strategies 


We  also  divided  the  interaction  strategies  categories  (data)  into  two  other  categories:  instructor  strategies  and  student 
strategies.  Codes  and  definitions  for  strategies  included  in  these  categories  are  displayed  in  Tables  2 and  3. 


Codes 

Constructs 

Definition 

TI 

Teacher  Initiative 

Asking  question  to  lead  and  facilitate  the  discussion 

TR 

Teacher  Response 

Answering  questions  to  respond  to  student 

TE 

Teacher  Evaluation 

Teacher  evaluation  about  student  response 

Table  2.  Codes  for  interaction  strategies  - instructor  strategies 


Codes 

Constructs 

Definition 

TI 

Teacher  Initiative 

Asking  question  to  lead  and  facilitate  the  discussion 

SI 

Student  Initiative 

Asking  question  to  negotiate  or  clarify  the  source  and  extent  of 
disagreement 

TR 

Teacher  Response 

Answering  questions  to  respond  to  student 

SR 

Student  Response 

Answering  questions  to  respond  to  teacher 

SE 

Student  Evaluation 

Willingness  to  critically  evaluate  work  of  others 

TE 

Teacher  Evaluation 

Teacher  evaluation  about  student  response 

Table  3.  Codes  for  interaction  strategies  - student  strategies 


We  also  analyzed  the  data  from  a qualitative  perspective,  seeking  to  understand  how  events  occurred  overtime.  Figures  1 and 
2 exemplify  the  graphs  we  made  to  help  inform  this  level  of  analysis. 


Figure  1.  Coding  of  all  chat  sesions  for  the  Information  Infrastructures  course 
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Figure  2.  Coding  of  all  chat  sesions  for  the  Instructional  Design  course 

After  coding  of  the  transcripts  was  complete  and  initial  patterns  were  emerging,  the  researchers  then  looked  at  other  data  sources 
as  points  of  triangulation  of  the  findings.  Focus  group  interview  notes  were  reviewed;  end  of  term  surveys  were  analyzed. 
Trends  in  these  data  were  then  compared  to  the  trends  and  patterns  established  from  the  discussion  transcripts.  Strategies  and 
techniques  were  adjusted  as  needed  based  on  the  data. 

Findings  and  Results 

Analysis  of  the  data  was  an  intensive  task,  cognitively  and  logistical ly.  Many  hours  were  spent  by  the  researchers, 
individually  and  as  a team,  reviewing  and  refining  our  findings.  The  results  of  our  efforts  are  presented  below,  organized 
according  to  our  research  questions. 

What  can  we  do,  as  designers  of  and  instructors  in,  a WBLE  to  assist  the  learner  in  the  effective  building  of  community  while 
learning  in  a Web-based  environment? 

Based  on  feedback  from  the  learners  during  implementation,  as  well  as  expert  review  of  the  course,  there  are  several  things 
that  designers  and  instructors  can  do  during  the  design,  development  and  implementation  stages  to  help  with  community  building 
in  WBI. 

One  strategy  that  proved  very  effective  was  ensuring  that  learners  have  sufficient  opportunities  to  interact  with  each  other  as 
well  as  with  the  instructor.  Our  research  indicates  that  learners  want  a variety  of  ways  to  interact  with  each  other.  Learners  in  the 
Information  Infrastructures  course  and  the  Instructional  Design  course  indicated  that  all  of  the  communication  technologies  (e.g., 
chat,  bulletin  boards,  email,  phone)  were  working  well  and  were  important  for  facilitating  interactions. 

Another  strategy  used  in  the  Instructional  Design  course  that  proved  effective  was  the  use  of  CSM  messages.  CSM  messages 
indicated  to  learners  what  they  Could  be  doing,  what  they  Should  be  doing,  and  what  they  Must  be  doing.  These  messages  were 
sent  out  by  the  instructor  once  or  twice  a week  to  remind  learners  of  tasks  for  the  week.  During  the  mid-term  evaluation,  learners 
indicated  that  the  CSM  messages  were  important  not  only  for  keeping  them  on  track,  but  also  for  letting  them  know  that  the 
instructor  was  there. 

Yet  another  strategy  that  proved  effective  for  community  building  was  the  use  of  teams  for  completing  work.  In  the 
Information  Infrastructures  course,  teams  of  2-4  worked  together;  in  the  Instructional  Design  course,  teams  of  2 worked  together 
(i.e..  Design  Buddies).  In  both  instances,  the  learners  indicated  that  the  team  members  contributed  very  positively  to  their  sense 
of  belonging  and  a sense  of  connection  with  others  in  the  course. 
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There  were  also  several  strategies  built  into  the  structure  of  the  Web  sites  to  assist  with  community  building.  These  included: 
access  to  multiple  communication  technologies,  posting  of  announcements  and  "what's  new"  updates,  and  pa-sonal  Web  pages 
for  each  learner  that  included  a picture  and  biographical  information.  All  of  these  approaches  appear  to  have  contributed  to  the 
ability  of  a community  to  form  as  the  learners  interacted  within,  and  used,  the  WBI  environment  for  learning. 

What  strategies  can  learners  use  to  assist  themselves  (individually  and  with  each  other)  in  community  building  while  engaged  in 
learning  in  Web-based  environments? 

Analysis  of  surveys,  as  well  as  transcripts  from  various  interactions  in  the  courses,  indicate  that  several  strategies  were  used 
by  learners  to  assist  themselves  with  community  building  and  learning  in  a WBI  environment.  A strategy  used  by  several 
learners  was  a daily  visit  to  the  Web  site  to  check  for  new  messages  on  the  bulletin  boards.  While  many  learners  indicated  that 
this  was  frustrating  ("takes  too  much  time"),  others  stated  that  the  frequent  visits  helped  them  with  establishing  a sense  of 
belonging  to  the  course. 

Two  other  closely  related  community  building  strategies  used  by  learners  are  encouragement  and  support.  Evidence  of  this 
was  seen  throughout  bulletin  board  postings,  chat  room  interactions  and  e-mail  messages  between  team  members/Design  Buddies 
as  well  as  between  WBI  participants  in  general. 

Several  learners  indicated  that  the  experience  was  somewhat  overwhelming.  This  comment  related  mainly  to  the  number  of 
messages  learners  had  to  read  on  bulletin  boards,  in  chat  sessions  and  in  e-mail.  One  strategy  mentioned  by  several  learners  was 
that  of  scanning;  that  is,  reading  for  content  not  for  detail,  in  order  to  keep  the  information  exchange  manageable. 

What  are  the  best  techniques/strategies  to  enhance  community  building  in  WBI? 

Overall,  several  strategies  and  techniques  for  community  building  in  WBI  emerged  from  the  initial  analysis.  The  strategies 
and  techniques,  compiled  across  those  discussed  in  the  research  sub-questions,  have  been  divided  into  two  main  areas: 
infrastructure  and  interactions  (see  Table  4). 


Infrastructure  Strategies 

Interaction  Strategies 

Access  to  multiple  communication  technologies. 

Read  for  content  not  for  detail. 

Posting  of  announcements  and  "what's  new"  updates. 

Encourage  and  support  fellow  learners  in  their  efforts. 

Personal  Web  pages  for  each  learner. 

Use  CSM  messages  to  indicate  to  learners  what  they  Could  be 
doing,  what  they  Should  be  doing,  and  what  they  Must  be 
doing  in  terms  of  the  course. 

Learners  have  sufficient  opportunities  to  interact  with  each 
other  as  well  as  with  the  instructor. 

Use  of  teams  for  completing  work  in  the  course. 

A daily  visit  to  the  Web  site  to  check  for  new  messages  on  the 
bulletin  boards. 

Table  4.  Strategies  and  techniques  for  community  building  in  on-line  environments. 


Infrastructure  strategies  are  the  responsibility  of  the  instructor.  They  create  the  environment  that  enables  or  inhibits  the 
formation  of  community.  Interaction  strategies  take  place  during  the  class,  they  involve  actions  from  both  the  students  and  the 
instructor/facilitator. 

We  have  also  represented  the  strategies  in  terms  of  target  audience,  i.e.,  instructor  or  learner  (see  Table  5). 


Instructor  Strategies 

Learner  Strategies 

Provide  multiple  opportunities  for  interaction. 

Visit  the  course  Web  site  daily  (or  every  other  day  at  a 
minimun). 

Send  out  management  related  messages  (e.g.,  CSMs)  on  a 
regular  basis. 

Provide  encouragement  and  support. 

Establish  teams  so  that  learners  work  together  to  complete 
tasks 

Scan  material  posted  on  the  Web  site  - do  not  read  for  detail. 

Keep  the  Web  site  up  -to-date  and  add  in  new  information  on  a 
regular  basis  to  keep  things  "fresh." 

Table  5.  Instructor  and  Learner  Strategies  for  community  building  in  on-line  environments. 

Cominunity  Building:  A Process-based  Model 

It  would  appear  that  some  level  of  community  building  did  occur  in  both  the  Information  Infrastructures  and  the  Instructional 
Design  courses.  Specifically,  by  looking  at  graphs  of  the  interaction  data  (as  recorded  in  the  bulletin  board  and  chat  transcripts 
(see  examples  in  Figures  1 and  2)),  we  see  evidence  that  community  was  established  over  time  and  that  there  were  certain  points 
in  the  term  where  participants  were  "closer"  in  terms  of  a community  than  others. 
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The  analysis  completed  to  date  has  enabled  the  creation  of  the  beginnings  of  a theoretical  model  for  community  building  and 
its  potential  relationship  to  learning  (see  Figure  3).  While  the  model  is  not  complete,  it  does  exemplify  a significant  step  toward 
identification  of  strategies  and  techniques  that  can  enable  the  creation  of  community  in  WBI. 


Figure  3.  Creating  Community  for  Learning 

The  model  in  Figure  3 illustrates  how  community  building  strategies  may  lead  to  community  building  activities  that  take 
place  during  a class,  which  leads  to  the  existence  of  community.  The  existence  of  community  is  predicted  to  result  to  higher 
levels  of  learning.  This  learning  can  be  measured  through  the  coding  of  instances  of  socially  constructed  meaning,  and 
collaborative  learning. 

As  discussed  earlier,  in  the  design  phase  of  a course,  the  instructor  can  implement  a number  of  strategies  that  provide  a basic 
infrastructure  for  community  building.  During  the  class,  both  instructor  and  students  can  use  interaction  strategies  for  community 
building.  The  two  sets  of  strategies  lead  to  interactions  during  the  class,  again  by  students  and  instructor,  and  these  interactions 
lead  to  the  emergence  of  a community. 

Discussion  & Suggested  Next  Steps 

This  study  investigated  strategies  and  techniques  for  community  building  in  WBI.  Overall,  participants  in  both  courses 
indicated  some  sense  of  community,  albeit  limited  in  scope  (i.e.,  the  learners  indicated  a stronger  connection  with  their  team 
members  than  with  the  larger  class  group).  While  more  research  is  needed,  our  study  indicates  that  enabling  and  encouraging  the 
use  of  these  strategies  and  techniques  may  contribute  to  the  long-term  viability  and  use  of  WBI  in  institutions  of  higher 
education. 

As  we  continue  work  in  this  area,  we  are  also  beginning  to  consider  different  questions  and  issues  to  explore.  One  issue  we 
offer  as  an  area  for  additional  study  is  that  of  adjustments  to  change.  As  compared  to  face  to  face  classes,  WBI  courses  place 
more  demands  on  both  instructor  and  students  in  terms  of  amount  of  time  worked,  and  number  of  student -instructor  and  student  - 
student  interactions.  As  with  regular  face  to  face  classes,  it  is  better  to  wait  with  the  evaluation  of  a WBI  course  until  it  has  been 
taught  at  least  twice.  The  first  time  around  is  typically  a learning  experience  for  the  instructor  and  the  learner;  better  to  use  this  as 
a vehicle  to  determine  what  works  and  what  doesn’t,  and  then  make  additional  judgments  the  second  — or  third  — time  around. 

We  also  recognize  the  need  for  assisting  others  in  the  implementation  of  community  building  strategies  in  their  own  WBI 
experiences.  Models  and  tools  to  assist  with  this  effort  would  go  a long  way  toward  helping  others  interesting  in  building 
community  in  their  own  WBI.  We  are  currently  developing  a model  for  community  building  in  WBLEs,  and  encourage  others  to 
also  explore  this  area. 

Conclusion 

By  delivering  engaging  and  meaningful  instruction  in  WBI  environments,  the  university  can  increase  its  visibility  and 
viability  in  the  21st  century  educational  arena.  Further,  results  from  this  study  can  be  used  to  guide  and  facilitate  the  design, 
development,  and  implementation  of  WBI  to  increase  interaction  and  engagement.  This,  in  turn,  can  potentially  lead  to  higher 
retention  (our  experience:  94%)  and  satisfaction  (our  experience:  high)  in  on-line  courses  at  institutions  of  higher  education. 
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Abstract 

This  was  a case  study  analysis  of  instructional  designers  and  their  preparation  for  professional  practice.  Seven  instructional 
designers  were  selected  to  participate  in  this  study  based  on  the  following  criteria:  employment  at  a company  developing 
instructional  materials  for  external  clients,  three  to  five  years  experience,  and  attendance  at  a graduate  program  of  instructional 
design  or  technology. 

Each  participant  was  interviewed  on  two  occasions  using  a semi-structured  interview  format.  Case  studies  were  presented  on 
the  analysis  of  the  interviews  and  quotes  were  used  to  accurately  express  the  perceptions  and  feelings  of  each  participant.  The 
cross-case  analysis  identified  major  themes,  which  emerged  from  the  interviews. 

Individual  differences  as  well  as  common  perspectives  on  preparation  and  professional  practice  were  noted  between  the 
participants.  This  study  confirmed  prior  research  into  the  preparation  of  instructional  designers  for  professional  practice.  The 
theoretical  body  of  knowledge  as  well  as  the  practical  knowledge  and  competencies  used  to  solve  problems  in  a variety  (f 
environments  define  this  practice.  This  research  identified  three  profiles  of  instructional  design  practice  that  correlate  with  the 
expert-novice  continuum:  Innovator,  Traditionalist,  and  Practitioner.  This  study  also  identified  instructional  designers' 
perceptions  of  effective  graduate  preparation  for  instructional  design.  Recommendations  for  instructional  technology  programs 
and  for  future  research  are  provided. 

Background 

Instructional  Design:  Science  and  art.  A blank  canvas  and  a variety  of  paintbrushes  and  pigments  are  merely  tools;  the 
paintbrushes  form  textures  and  the  pigment  color.  Their  potential  to  become  a work  of  art  lies  in  the  artist's  knowledge  about 
materials  and  composition,  intuition  and  creativity.  Applied  to  the  canvas,  textures  form  layers  that  the  artist  explores  and  adapts 
into  shapes  and  meanings  until,  like  pieces  of  a puzzle,  they  fall  into  place  and  form  a personal  message.  Instructional  Design 
(ID)  practice  is  also  both  science  and  art.  Its  blank  canvas  is  an  organization's  need  or  learning  opportunity  and  the  potential  for 
an  effective  solution.  Just  as  the  painter's  tools  fill  a canvas  with  texture  and  color,  a multitude  of  systematic  models  outline  the 
ID  process  and  methods  adapted  from  a variety  of  disciplines  are  available  to  the  designer.  Their  potential  as  tools  for  effective 
practice  lie  in  the  instructional  designer's  knowledge,  creativity,  intuition,  and  analytical  skills.  Applied  to  a design  problem,  the 
ID  tools  form  layers  that  the  instructional  designer  explores  and  adapts  until,  like  pieces  of  a puzzle,  they  fall  into  place  and  form 
an  innovative  solution  that  meets  target  audience  needs. 

Instructional  design  is  a young  discipline  commonly  defined  by  its  scholars  as  a systematic  and  reflective  problem-solving 
process  used  to  analyze  instructional  and  informational  needs,  identify  solutions,  and  design  and  develop  support  for  learners  of 
any  age,  in  any  environment.  ID  borrows  from  a broad  array  of  theories,  research,  and  experience-based  models  in  the  fields  of 
engineering,  education,  psychology,  cognitive  science  and  business,  among  others. 

An  instructional  designer  is  like  an  engineer.  Both  plan  their  work  based  upon  principles  that  have  been  successful  in  the 

past...  Both  try  to  design  solutions  that  are  not  only  functional  but  also  attractive  or  appealing  to  the  end  user.  Both  the 

engineer  and  the  instructional  designer  have  established  problem-solving  procedures  they  use  to  guide  them  in  making 

decisions  about  their  designs  (Smith  <&  Ragan,  1999,  p.2) 

Instructional  design  literature  often  equates  the  birth  of  ID  with  the  onset  of  World  War  II  when* educational  psychologists 
and  educational  technologists  joined  forces  to  meet  the  training  needs  of  the  military  (Richey,  1993;  Saettler,  1990).  ID  practice 
has  since  evolved  to  address  organizations'  needs  in  environments  that  range  from  K-12  to  higher  education,  government  and 
military  training,  to  a variety  of  corporate  settings  found  in  business  and  industry.  Instructional  designers  in  K-12  or  higher 
education  environments  are  often  placed  in  positions  as  educational  technologists  with  the  responsibilities  that  range  from 
curriculum  development  to  incorporation  of  new  teaching  and  learning  pedagogues.  Military -based  instructional  designers  may 
need  to  develop  training  materials  for  classroom  and  field  training  but  must  function  within  a well-established  hierarchy  and 
policies  that  dictate  performance  objectives  and  goals.  Finally,  instructional  designers  in  corporate  environments  may  contend 
with  novel  situations  that  vary  from  project  to  project.  In  internal  training  departments,  this  often  includes  development  of  in- 
house  training  or  information  support  materials.  In  the  case  of  consulting  companies,  materials  are  designed  and  developed  with 
or  for  external  clients. 

The  models  used  to  describe  the  ID  process  can  be  traced  back  to  the  early  1 960s  with  the  introduction  of  the  first  visual 
model.  Instructional  Systems  in  1961  (Banathy,  1968).  Using  flow-charts  and  descriptive  text,  this  model  presented  the  stages  of 
instruction  systems  design  as  analysis  of  a system,  solutions  for  the  system’s  problems,  and  subsequent  development  of  a new 
system.  The  Instructional  Systems  model  emphasized  analysis  as  an  iterative  process  rather  than  a "linear- causal  orientation  of 
classical  scientific  thinking"  (Banathy  in  Gagne,  1985,  p.87) 

The  iterative  nature  ID  practice  remained  a guiding  factor  in  the  numerous  instructional  design  models  that  were  created 
following  the  publication  of  Banathy's  Instructional  Systems  model  (Gustafson,  1981;  Hannafin,  1986;  Reigeluth  & Nelson, 
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1997;  Richey,  1993;  Rowland,  1993;  Seels,  1989),  Each  model  was  an  effort  to  reflect  the  evolution  of  the  field  and  advances  in 
learning  theory,  cognitive  science,  and  system  theory.  Several  adaptations  were  introduced  to  the  ID  process,  including  additions 
that  helped  streamline  the  process  such  as  rapid  prototyping  and  strategies  such  as  performance  support  that  helped  create 
materials  appropriate  to  today's  work  environments.  Technologies  available  for  learning  materials  changed  rapidly  as  did  the 
opportunities  for  mass  distribution  to  learning  audiences  via  computer  technologies  and  the  World  Wide  Web,  In  spite  of  these 
adaptations,  the  conceptual  framework  of  instructional  systems  design  models  remained  virtually  unchanged  since  their 
development  in  the  1970s.  They  continued  to  emphasize  the  components  that  defined  the  ID  process;  analysis,  design, 
development,  and  evaluation  (Dick  & Carey,  1996;  Gagne,  Briggs,  & Wager,  1988;  Richey,  1986;  Smith  & Ragan,  1993), 

Research  in  ID  Preparation  and  Practice.  During  the  years  that  followed  the  introduction  of  the  first  systems  design  models 
for  instruction,  higher  education  institutions  began  offering  graduate  programs  in  instructional  technology  and  included 
instructional  design  as  a part  of  their  core  curriculum.  Many  of  these  programs  evolved  from  audio-visual  or  communications 
programs.  The  preparation  of  instructional  designers  was  based  on  prescriptive  theories  that  focus  on  what  designers  should  do 
when  designing  instructional  in  actual  practice.  ID  courses  treated  the  ID  process  as  a procedural,  linear  model  used  to  solve 
design  problems.  This  approach  led  the  instructional  designer  through  a process  of  solution  identification,  moving  progressively 
from  the  problem  towards  the  desired  goal  using  a top  down,  procedural  process  (Ertmer  & Cennamo,  1995). 

As  in  any  emergent  profession,  each  environment  of  practice  has  helped  shape  and  define  ID  practice  and,  ideally,  each 
environment  of  practice  should  influence  the  professional  preparation  of  its  practitioners.  This  was  the  focal  area  of  research  into 
professional  preparation  in  the  1980s  and  1990s.  We  learned  how  expert  instructional  designers  identify  problems  and  interpret 
instructional  needs  (Rowland,  1991,  1992;  Perez,  1995,  Perez  and  Emery,  1995).  We  explored  the  dynamics  of  professional 
practice  from  designers  who  were  from  a variety  of  educational  backgrounds  (Pitlik,  1995;  Atchison,  1996)  and  we  learned  about 
the  diverse  nature  of  ID  practice  (ASTD,  1983;  IBSTPI,  1986,  2000;  Atchison,  1996), 

The  competencies  that  were  identified  covered  a full  range  of  procedural,  problem-solving,  collaborative,  management  and 
technology  skills  and  the  dynamics  of  professional  practice  pointed  to  a need  for  increased  focus  on  the  transfer  of  theory  into 
practice.  Researchers  recommended  that  graduate  instructional  technology  (IT)  programs  provide  learning  experiences  reflective 
of  the  complex  environments  found  in  actual  professional  ID  practice  (Quinn,  1995;  Tripp,  1994).  Their  recommendations 
promoted  such  methods  as  real-world  projects,  case  studies,  and  apprenticeship  models  (Rowland,  1992;  Roland,  Parra,  Basnet, 
1994;  Perez,  1995;  Quinn,  1994).  ID  instructors  implementing  these  strategies  began  to  explore  coaching  methods  that  would 
encourage  their  students  to  consider  multiple  perspectives  and  reflect  on  their  design  decisions  (Tripp,  1991,  1994;  Ertmer  & 
Cennamo,  1995).  Many  of  the  strategies  were  adopted  from  other  design  fields,  such  as  architecture  and  engineering,  which  had 
successfully  implemented  open-ended  learning  experiences  that  encouraged  reflective  practice.  Their  interest  was  to  incorporate 
some  of  the  skills  that  research  into  professional  practice  and  cognitive  learning  theory  uncovered  such  as  collaboration,  problem- 
solving, management,  and  effective,  yet  flexible  implementation  of  the  ID  process  (Rowland  et  al,,  1994). 

With  instructional  technology  programs  across  the  country  exploring  ways  to  adapt  their  core  curriculum  in  this  way, 
questions  about  the  relationship  between  graduate  IT  programs  and  professional  practice  emerged.  What  aspects  of  the  graduate 
IT  education  experience  did  instructional  designers  find  effective  in  preparing  them  for  professional  practice?  Were  they  able  to 
transfer  the  core  knowledge,  strategies  and  methods  they  learned  to  actual  practice?  Or,  did  they  struggle  to  draw  connections 
between  their  graduate  education  and  professional  practice?  The  purpose  of  this  paper  is  to  present  a comparative  case  study  that 
explored  the  complex  relationship  between  preparation  and  practice  through  an  examination  of  instructional  designers  and  their 
perceptions  of  academic  preparation.  Seven  instructional  designers  in  ID  consulting  firms  provided  insight  into  their  academic 
and  professional  experiences  and  their  recommendations  for  graduate  IT  programs.  Through  interviews,  observations,  and 
product  reviews,  this  study  attempted  to  examine: 

• What  instructional  designers  in  this  environment  actually  do, 

• What  parts  of  their  academic  training  (content  and  methods)  best  prepared  them  for  this,  and 

• What  gaps  existed  in  their  professional  preparation,  as  demonstrated  by  the  knowledge  and  skills  they  had  to  develop  in 
actual  practice. 

Method 

The  exploratory  nature  of  an  inquiry  into  characteristics  of  instructional  design  practice  and  academic  preparation,  suggested 
an  interpretive  case  study  approach  using  naturalistic,  qualitative  methods.  An  interpretive  case  study  employs  naturalistic 
inquiry  to  inductively  understand  participants'  experiences  in  the  environments  in  which  they  practice.  Thick,  rich  descriptions 
illustrate  complexities  and  enable  an  in-depth  exploration  of  phenomena  (Lincoln  and  Guba,  1985;  Patton,  1990).  Broad  topics 
initiated  the  inquiry,  which  were  later  refined  to  themes  and  explicit  questions  that  led  to  a tentative  hypothesis  and  eventual 
conclusion  (Marshall  & Rossman,  1989). 

The  diversity  of  practice  environments  presents  a continuing  need  to  interpret  what  designers  do  in  various  domains  so  that 
we  may  effectively  prepare  novice  instructional  designers  for  successful  practice.  In  this  study,  I focused  on  one  specific  domain 
of  practice,  companies  that  produce  ID  products  for  external  clients.  This  enabled  me  to  gain  an  in-depth  understanding  of  the 
effectiveness  of  academic  preparation  and  the  dynamics  of  professional  practice  in  this  particular  setting.  Purposive  sampling 
identified  the  seven  academically  trained  instructional  designers  from  four  Instructional  Technology  graduate  programs.  The 
participants  had  been  working  in  this  capacity  for  no  less  than  three  and  no  more  than  five  years.  This  time  span  ensured  that  the 
designers  had  sufficient  professional  experience  to  reflect  on  actual  practice,  yet  could  still  recall  the  details  of  their  academic 
preparation. 
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In  naturalistic  inquiry,  the  design  of  the  study  is  emergent.  Data  collection  and  analysis  were  conducted  simultaneously.  In- 
depth  interviews  followed  a semi-structured  format  in  the  natural  work  setting  of  the  instructional  designers.  Questions  emerged 
to  elicit  the  designers'  approach  to  ID  practice.  The  participants  were  encouraged  to  speak  freely  about  any  topic  related  to 
academic  preparation  and  ID  practice.  They  described  adaptations  they  had  made  to  the  methods  and  strategies  they  learned  in 
graduate  school  and  what  they  felt  best  prepared  them  for  actual  practice.  A tour  of  each  participant's  work  environment,  a review 
of  recently  completed  projects,  and  company  literature  ensured  a holistic  understanding  of  the  phenomenon  under  study.  The 
participants  received  a copy  of  their  interview  transcripts  and  participated  in  a follow-up  telephone  interview  that  confirmed  the 
accuracy  of  the  transcripts  and  my  interpretations  of  their  experiences.  The  telephone  interview  also  took  the  form  of  a semi- 
structured  format,  focusing  on  questions  that  emerged  from  a preliminary  analysis  of  the  initial  interview  responses. 

All  interviews  and  supporting  materials  were  coded  based  on  the  Miles  and  Huberman  procedure,  beginning  with  an  initial 
coding  scheme  that  was  subsequently  developed  for  each  participant  after  the  first  interview.  The  constant  comparative  method  of 
Glaser  and  Strauss  (1967)  as  interpreted  for  naturalistic  inquiry  by  Lincoln  and  Cuba  (1985)  guided  analysis  of  the  data;  social 
phenomena  were  compared  across  categories  as  they  were  recorded  and  classified.  This  description,  interpretive  commentary  and 
direct  quotes  from  the  interviews  conveyed  the  meaning  of  everyday  life  at  the  site  and  ground  the  categories  with  full 
descriptions  of  the  natural  sequence  of  events  prior  to  and  during  actual  practice. 

Results  and  Discussion 

A cross-case  analysis  is  presented  according  to  the  emergence  of  themes  within  three  major  categories,  strength  of  academic 
preparation,  professional  practice,  and  skills  learned  on  the  job,  and  the  participants’  response  to  the  final  question,  "What 
recommendations  do  you  have  for  graduate  IT  programs? 

Strength  of  Academic  Preparation.  Academic  programs  that  prepare  instructional  designers  are  most  likely  to  be  in  a School 
of  Education  (Heinich,  1995).  Of  the  four  instructional  technology  graduate  programs  represented  in  this  study,  one  constituted  a 
separate  college  and  three  were  programs  housed  within  schools  of  education.  One  might  therefore  assume  that  the  majority  of 
students  entering  these  programs  would  come  from  K-12  education  backgrounds.  However,  students  enrolling  in  IT  graduate 
programs  come  from  a variety  of  disciplines,  and  many  enter  graduate  school  soon  after  they  complete  their  undergraduate 
degrees.  This  was  the  case  for  the  five  of  the  seven  participants  in  this  study  who  had  earned  degrees  in  English,  environmental 
science,  and  communications.  The  four  participants  who  spoke  about  their  decision  to  attend  graduate  school  soon  after  earning 
their  bachelor's  degrees  felt  that  there  were  limited  employment  opportunities  for  their  major.  As  Casey  explained,  "Being  an 
English  major  and  all,  I don't  think  that  left  me  a lot  of  (career)  options,"  and  she  didn't  feel  ready  to  enter  the  workforce;  "I 
wasn't  sure  what  I wanted  to  do  with  my  life  so  I went  to  graduate  school." 

Instructional  design  is  an  emergent  profession  whose  members  continue  to  define  its  practice  according  to  the  type  of 
environment  and  the  perceptions  of  those  who  have  a stake  in  the  instructional  outcome  (Ritchie,  1999).  As  a result,  students 
entering  graduate  IT  programs  often  are  unable  to  specify  precisely  what  job  they  will  hold  when  they  graduate.  All  of  the 
participants  knew  that  they  would  be  integrating  technology  with  instruction  in  their  work,  but  thought  that  their  primary 
responsibility  would  be  the  development  of  instructional  multimedia  products.  As  Anne  noted,  "My  expectation  was  that  it  was 
going  to  be  more  technical  than  it  was.  I thought  I would  be  more  of  a developer."  Instead,  these  individuals  found  that  their  work 
focused  more  directly  on  the  design  of  instruction.  Additionally,  all  but  one  participant  thought  they  would  be  designing  and 
developing  educational  software  applications  for  children.  The  exception,  Anne,  chose  her  graduate  program  because  of  her 
experience  working  for  a distance  education  program  in  a university  medical  school. 

Foundations  in  Specialized  Knowledge.  A professional  education  begins  with  an  introduction  to  the  profession’s  "specialized 
body  of  knowledge,"  those  explanatory  and  applied  theories  and  doctrines  "that  must  inform  and  guide  and  provide  a repertoire 
for  reflection"  (p.  34,  Harris,  1993)  in  practice.  Although  Schon  (1983)  argued  that  professional  expertise  results  from  practical 
knowledge,  the  development  of  the  tacit,  higher  level  problem  solving  skills  used  in  actual  practice,  he  recognized  that  this 
specialized  knowledge  within  each  profession  has  its  place.  This  "technical  knowledge,"  serves  as  the  framework  upon  which 
professionals  begin  to  build  the  knowledge  base  that  guides  the  experiences  leading  to  professional  expertise.  In  the  field  of 
instructional  technology,  this  foundation  for  professional  practice  commonly  falls  into  three  categories:  instructional  design, 
learning  theory,  and  technology.  These  are  the  predominant  topics  described  by  the  participants  in  this  study  as  forming  the 
foundation  for  their  professional  practice: 

Grasping  at  the  fundamentals...  I think  that's  basic.  Just  like  it  is  for  the  architect  who  needs  basic  math,  you  have  to  know 

that  to  go  to  the  higher  levels.  They  are  key  foundational  pieces  for  my  practice  and,  depending  on  how  the  design  builds  on 

that  is  whether  or  not  the  instruction  is  effective.  (Laura) 

Several  participants  also  considered  research  methods  a part  of  their  foundation  because  it  enabled  them  to  understand  the 
research  literature  and  theoretical  underpinnings  of  the  concepts  they  explored  in  their  other  classes.  For  some,  the  research 
methods  course  also  enabled  them  with  skills  to  interpret  research  on  current  trends  and  topics  relevant  to  products  they 
designed. 

The  participants  also  noted  two  areas  that  project  management  and  human  relations  should  be  included  in  the  foundations 
courses  for  ID  practice.  They  recognized  that  these  skills  develop  in  actual  practice,  but  felt  strongly  that  an  exposure  to  the 
management,  communication  and  interpersonal  skills  that  they  use  on  a daily  basis  would  give  instructional  designers  a 
foundation  that  they  could  build  upon.  Laura  was  the  only  participant  who  attended  a project  management  or  human  relations 
course;  she  found  that  the  case  scenarios  her  instructors  used  provided  her  with  a springboard  for  identifying  solutions  to  similar 
challenges.  She  did  not  begin  to  comprehend  the  complexity  of  management  and  conflict  resolution  until  she  applied  these  skills 
in  actual  situations. 
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Transitions  to  practice.  Some  participants  found  that  traditional  classroom  activities  such  as  presentations,  guided 
discussions,  and  reaction  or  research  papers  were  appropriate  for  their  explorations  of  the  history  of  the  field  or  the  learning 
theories  that  form  the  foundation  of  design.  Three  of  the  participants,  for  instance,  expressed  an  appreciation  toward  their 
instructor's  dynamic  lectures  and  the  breadth  of  materials  they  explored  in  their  learning  theory  courses.  Although  they  continue 
to  study  and  review  learning  theories  as  the  need  arises,  this  foundation  enabled  them  to  help  the  client  understand  and  determine 
"the  pedagogical  approach"  that  is  most  appropriate  for  the  target  audience.  Other  participants  did  not  invest  in  these  courses 
because  they  could  not  see  the  relevance  of  the  topics  they  were  studying  to  what  they  envisioned  as  professional  practice  at  the 
time. 

The  same  participants  who  appreciated  the  traditional  classroom  environment  for  their  exploration  of  theories  agreed  with 
their  counterparts,  however,  that  applied  skills  and  knowledge  should  directly  reflect  actual  practice.  ''The  real-world  projects 
and  the  combination  of  ID  with  technology  were  the  real  strength  of  our  program  (Sharon).  The  participants  identified  four 
instructional  methods  that  they  considered  appropriate  for  developing  their  skills  and  knowledge  for  professional  practice: 
discussions  with  expert  instructional  designers  and  business  owners,  explorations  of  real-world  examples,  real-world  projects, 
and  case-studies. 

Those  who  had  the  opportunity  to  participate  in  real-world  activities  expressed  an  appreciation  towards  their  instructors  for 
coaching  them  throughout  their  projects  and  helping  them  reflect  on  their  options  and  evaluate  their  decisions.  The  instructor- 
coaches  guided  the  learners  in  a review  and  reflection  on  their  decisions  and  the  resulting  outcomes.  In  doing  so,  learners 
internalize  the  design  process,  which  leads  to  a holistic  awareness  of  the  interdependence  between  its  component  parts  (Schon, 
1987).  If  an  instructional  designer  understands  the  purpose  and  interrelationship  of  the  component  parts  of  the  ID  process,  he  or 
she  understands  how  to  adapt  its  component  parts  as  needed  to  obtain  acceptable  results.  This  leads  to  effective  design  and 
development  in  almost  any  situation  (Jonassen,  1994). 

One  of  the  participants,  Casey,  for  instance,  described  her  "real-world"  ID  experience  in  an  elective  ID  class  facilitated  by  an 
instructor  who  coached  the  students  through  each  stage  of  the  Dick  & Carey  model  as  they  created  instructional  products. 
Coupled  with  the  instructor's  guided  discussions,  her  review  of  papers  they  submitted  on  related  topics,  and  her  one-on-one 
feedback,  Casey  began  to  grasp  the  purpose  of  the  stages  of  the  ID  model  and  recognized  the  value  of  the  ID  process.  "If  you 
look  at  (the  ID  process)  in  pieces,  each  of  the  pieces  is  adapted  in  some  way." 

From  Preparation  to  Practice.  An  analysis  of  the  individual  differences  and  specific  issues  connecting  perceptions  of 
academic  preparation  to  professional  practice  emerged  as  three  profiles  of  ID  practice:  Innovator,  Traditionalist,  and  Practitioner. 
Within  each  of  the  three  categories,  the  participants  reflected  similar  types  of  expertise.  The  Innovators  reflect  the  skills  and 
characteristics  used  to  describe  the  successful  ID  professional.  They  are  "Innovators"  because  they  exhibited  a common 
characteristic;  an  inner  confidence  that  enables  them  to  think  "outside-the  box"  and  motivates  them  take  the  creative  risks  that 
lead  to  innovative  solutions.  The  single  "Traditionalist"  follows  a unique  path  towards  expertise,  a traditional  model  of 
apprenticeship.  The  "Practitioners"  exhibit  some  of  the  skills  exhibited  by  the  Innovator  to  a lesser  degree,  but  tend  to  accept 
instructional  problems  from  clients  and  their  requested  solutions  at  face  value. 

Many  of  the  different  characteristics  that  describe  the  expert  to  novice  continuum  are  reminiscent  of  the  characteristics 
proposed  for  Innovator,  Traditionalist,  and  Practitioner.  Towards  one  end  of  the  continuum,  the  Innovator  exhibited 
characteristics  of  the  expert,  who  is  able  to  solve  problems  in  new  situations  by  mentally  organizing  previously  acquired 
knowledge  and  applying  it  in  a meaningful  way  (Gage  & Berliner,  1984).  According  to  the  Dreyfus  model  of  skill  acquisition, 
the  Innovator  is  similar  to  the  fifth  and  final  level  on  the  continuum  between  novice  and  expert,  appearing  flexible  and  easily 
adapting  to  the  demands  of  new  situations  (Dreyfus  & Dreyfus,  1986).  The  Traditionalist,  Jenny,  was  unique.  Although  she 
considered  herself  a novice  and  was  cautious  about  making  decisions  without  the  guidance  of  her  mentors,  she  drew  from  her  ID 
experiences  to  consider  multiple  influences  when  approaching  problems  and  alternative  solutions.  The  Traditionalist  had  begun 
to  internalize  the  rules  and  procedures  she  followed  and  thus,  as  the  proficient  performer  (level  four)  in  the  Dreyfus  model,  she 
was  beginning  to  use  her  previous  experiences  to  make  intuitive  decisions  (Dreyfus  & Dreyfus,  1986).  The  Practitioner  exhibited 
some  characteristics  of  the  novice  designer,  who  bases  his  or  her  judgments  on  surface  issues  that  are  immediately  apparent  (Chi, 
Glaser  & Far,  1988).  The  Practitioner  is  similar  to  the  advanced  beginner  (level  two)  in  the  Dreyfus  model,  drawing  from 
previous  experience,  but  requiring  guidelines  to  perform  at  an  acceptable  level  (Dreyfus  & Dreyfus,  1986). 

Innovators.  The  Innovators  considered  themselves  lifelong  learners  and  viewed  their  graduate  experience  as  a foundation  for 
practice  and  continuing  their  education.  Their  most  noteworthy  characteristic  was  their  pride  in  their  profession  and  the  products 
they  created.  As  did  the  experts  in  the  expert-novice  problem-solving  research  conducted  in  the  early  1990s,  the  Innovators  were 
not  afraid  of  ambiguity;  they  approached  each  new  challenge  anticipating  that  there  were  multiple  influences  and  solutions  to  any 
situation. 

Whether  the  project  called  for  new,  original  materials,  supplemental  activities  for  existing  content,  or  repurposed  coursework, 
the  Innovators  extracted  as  much  information  as  they  could  about  the  target  audience  and  their  learning  or  work  environments. 
Expert  instructional  designers  often  spend  more  time  than  novices  on  the  needs  assessment  because  they  interpret  problems  as  ill- 
defined  and  consider  multiple  factors  that  might  influence  the  situation.  They  also  spent  more  time  on  the  front-end  analysis  of 
the  tasks  that  the  learners  need  to  perform  (Rowland,  1992;  Tripp,  1991,  1994;  Perez,  1995;  Perez  & Emery,  1995;  Quinn,  1995). 

Experts  guide  practice  by  creating  mental  prototypes  that  integrated  prior  experiences  with  their  technical  and  practical 
knowledge  (Boreham,  1987).  This  process  requires  both  intuitive  and  analytical  skills  to  analyze  the  complex  situations  that  exist 
in  actual  practice  and  identify  novel  solutions  (Hammond,  1980;  Dreyfus  & Dreyfus,  1986).  Laura  attributed  her  ability  to  push 
the  envelope  to  her  experience  as  an  improvisation  comedienne  and  Josh  found  creative  ways  to  adapt  experiences  and 
knowledge  from  other  disciplines  to  identify  needs  and  create  new  solutions  for  his  clients.  Casey  accomplished  this  by 


212 


surrounding  herself  with  experts  (she  saw  all  of  her  colleagues  as  potential  teachers),  encouraging  collaboration  among  her  team 
members,  and  guiding  them  through  the  ID  process.  Likewise,  all  of  the  Innovators  helped  their  colleagues  see  "outside-the-box" 
by  taking  risks  and  guiding  them  in  creative  explorations  of  the  design  process.  Since  the  Innovators  were  confident  in  their 
expertise,  the  expertise  of  others  did  not  intimidate  them  and  they  welcomed  collaborators*  contributions  to  the  creative  process. 
This  aspect  of  expert  practice  has  not  yet  been  explored  in  the  research  into  expert-novice  development. 

Practical  knowledge  is  an  intuitive  artistry  involving  the  interpretation  of  theories  and  methods  within  the  expert  practitioner's 
domain  of  practice  and  adapting  them  to  meet  unique  specific  needs  (Brumbaugh  & Lawrence,  1963).  The  Innovators  were 
similar  to  Pitlik’s  (1995)  "Adapters"  in  that  they  had  confidence  in  the  ID  process,  "it  can  be  effective  in  almost  any  situation" 
(Josh),  which  they  see  as  utilitarian  rather  than  prescriptive.  They  did  not  allow  an  ID  model  to  dictate  how  they  practiced  design, 
rather  they  adapted  what  they  learned  from  the  model  to  interact  with  a situation  and  their  colleagues.  The  "Innovator"  differed 
from  the  "Adapter,"  however,  by  adapting  ID  methods  and  strategies  within  each  stage  of  the  ID  process  rather  than  adapting  to 
the  process  by  foregoing  critical  stages  of  the  model  such  as  front-end  analysis.  "If  you  look  at  it  in  pieces,  each  of  the  pieces  is 
adapted  in  some  way"  (Casey). 

Shifts  in  the  way  organizations  function  in  society,  such  as  the  technology  and  communications  explosion,  a global  economy 
and  shifting  employee  roles,  continue  to  influence  the  evolution  of  the  instructional  design  profession  (ASTD,  1993).  The 
concept  of  change  in  organizations  is  a constant,  and  organizations’  members  must  have  the  skills  necessary  to  increase  efficiency 
and  productivity  while  identifying  innovative  solutions  that  will  give  them  the  competitive  edge  in  their  market  (Davies,  1997). 
Instructional  designers  must  meet  similar  demands,  but  they  also  need  to  function  as  change  agents  who  can  guide  employees  in 
the  development  of  their  problem-solving  skills,  their  ability  to  meet  new  challenges  and  the  development  of  their  lifelong 
learning  skills.  The  Innovators  in  this  study  seemed  best  equipped  to  meet  this  demand  because  they  exhibited  the  ability  to  lead 
their  colleagues  in  the  process  of  identifying  needs  and  designing  effective  solutions.  The  Innovators  founds  ways  to  create 
efficient  and  timely  instructional  solutions  by  adapting  to  the  ID  process,  yet  they  tried  to  do  so  without  sacrificing  the  integrity 
of  the  instructional  materials  they  created. 

Traditionalist.  Like  the  Innovators,  Jenny  fully  embraced  the  ID  process  as  a collection  of  methods  and  strategies  for  defining 
problems  and  identifying  solutions.  She  considered  herself  a novice  designer,  however,  and  was  cautious  about  moving  forward 
on  her  own.  In  the  areas  in  which  she  felt  competent,  whether  note-taking  during  needs  assessments,  conducting  preliminary  task 
analyses,  or  designing  instructional  strategies,  Jenny  paid  careful  attention  to  detail  and  sought  to  emulate  the  expert  designers 
who  guided  her  projects. 

Jenny  felt  her  graduate  experience  might  have  been  more  transferable  to  professional  practice  had  she  been  cognizant  of  her 
professional  needs  rather  than  limiting  her  focus  to  her  current  interests  at  the  time.  Despite  the  limitations  Jenny  set  upon  her 
own  graduate  experience,  she  still  felt  the  foundation  she  gained  in  ID  enabled  her  to  enter  the  field.  Osman  and  Hannafin  (1992) 
explained  that  expertise  develops  over  time  as  the  novice  works  through  problems  that  progressively  increase  in  difficulty.  Under 
the  guidance  of  experts,  the  skills  are  internalized  with  each  new  experience.  This  foundation  prepared  Jenny  for  a life  of  learning 
through  continuing  education  and,  more  importantly,  experience  working  as  an  apprentice  to  expert  practitioners. 

Practitioners.  The  Practitioners  viewed  their  graduate  experience  as  a foundation  for  practice,  yet  they  felt  that  the  study  of 
instructional  design  was  more  of  a means  to  an  end,  their  degree,  rather  than  a part  of  their  lifelong  learning  experience. 
Instructional  design,  they  felt,  enabled  the  Practitioners  to  enter  the  field.  It  also  "provided  a framework  for  the  process"  (Anne, 
Chris)  and  "without  this  background  knowledge,  you  run  the  risk  of  designing  something  that  is  not  instructionally  sound" 
(Sharon). 

The  Practitioners’  focused  their  continuing  education  on  remaining  current  with  the  latest  technologies  and  learning  theories 
related  to  their  definition  of  ID  practice  in  the  "real  world:"  instructional  strategies,  interaction  and  interface  design.  They  agreed 
that  ID,  "in  theory,"  influenced  their  decision-making,  but  felt  that  the  systematic  ID  process  they  learned  in  sdiool  had  limited 
application  in  a client-based,  corporate  environment.  The  Practitioners  attributed  this  to  the  expectations  of  the  clients  who 
usually  "internally  identified  the  need  and  solution"  because  "they  know  exactly  what  they  want  presented  and  they  have  the 
content  for  us  to  create  interactions  (Sharon)." 

Novices  characteristically  interpret  problems  at  face  value  and  consider  few  influencing  factors,  one  at  a time.  They  tend  to 
move  immediately  to  the  generation  of  solutions  and  attempt  to  app^  them  (Berliner,  1986,  1991;  Chi,  Feltovich,  and  Glaser, 
1981;  Schon,  1983,  1987;  Chi  etal.,  1988;  Eraut,  1994).  Likewise,  the  Practitioners  took  design  problems  at  face  value  and  did 
not  question  the  client’s  instructional  objectives  or  proposed  solutions.  They  firmly  believed  that  "the  client  drives  so  much  of 
everything  (Chris)’’  and  "actually  having  real  clients  makes  it  less  likely  that  the  instructional  design  process  is  going  to  be 
followed  (Sharon)." 

This  was  the  most  noteworthy  characteristic  of  the  Practitioners,  yet  it  is  the  most  difficult  to  understand.  Although  they  had 
similar  graduate  experiences  to  the  innovators,  (Chris  and  Sharon  [Practitioners],  and  Josh  and  Casey  [Innovators]  attended  the 
same  graduate  program)  the  Practitioners'  description  of  their  approach  to  the  ID  process  was  reflective  of  novice  problem- 
solving characteristics.  They  described  their  process  in  concrete  terms  and  tended  to  see  single  factors  influencing  situations  and 
driving  instructional  solutions  (Berliner,  1986,  1991;  Chi,  Feltovich,  and  Glaser,  1981;  Schon,  1983,  1987;  Chi  et  al.,  1988; 
Eraut,  1994).  This  difference  suggests  that  the  Practitioners  develop  expertise  at  a different  rate  than  their  counterparts,  the 
Innovators.  Factors  affecting  this  discrepancy  may  include  the  climate  of  the  instructional  designer's  work  environment,  projects 
or  individual  personality  traits.  Chris  and  Sharon,  for  instance,  entered  a work  environment  that  focused  on  projects  that 
supplemented  existing  content.  Coupled  with  their  perception  that  instructional  design  in  actual  practice  begins  with  instructional 
strategies,  they  did  not  consider  alternative  needs  or  solutions  to  the  learning  needs  they  were  addressing.  Anne  was  a Practitioner 
who  felt  that  she  was  a novice  instructional  designer  looking  to  her  graduate  experience  to  guide  her  actions  in  her  new  job.  Her 
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previous  job  did  not  require  her  to  employ  the  entire  ID  process  and  she  believed  she  developed  habits  that  compromised  the 
quality  of  ID,  such  as  moving  directly  to  the  design  and  development  of  instructional  solutions.  Innovators  Josh  and  Casey,  on 
the  other  hand,  found  opportunity  to  adapt  each  stage  of  the  ID  process  to  extract  the  information  they  needed.  Josh  demonstrated 
his  skills  to  potential  employers  and  they  hired  him  as  both  instructional  designer  and  content  expert.  As  a result  of  his  dual  role, 
he  found  that  he  had  considerable  influence  on  client  decisions  about  the  ID  process  and  their  selection  of  alternative  solutions. 
Casey  responded  to  the  environment  where  she  worked,  by  leading  experts  from  a diverse  range  of  disciplines  through  the  ID 
process. 

Like  the  advanced  beginners  described  in  the  Dreyfus  Model  of  Skill  Acquisition  (Dreyfus  & Dreyfus,  1986),  the 

Practitioners  followed  the  rules  they  understood.  They  practiced  their  craft  by  organizing  content  and  designing  the 

"presentation”  and  "interactivity"  based  on  the  clients'  guidelines.  They  did  not  see  the  ID  process  as  a way  of  exploring  a 
situation  in  multiple  way  s.  They  valued  its  structure,  yet  considered  it  an  ideal,  a "theory,"  that  could  not  exist  in  a world  where 
the  primary  source  for  the  need  and  identification  of  the  solution  was  the  client  and/or  subject  matter  expert.  For  Anne,  this 
discrepancy  between  academic  preparation  and  practice  diminished  her  sense  of  accomplishment  because  it  made  her  "feel  guilty 
about  the  product  that  you  produce  in  the  commercial  world." 

Skills  learned  on  the  job.  The  most  common  skills  the  participants  learned  on-the-job  were  the  collaborative,  management 
and  interpersonal  skills.  The  participants  also  noted  that  several  of  the  skills  they  learned  on-the-job  expanded  upon  the 
foundations  they  formed  in  graduate  school.  This  included  technology  as  well  as  ID  related  skills  such  as  learning  theory, 
research  and  evaluation.  (It  is  important  to  note  here  that  some  gaps  in  this  analysis  may  exist  because  a respondent  did  not 

comment  on  it  rather  than  indicating  that  this  skill  was  not  learned  on  the  job.)  Laura  expressed  this  sentiment  about  her 

technology  skills,  "I  think  there  is  definitely  a technology  learning  curve  that  never  stops,"  and  most  of  the  participants  anticipate 
that  this  knowledge  base  will  continue  to  grow  throughout  their  careers. 

Participant  Recommendations.  Most  of  the  participants  pointed  to  a need  for  courses  that  address  the  dynamics  of 
professional  practice  and  prepare  them  for  the  challenge  to  continue  developing  these  skills  on-the-job.  Several  of  the 
participants,  for  instance,  noted  a discrepancy  between  their  academic  focus  on  K-12  applications  of  instructional  design,  and 
their  application  of  adult  learning  theory  and  in  actual  practice.  All  but  one  participant  thought  they  would  be  working 
exclusively  with  K-12  learners  in  professional  practice  and,  consequently,  none  of  the  participants  sought  additional  coursework 
in  adult  learning  while  they  were  enrolled  in  graduate  school.  And,  as  one  participant  noted,  employers  often  assume  that 
graduates  of  instructional  design  programs  have  a background  in  adult  learning  theory  and  exposure  to  methods  that  can  meet  the 
needs  of  the  diverse  target  audiences  they  encounter  in  actual  practice. 

Organizations  today  are  focusing  on  their  future  by  creating  work  environments  that  encourage  collaboration,  teamwork,  and 
shared  knowledge.  Skills  in  the  areas  of  human  relations,  social  intelligence,  are  critical  to  successful  professional  practice.  With 
the  exception  of  Jenny,  all  of  the  participants  found  that  team  building  and  collaboration  were  significant  components  of  their 
professional  practice.  (Jenny  anticipated  that  she  too  would  need  these  skills  as  she  moved  into  a project  management  role  in  the 
future).  Consequently,  they  felt  they  were  continually  challenged  to  improve  their  professional  interpersonal  skills.  Laura  was  the 
only  participant,  however,  to  attend  a course  in  human  relations;  her  college  included  this  course  as  a part  of  the  core  curriculum. 
She  noted  that,  although  these  skills  had  not  fully  developed  when  she  began  to  practice  instructional  design  professionally,  the 
course  laid  a foundation  for  her  interpersonal  skills  development.  She  was  able  to  recall  issues  covered  in  class  activities  and 
apply  these  problem-solving  skills  to  similar  situations. 

The  participants  noted  several  areas  of  project  management  that  they  developed  on-the-job  including  client  negotiations, 
delegation,  scheduling,  budgeting,  and  project  projection,  among  others.  Most  of  the  participants  recognized  that  project 
management  skills  are  applied  in  complex  multidimensional  situations  and,  therefore,  must  develop  in  professional  practice.  They 
emphasized,  however,  that  project  management  experiences  in  graduate  school  might  provide  them  with  the  tools  and  methods 
that  would  ease  their  ease  their  transition  to  professional  practice. 

Recommendations:  instructional  methods  for  academic  programs.  The  instructional  designers  in  this  study  entered  their 
graduate  programs  without  an  understanding  of  instructional  design  as  a method  or  a profession.  The  vision  of  professional 
practice  they  held  often  influenced  their  perceptions  of  what  coursework  was  relevant  and  guided  their  focus  throughout  their 
graduate  programs.  Consequently,  they  felt  they  were  inadequately  prepared  in  these  areas  of  professional  practice.  Exposure  to 
ID  professionals  when  students  enter  their  graduate  programs  could  help  novice  designers  build  an  accurate  perception  of 
professional  practice  early  in  their  academic  experience.  Subsequent  visits  can  focus  on  a variety  of  issues  or  environments  of 
practice;  as  the  students'  conceptual  and  applied  understanding  of  the  ID  process  and  related  disciplines  increase,  so  will  their 
depth  of  inquiry  about  professional  practice. 

Most  of  the  participants  in  this  study  assumed  that,  as  an  instructional  designer  on  production  teams,  they  would  play  a 
significant  role  developing  components  for  products.  A panel  discussion  with  a production  team,  including  designers,  project 
managers,  and  developers  could  help  students  understand  the  interrelationship  between  designers  and  developers  in  actual 
practice  and  the  skills  an  instructional  designer  applies  to  this  stage  of  the  design  process. 

The  results  of  this  study  suggest  that  multiple  factors,  including  learning  readiness,  prior  experience  and  learning  styles,  can 
affect  the  novice  designer's  reception  and  cognition  of  these  disciplines  as  foundations  for  professional  practice.  While  some 
participants  were  able  to  draw  connections  between  the  core  topics  they  covered  in  their  graduate  studies  and  ID  practice,  some 
interpretations  misled  them.  For  instance,  the  perception  that  the  ID  process  is  separate  from  the  design  and  development  of 
technology -based  instruction  or  the  conception  that  constructivist  learning  environments  require  opposing  design  methodologies 
to  the  ID  process.  Most  of  the  participants  did  not  draw  critical  connections  until  they  began  professional  practice.  Some 
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participants  never  drew  these  connections.  This  suggests  that  novice  instructional  designers  would  benefit  from  an  increased 
focus  on  the  relationship  between  the  concepts  and  skills  they  are  learning  and  actual  practice. 

Connections  between  specific  content  areas  need  to  reflect  how  each  area  influences  actual  practice.  Professional 
instructional  designers  do  no  apply  their  ID,  learning  theory,  technology,  or  project  management  skills  in  isolation;  these  skills 
and  knowledge  are  interdependent  and  interrelated,  IT  programs  and  instructors  can  help  the  beginning  designer  understand  this 
interrelationship  by  modeling  application  of  the  various  disciplines  in  each  of  the  core  IT  classes.  This  strategy  should  begin 
when  students  enter  their  graduate  program.  A survey  course  on  the  history  of  the  field,  such  as  the  course  Josh  described  for 
instance,  can  provide  a global  understanding  of  the  role  each  foundation  holds  in  professional  practice.  The  interrelationship 
between  the  theoretical  and  practical  knowledge  of  the  field  can  then  increase  in  complexity  as  the  students  move  into  advanced 
courses,  and  from  exploration  towards  actual  application  of  these  skills  in  the  design  and  development  of  instructional  materials. 

The  adoption  of  such  an  integration  strategy  might,  in  many  IT  programs,  place  a significant  demand  on  IT  instructors  by 
necessitating  increased  collaboration  in  the  design  and  implementation  of  the  program  area  courses.  One  possible  alternative  is  a 
method  that  I explored  in  the  University  of  Virginia  ID  course  that  I have  co-facilitated;  we  added  a lab  to  the  core  ID  classes  and 
used  this  time  to  facilitate  student  explorations  of  the  practical  relationship  between  their  ID  and  the  other  disciplines.  Again, 
student  response  to  this  opportunity  was  enthusiastic,  although  it  required  considerable  guidance  to  keep  the  students  on  track  and 
engaged  in  each  others'  explorations.  This  method  could  be  applied  to  all  of  the  core  IT  classes  and,  although  there  were  be 
overlap  in  some  instances,  the  students  would  begin  to  understand  how  these  concepts  apply  to  professional  practice. 

Several  of  the  participants  in  this  study  also  indicated  that  the  skills  and  knowledge  they  developed  in  graduate  school  were 
only  the  building  block  for  a foundation  that  they  expect  to  continue  expanding  throughout  their  career.  Student  resource 
portfolios,  another  method  that  we  applied  in  the  University  of  Virginia  ID  courses,  can  help  novice  instructional  designers 
organize  their  academic  course  materials  so  they  can  serve  as  reference  materials  for  this  ongoing  exploration.  In  addition  to  their 
use  as  a resource  for  professional  practice,  the  process  of  creating  the  portfolio  requires  the  novice  designer  to  analyze  the 
materials  and  consider  how  they  might  apply  to  actual  practice.  This  can  increase  the  relevance  these  topics  hold  for  the  novice 
instructional  designer.  It  can  also  serve  to  demonstrate  that  the  foundation  acquired  in  graduate  school  provides  a framework  for 
lifelong  learning. 

Prior  research  into  the  development  of  practical  knowledge  and  professional  and  ID  skill  development  suggest  that 
experiences  reflecting  actual  practice  are  critical  the  development  of  a professional  designer.  Likewise,  all  of  the  participants 
recommended  experiences  reflective  of  professional  practice  and  most  of  them  recommended  real-world  ID  projects  with  actual 
clients.  The  participants  who  had  this  opportunity  emphasized  their  appreciation  for  their  instructor-coaches  who  led  them  from 
simple  to  complex  applications  of  the  ID  process  and  provided  guidance  and  feedback  that  encouraged  them  to  consider  multiple 
perspectives  and  potential  solutions. 

In  addition  to  the  real-world  projects,  many  of  the  participants'  programs  offered  credit  for  internships  in  work  environments 
thought  to  reflect  the  type  of  work  environment  the  student  planned  to  enter  in  professional  practice.  Internships  can  provide 
novice  instructional  designers  with  experience  applying  the  technical  and  practical  knowledge  expected  of  professional 
instructional  designers.  On  the  other  hand  (and  as  evidenced  by  one  of  the  participants  in  this  study),  internships  may  not  always 
provide  the  student  with  an  experience  reflective  of  professional  practice  in  instructional  design.  This  investigator  contends  that 
the  instructor  should  ensure  the  student  is  placed  in  the  appropriate  work  environment.  Under  the  guidance  of  a university 
professor  who  can  encourage  reflection  on  practice  and  analysis  of  the  experience,  the  intern  can  successfully  explore  the 
dynamics  of  professional  practice. 

The  participants  in  this  study  also  described  their  struggles  with  a variety  of  challenges  including  project  management,  client 
negotiations,  team  dynamics,  and  ID  projects  in  environments  that  were  foreign  to  them.  Other  fields  such  as  business,  law,  and 
medicine  and,  more  recently,  the  field  of  instructional  design,  have  demonstrated  the  effectiveness  of  case-based  learning.  Case 
studies  present  realistic  narratives  of  real  world  situations  and  problems.  Case  study  analyses  facilitated  by  instructor-coaches  can 
expose  teams  of  novice  instructional  designers  to  multiple  environments.  Teams  can  explore  messy  situations  in  a safe 
environment  and  gain  experience  solving  the  types  of  problems  they  may  encounter  in  actual  practice. 

As  the  participant,  "Laura,"  explained,  she  drew  from  case  study  experiences  that  addressed  issues  similar  to  the  challenges  she 
faced  in  actual  practice. 

Future  Research 

This  Study  of  instructional  designers'  perceptions  of  academic  preparation  and  practice  has  been  reported  with  an  emphasis  on 
the  transfer  of  skills  and  knowledge  developed  in  graduate  school  to  actual  practice  and  those  skills  practitioners  developed  on- 
the-job.  It  was  found  that  ID  professionals,  who  are  assured  of  confidentiality,  might  be  open,  honest  and  willing  to  reflect  on 
their  successes  and  failures  as  instructional  designers.  Despite  significant  differences  between  each  participant's  experiences, 
several  common  themes  emerged  suggesting  that  IT  programs  are  addressing  many  of  the  skills  that  novice  instructional 
designers  employ  in  professional  practice.  This  is  in  line  with  prior  research  into  the  competencies  of  professional  instructional 
designers  (ASTD,  1983;  IBSTPI,  1986;  Atchison,  1996). 

This  study  provided  a glimpse  into  the  professional  lives  and  academic  preparation  of  instructional  designers  who  develop 
instructional  materials  for  external  clients  including  products  for  educational  publishers,  government  and  related  industries,  and 
entertainment  clients,  among  others.  The  dynamics  of  ID  practice  vary  significantly  across  practice  environments  and  each 
situation  presents  unique  demands  for  the  ID  practitioner.  Further  qualitative  studies  addressing  a variety  of  practice 
environments  will  provide  data  on  the  relationship  between  academic  preparation  and  these  specific  work  environments.  The 
investigation  of  additional  ID  work  environments  can  help  to  identify  the  variations  in  professional  ID  practice.  Subsequent 
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comparisons  across  practice  environments  will  enable  us  to  increase  the  generalization  of  our  findings  to  multiple  environments 
in  the  field.  It  would  also  be  interesting  to  develop  detailed  case  studies  about  the  production  and  development  of  an  entire 
project  in  each  of  these  environments. 

This  study  presented  a variety  of  academic  environments  and  stressed  the  need  to  establish  the  relevance  of  the  theories, 
concepts,  and  procedures  explored  in  graduate  IT  programs  by  demonstrating  their  interrelationship  and  interdependence  in  ID 
practice.  Further  studies  into  the  structure  of  academic  programs  and  the  interrelationship  of  the  curricula  is  warranted.  This  study 
identified  some  of  the  teaching  methods  that  are  perceived  to  be  effective  in  learning  ID  and  pointed  to  some  teaching  methods 
that  are  thought  to  be  ineffective  by  some  designers.  The  result  of  this  study  also  suggests  further  research  in  this  area.  A profile 
of  the  professional  instructional  designer  is  ripe  for  investigation.  An  exploration  in  the  relationship  between  ID  students  and  the 
three  profiles.  Innovator,  Traditionalist,  and  Practitioner  can  help  provide  insight  into  the  characteristic  of  successful  transfer 
from  preparation  to  practice. 

This  study  paralleled  prior  research  that  identified  proficient  designers  as  flexible  and  adaptable,  able  to  solve  complex  ID 
problems  in  a diverse  range  of  environments,  and  create  innovative  solutions.  It  was  found  that  instructional  designers  may  begin 
to  develop  these  skills  through  instructor  coaching,  guided  reflection,  and  experiences  that  emulate  actual  practice  such  as  real- 
world  projects  and  case  studies.  For  some  instructional  designers,  these  methods,  along  with  the  foundations,  skills  and 
knowledge  that  novice  designers  develop  in  graduate  school  and  in  their  early  years  of  practice,  enable  them  to  apply  what  they 
learn  to  professional  practice  and  lifelong  learning. 
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Abstract 

Younger  students  live  in  a media-centric  world.  Researchers  have  shown  that  youth  today  spend  more  time  watching 
television  and  movies  than  most  any  other  leisure-time  activity  (Pearl,  1982).  In  addition,  the  presentation  speed  of  passages  on 
commercial  television  has  increased  significantly  in  the  past  fifty  years  (Stephens,  1996).  Researchers  have  shown  that  viewers 
automatically  learn  to  cope  with  symbolic  presentation  methods  through  rqjeated  exposure  to  television  and  visual  patterns 
(Abelman,  1995;  Bargh,  1988;  Carr,  1982).  Because  of  their  increased  exposure  to  rapid  sequence  and  presentation  speed 
brought  on  by  fast- cuts/montage  found  in  television  programs  aimed  at  youth,  it  may  also  be  assumed  that  these  individuals  can 
comprehend  these  messages  on  a much  wider  scale  than  can  their  adult  counterparts.  One  cannot  assume  that  exposure  to  rapid 
presentation  speed  is  simply  a passive  viewing  activity.  Further,  not  only  are  these  advancements  in  media  technology  changing 
the  way  viewers  look  at  and  interpret  video  media,  but  most  importantly,  the  widespread  availability  of  production  techniques 
provide  easy  access  to  capabilities  that  allow  people  to  use  video  media  to  easily  create  their  own  content.  It  has  been  widely 
shown  fashion  (Tyner,  1998)  that  these  acquisition/production  opportunities  also  increase  exponentially  one's  ability  to 
comprehend  content  delivered  in  like. 

Media  educators  and  theorists  for  years  have  been  analyzing  Marshall  McLuhan 's  famous  quip,  the  medium  is  the  message. 
In  some  regard,  McLuhan 's  statement  may  be  a retort  to  later  critics  of  educational  media  like  Richard  Clark  (1983)  who 
claimed  that  media  are  "mere  vehicles  that  deliver  instruction  but  do  not  influence  achievement  any  more  that  the  truck  that 
delivers  groceries  causes  changes  in  nutrition"  (p.445).  Further,  communications  theorists  like  Walter  Ong  (1982)  not  only 
agreed  with  McLuhan,  but  extended  the  meaning  of  McLuhan’s  message  to  also  imply  that  the  types  of  media  people  use  define 
the  way  they  think.  Ong’s  notions  bring  to  mind  possible  questions  as  to  whether  today’s  mediacentric  youth  perhaps  think 
differently  than  previous  generations,  with  implications  as  to  the  kinds  of  instructional  strategies  that  will  be  successful  in 
motivating  them  to  learn  and  providing  perceptual  stimuli  for  recognition  and  recall. 

Background 

Media  educators  and  theorists  for  years  have  been  analyzing  Marshall  McLuhan’s  famous  quip,  the  medium  is  the  message 
(Meyrowitz,  1985).  In  some  regard,  McLuhan’s  statement  may  be  a retort  to  later  critics  of  educational  media  like  Richard  Clark 
(1983)  who  claimed  that  media  are  “mere  vehicles  that  deliver  instruction  but  do  not  influence  achievement  any  more  that  the 
truck  that  delivers  groceries  causes  changes  in  nutrition”  (p.445).  Further,  communications  theorists  like  Walter  Ong  (1982)  not 
only  agreed  with  McLuhan,  but  extended  the  meaning  of  his  message  to  also  imply  that  the  types  of  media  people  use  define  the 
way  they  think.  Ong’s  notions  bring  to  mind  possible  questions  as  to  whether  today’s  mediacentric  youth  perhaps  think  and 
cognitively  perceive  differently  than  previous  generations,  with  implications  as  to  the  kinds  of  instructional  strategies  that  will  be 
successful  in  motivating  them  to  learn  and  providing  perceptual  stimuli  for  recognition  and  recall. 

The  increased  predominance  of  digital  technology  in  our  daily  lives  is  no  accident.  In  1996,  the  Federal  Government 
mandated,  through  the  Telecommunications  Act  of  1996,  major  changes  in  the  way  television  signals  are  to  be  transmitted. 
Digital  and  high  definition  (HDTV)  television  would  become  the  standard  by  2006.  In  exchange  for  the  broadcast  industry 
having  to  absorb  the  multi-millions  of  dollars  in  cost  for  this  conversion,  the  FCC  also  made  significant  favorable  changes  in 
limits  on  cross-ownership,  which  have  encouraged  an  overall  industry-wide  consolidation.  The  end  result  has  been  a phenomenon 
called  digital  convergence:  the  combining  of  the  telephone,  the  computer,  and  television  into  one  technological  box.  Alan 
November  (1998)  re-labeled  this  new  technological  machinery,  calling  it  a "Digital  Combine",  in  obvious  reference  to  the 
agricultural  combine  that  was  invented  during  the  1930s.  Traditional  fanners  who  worked  the  land  by  hand  fought  this  advancing 
technology  as  depicted  so  well  in  the  movie  The  Grapes  of  Wrath.  In  a famous  scene  the  character,  played  by  Henry  Fonda,  was 
complaining  to  the  other  squatters  that  these  new  Cats  were  pushing  them  off  their  land.  November’s  analogy  is  clear.  Educators 
who  cling  to  (i.e.  squat  on)  their  old  ways  will  suffer  a similar  fate  and  will  be  pushed  off  their  turf  if  they  do  not  learn  how  to  co- 
exist with  the  latest  technological  cat  - new  media  technologies  that  are  so  prevalently  being  used  and  mastered  by  today’s  youth. 
A change  in  educational  techniques  is  inevitable.  What  must  be  done  is  to  figure  out  how  and  when,  not  if,  this  new  combine  will 
be  incorporated  into  instructional  designers’  thinking  about  the  current  batch  of  students,  who  are  steeped  in  exposure  to  new 
media. 

As  a result  of  digital  convergence  and  other  new  ideas,  the  predominant  types  of  production  techniques  in  use  today  in 
electronic  media  are  changing.  Encouraged  by  the  successes  of  early  pioneers  of  rapidly  paced  music  video  montages  on 
networks  like  MTV,  VHl,  and  Nickelodeon,  and  helped  by  rapid  advances  in  technology,  today’s  television  producers  regularly 
communicate  fairly  complex  messages  using  fast-cuts  and  video  montage  (Stephens,  1996).  Significant  increases  in  viewership 
of  these  types  of  programs  appear  to  be  providing  fertile  opportunities  for  today’s  youth  to  practice  their  ability  to  receive  and 
understand  this  fast-paced  message  delivery.  According  to  Seward-Barry  (1997),  sleeping  has  become  the  only  activity  that 
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occupies  children’s  time  more  than  watching  television  (or,  more  recently,  playing  video  games).  For  these  reasons,  a change  in 
the  way  in  which  educators  look  at  learning  from  electronic  media  may  be  warranted.  This  thinking  appears  to  be  incompatible 
with  earlier  communications  theorists  like  Edgar  Dale  (1969),  who  felt  that  television  finished  somewhere  in  the  middle  of  a 
twelve-point  influence  scale  in  its  ability  to  convey  contextual  ideas.  Moreover,  using  television  in  education  appears  to  be 
anathema  to  recent  spokesmen  like  Neil  Postman  (1986),  who  feel  that  it  adds  nothing  positive  to  the  mix  and  may  even  be 
detrimental  to  learning  and  cognition.  However,  new  media  that  employs  visual  imagery  supplemented  by  motion,  sound,  and 
computer  editing  is  perhaps  finally  positioned  to  evolve  into  a similar  definitional  prominence  to  the  peop  le  who  use  them,  just  as 
print  media  had  done  for  the  past  five  hundred  years  since  Guttenberg  (Stephens,  1996,  p.  69).  Corcoran  (1981)  suggested  that 
intelligence  is  a skill  in  a particular  medium  and  that  symbolic  codes  used  in  that  medium  that  serves  communication  purposes 
and  are  internalized  by  a receiver  also  serves  as  a authentic  tool  of  thought.  This  new  way  of  thinking  may  be  at  the  root  of 
generational  differences  in  communication  techniques  and  over-dependence  on  the  right  brain  for  thinking  in  today's  youth.  It  has 
also  brought  to  mind  a revelation  on  the  part  of  this  author.  Just  because  a student  does  not  know  the  words  to  communicate  his 
or  her  thoughts  does  not  necessarily  mean  that  he  or  she  is  not  having  any  intelligent  thoughts.  Perhaps,  the  ideas  are  coming  to 
these  individuals  in  different  ways.  Investigation  and  follow-up  has  made  some  things  much  more  clear.  Educators  need  to  look 
differently  on  communicating  and  educating  today's  mediacentric  youth. 

The  Medium  is  the  Message  Re-visited 

The  idea  that  the  learning  process  might  be  changing  as  the  result  of  the  types  of  predominant  media  being  used  is  appears  to 
be  backed  up  by  several  research  studies  by  Jonassen  (1996)  and  others  in  which  technology/media  has  been  successfully 
evaluated  as  type  of  cognitive  mind  tool.  In  other  words,  by  merging  many  formerly  distinct  knowledge  situations,  new  media 
appear  to  be  “breaking  down  the  boundaries  among  various  disciplines,  opening  new  dialogues,  and  fostering  the  development  of 
cross-disciplinary  areas  of  study”  (Meyrowitz,  1985,  p.  327).  These  new  kinds  of  electronic  media,  Meyrowitz(1985)  speculated, 
may  be  "introducing  our  children  to  a different  way  of  thinking  that  involves  the  integration  of  multiple  variables  and  overlapping 
lines  of  simultaneous  actions"  (p.  326).  This  assertion  proposes,  among  other  things,  that  electronic  media  in  general  (and 
television  in  particular)  may  have  already  greatly  reduced  the  influence  that  time  and  location  used  to  have  on  what  people  know. 
It  is,  therefore,  not  unreasonable  to  propose  that  a transition  may  be  in  process  in  which  the  youth  of  today  think  about  things,  all 
of  which  appears  to  be  at  odds  with  the  linear  thinking  processes  associated  with  print  media  (Stephens,  1996).  Further,  it  is  also 
possible  that  today’s  youth  may  be  moving  away  from  a “one-thing-at-a-time,  one-thing-after-another,  and  take-time-to-think 
world  of  reading"  (Meyrowitz,  1985,  p.  326)  towards  McLuhan’s  world  of  “interconnected  layers  of  information”  in  which  “a 
continual  superimposition  of  complex  contextual  matrices,  all  arrive  (sic)  into  the  brain  at  an  electric  speed”  (McLuhan,  1964,  p. 
91). 

The  Medium  is  the  Message,  Part  1 

Perhaps  the  best  way  to  analyze  the  impact  of  new  media  on  teacher  youth  in  the  digital  age  is  to  re-look  at  McLuhan's 
medium  is  the  message  ideas  with  a view  to  some  modem  interpretations.  One  interpretation  of  the  medium  is  the  message 
concept  involves  how  we  should  evaluate  the  innate  goodness  or  badness  of  the  media  we  use.  In  general,  this  assessment  can  be 
interpreted  in  four  ways.  Technology  (As  an  aside,  the  terms  technology  and  media/new  media  should  be  inter-changeable  here) 
needs  to  be  assessed  in  the  following  ways: 

• What,  if  anything,  new  does  it  bring  to  the  table  (i.e.  what  good  is  it  doing  for  the  culture  who  is  using  it?) 

• What  does  its  introduction  into  a culture  make  obsolete? 

• What,  if  anything,  does  it  bring  back  that  might  have  been  obsoleted  by  a previous  new  technology? 

• What  happens  when  people  using  this  new  technology  over-depend  on  it? 

In  most  general  terms,  this  interpretation  of  the  usefulness  of  technology  has  very  specific  implications  for  educators  who  are 
looking  into  teaching  and  communicating  with  today's  media-centric  youth.  First,  new  media  provide  complete  interactivity  at  a 
very  small  price.  Without  having  to  leave  their  classrooms,  students  can  interactively  explore  cloud  formations  that  occur  during 
major  hurricanes,  compare  bone  structures  between  humans  and  apes,  look  at  generational  genetic  similarities,  and  explore  the 
human  body  in  very  dramatic  form.  Streaming  video  and  the  Internet  are  helping  to  redefine  the  distribution  of  educational 
materials.  For  example,  the  National  Library  of  Medicine  at  the  University  of  Maryland  has  put  online  the  Visible  Human 
Project,  in  which  students  can  view  an  actual  human's  anatomy  from  the  top  of  his  head  all  the  way  throughout  to  his  feet,  using 
cat  scan  technology  that  has  been  exported  to  a Quick  Time  Video.  These  virtual  field  trips  are  becoming  increasingly  prevalent 
and  commonplace  on  the  Internet.  Second,  new  media  technologies  (the  Internet,  in  particular)  appear  to  empower  and  encourage 
students  to  communicate  using  the  media.  Producers  of  new  media  are  offered  broad  new  opportunities  to  immediately  practice 
their  skills  through  ready-made  distribution  channels  for  a new  population  of  motion  picture  and  video  practitioners.  Marc  Davis 
from  the  MIT  Media  Labs  likened  his  phenomenon  to  that  of  younger  musicians  practicing  in  their  garage.  "In  the  spirit  of  garage 
bands,  the  Internet  and  new  video  technologies  provide  ready-made  distribution  channel  for  a new  population  of  motion  picture 
(and  video  (sic))  producers,  as  practitioners  of  garage  cinema"  (Davis,  1996).  A more  famous  example  was  the  way  that  the  Blair 
Witch  project  got  off  the  ground,  using  the  Internet  first  as  a means  to  introduce  aid  market  the  product  prior  to  its  general 
release  in  major  movie  houses  across  the  country.  Several  Internet  sites  like  Always  I. com  are  popping  up  that  encourage  amateur 
producers  to  post  their  work.  No  longer  do  today's  youth  have  to  wait  to  grow  up  and  move  to  Hollywood  in  order  to  get  their 
works  published.  These  new  opportunities  are  encouraging  youth  all  over  the  country  to  create  and  communicate  using  media 
technologies.  Classroom  teachers  need  to  tap  into  these  opportunities  and  role  models  to  make  connections  with  their  students, 
who  are  already  spending  considerable  amounts  of  their  leisure  time  in  these  activities. 
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If  one  follows  John  Keller's  ARCS  motivational  model  (Keller,  1983),  it  shouldn't  be  too  hard  to  see  how  relevance  and 
success  with  new  media  may  be  translated  into  making  successful  connections  with  today's  mediacentric  youth.  A word  of 
caution  - new  media  cannot  be  looked  at  without  also  keeping  in  mind  the  fourth  law  of  media  evaluation  mentioned  previously. 
Over-depending  on  new  technologies  does  have  its  downside.  New  media  tends  to  affect  right-brain  development,  whereas  text- 
based  cognition  is  left-brain.  An  over-dependence  on  one  side  or  the  other  is  not  fully  developing  one's  potential  and  can  leave  a 
child  ill  equipped  to  fully  function  in  the  world.  As  Robert  Doman,  Jr.  (1984)  very  often  preached,  one  should  teach  to  one's 
strengths  and  remediate  any  weaknesses,  and  not  the  other  way  around.  A failure  to  do  this  is  doing  our  youth  a disservice. 

Teach  to  the  Strengths,  Remediate  the  Weaknesses 

The  obvious  point  here  is  that  educators  should  teach  to  a student's  strengths  (i.e.  right-brain  development)  I order  to 
remediate  their  possible  weaknesses  (i.e.  text -based  or  left-brain  cognition).  Use  visual  skills  to  get  at  textual.  Using  media  as  a 
cognitive  development  tool  has  incurred  significant  success  in  several  literacy  projects  recently.  These  projects  utilized  students' 
fascination  with  the  technology  of  television  and  video  production  as  a hook  to  encourage  them  to  develop  their  non-verbal  (i.e. 
oral  and  visual)  story-telling  skills  that  eventually  translated  into  their  acquiring  increased  text -based  communication  abilities. 
These  students  haven't  needed  too  much  prompting  to  want  to  write  about  their  own  personal  visual  experiences  and/or  story 
lines.  Marco  Torres'  work  with  inner-city  youth  in  east  Los  Angeles,  as  well  other  similar  projects  in  San  Antonio,  Texas,  and  in 
Ohio  have  been  very  successful  in  bringing  otherwise  lost  children  back  into  educational  the  fold  so-to-speak.  By  first  teaching 
students  how  to  communicate  non-verbally  and  then  having  them  utilize  their  own  projects  to  develop  their  verbal 
communication  skills  has  paid  big  dividends. 

Remediating  weaknesses  and  learning  disabilities  appears  to  be  another  way  that  new  media  has  helped  shape  personality  and 
development.  For  example,  researchers  have  found  that  video  games  can  interact  with  subjects  to  positively  influence  cognition. 
Physicians  and  researchers  at  the  Medical  Center  at  the  University  of  Eastern  Virginia  have  been  able  to  attain  significant  results 
in  treating  ADHD  students  with  a non-invasive,  non-medical  treatment  using  video  games  to  increase  attention  spans  and  to 
increase  cognitive  activity.  Subjects  are  hooked  up  to  bio-feedback  apparatus  that  monitors  attention  while  they  play  popular 
video  games.  As  subjects'  attentions  drift,  the  feedback  mechanism  makes  the  joystick  attached  to  the  game  more  difficult  to 
operate.  The  subjects  are  prompted  to  again  attaid  to  what  they  are  doing.  They  are  incented  to  stay  on  task  because  of  a built-in 
desire  to  play  the  game.  Positive  long-term  effects  appear  to  warrant  the  use  of  this  program  to  cause  subjects  to  change  their 
cognitive  behavior  (i.e.  attending  to  a task)  relative  to  capabilities  and  habits  prior  to  entering  the  program.  Most  can  be  taken  off 
therapeutic  medications  in  the  process. 

Research  appears  to  back  up  these  projects.  While  some  have  deprecated  the  over-use  of  media  (Dale,  1969;  Neuman,  1976; 
Postman,  1986)),  others  (Cropper,  1966;  Nugent,  1982;  Paivio,  1986)  have  found  that  the  use  of  video  media  can  actually 
increase  one's  retention  for  stories  presented  via  the  combination  of  visual  and  auditory  information  that  those  presented  through 
only  a single  source.  It  is,  perhaps,  that  new  media  is  finally  positioned  to  take  its  place  in  education  because  the  availability  of 
easy-to-use  production  techniques  are  finally  beginning  to  rival  those  in  other  communication  vehicles  like  word  processing. 
What  appeared  to  have  taken  over  five  hundred  years  (from  the  invention  of  the  printing  press  to  word  processing)  has  only  taken 
video  media  a little  over  fifty  to  accomplish  (Stephens,  1996). 

The  Medium  is  the  Message,  part  2 

A second  interpretation  of  the  medium  is  the  message  concept  is  that  the  medium  one  uses  to  communicate  not  only  helps  to 
define  the  message,  but  also  those  who  utilize  that  medium.  The  impact  of  new  media  in  this  manner  can  be  demonstrated  in 
several  ways.  For  example,  some  researchers  have  looked  at  this  interpretation  and  came  up  with  the  idea  that  increased  usage 
and  dependence  on  this  form  of  media  might  be  a two-way  street.  Two  Stanford  University  researchers  have  looked  into  media 
interaction  and  examined  the  way  people  respond  to  media  and  media  events.  While  we  can  readily  apply  knowledge  gained  from 
media  experiences  to  real  life,  Reeves  and  Nass  (1996)  found  that  we  also  apply  experiential  knowledge  gained  from  real-life  to 
the  use  of  media.  Among  other  things,  they  examined  cultural  experiences  like  politeness,  flattery,  and  negativity  and  found  that 
people  essentially  react  to  media  in  identically  the  same  ways  they  do  to  other  people.  First,  their  studies  indicate  that  when  a 
computer  asks  a user  about  itself,  users  will  respond  more  positively  than  when  a computer  asks  about  how  well  another 
computer  is  doing.  In  other  instances,  people  believe  they  did  better  on  a computer-based  test  when  the  computer  flattCTS  them 
than  when  the  computer  offers  no  evacuation.  Moreover,  people  seem  to  like  a computer  more,  and  believe  it  did  a better  job 
when  it  flatters  them,  whether  or  not  the  praise  is  warranted  had  no  effect.  Third,  people  were  found  to  pay  attention  to,  and 
remember  negative  media  better  than  positive  media.  Additionally,  people  were  found  to  have  a better  memory  for  information 
that  follows  negative  media  than  for  information  that  follows  positive  media  and  vice  versa.  Lastly,  Nass  and  Reeves  showed 
several  instances  when  media  personalities  are  more  often  identified  solely  through  the  media  roles  they  play,  to  the  extent  where 
they  cannot  cause  people  to  identify  them  by  their  real  names.  Perhaps  what  the  media  equation  is  telling  us  is  that  we  need  to 
broaden  the  context  of  what  it  means  to  be  literate  to  new  forms  of  media  and  to  relate  to  media  just  as  if  it  is  a real  agent  in  our 
daily  lives. 

The  Role  of  Media  in  Education 

In  spite  of  these  findings,  researchers  and  educators  looking  into  the  intrinsic  instructional  value  of  video  media  have 
presented  conflicting  views  with  regards  to  the  role  visual  perception  plays  in  attention,  motivation,  and  recall.  There  have  been 
several  studies  that  looked  into  the  potential  benefits  mediated  coding  systems  have  on  cognition  (Davis,  1999;  Nugent,  1982; 
Paivio,  1986;  Seidman,  1981;  Walma  van  der  Molen,  2000).  On  the  other  hand,  early  theorists  have  had  little  good  to  say  about 
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television’s  ability  to  bring  anything  new  to  the  table  with  regards  to  using  it  as  an  educational  medium  (Berio,  1960;  Calvert, 
1989;  Ide,  1974;  Kozma,  1986).  This  may  have  been  due  to  the  limitations  imposed  by  the  technology  in  use  at  the  time. 
Technological  advances  in  commercial  television  production  techniques  that  allow  today’s  producers  to  readily  integrate  fast-cuts 
and  montage  have  added  to  this  benefit/conflict  controversy  by  providing  them  tools  to  more  easily  communicate  complex 
thought  using  a non-verbal  narrative  structure.  A rapidly  cut  montage  passage  has  been  found  to  add  clarity  to  a passage  because 
the  interpretive  whole  of  the  segment  is  more  than  the  sum  of  its  parts  (Hitchon,  1994;  Stephens,  1996).  In  other  words,  it  is  the 
composite  whole  of  all  the  visual  images  in  a passage  or  segment,  considered  at  all  once,  that  gives  extended  meaning  to  the 
montage.  In  addition,  newer  ideas  on  editing  techniques  have  evolved  that  emphasize  more  the  perceptual  continuity  of  a non- 
verbal composite  narrative  structure  rather  than  the  classical  point  of  view,  which  stressed  the  importance  of  applying  strict 
editing  rules  in  order  to  obtain  smooth  transitions  between  successive  shots  (d'Ydewalle,  1990). 

Fast-seeing  as  Cognitive  Activity 

Studies  need  to  be  designed  that  will  that  will,  hopefully,  extend  into  an  educational  setting  recent  studies  of  casual  viewers’ 
ability  to  absorb  and  comprehend  complex,  rapidly ^Dresented  visual  passages.  What  can  educational  video  producers  learn  about 
presentation  speed  and  editing  from  current  commercial  television  production  trends  that  incorporate  an  ever-increasing  number 
of  these  fast-cuts  and  montage  passages?  Is  it  possible  to  recognize,  recall,  and  get  the  gist  of  intellectual  content  solely  from 
rapidly  paced  visual  montage  that  is  not  supplemented  with  some  form  of  verbal  narrative?  How  does  one’s  learning/cognitive 
style  effect  his  or  her  ability  to  process  fast-cuts/montage  video  presentations  in  a classroom  setting?  Most  of  the  previous  studies 
into  the  impact  of  fast -cut  s/montage  have  concentrated  on  commercial  television  viewing  (Bryant,  1991;  Lang,  1998;  Lang, 
1999;  Lang,  2000;  Zillman,  1991).  While  there  have  been  occasional  studies  (Keller,  1976)  into  the  use  of  fast  cuts/montage  in  an 
instructional  setting,  they  have  looked  at  viewers  as  a collective  whole  with  little  regard  for  differences  in  audience  personalities, 
capabilities,  cognitive  style,  or  personal  traits.  Further,  the  subjects  of  all  earlier  studies  in  either  setting  have  been  college-level 
students  enrolled  in  communications  classes  (Keller,  1976;  Lang,  1999;  Lang,  2000;  Reeves,  1996),  rather  than  school-aged 
students  viewed  in  their  original  educational  environments. 

Construct-related  Validity  of  Leisure-time  Studies 

Media  researchers  have  looking  into  the  benefits  of  rapid  presentation  have  had  to  contend  with  a conflicting  view  that  holds 
that  the  relatively  fast  presentation  speed  of  televised  programs  creates  an  environment  that  may  be  detrimental  to  attention  and 
recall  (Alwitt,  1980;  McCollum,  1999;  Neuman,  1976).  To  the  contrary,  others  have  shown  that  presentation  speed  and 
rhythmicity  in  leisure-time  media  can  actually  heighten  enjoyment,  enhance  motivation,  and  can  “play  an  important  part  in 
determining  the  affective  or  emotional  response  of  message  receivers”  (Seidman,  1981,  p.  49).  Intraub’s  (1999)  recent  studies 
into  conceptual  masking  have  shown  that  humans  are  able  to  recognize  and  recall  pictorial  presentations  when  a minimal  amount 
of  lag  time  separates  individual  images.  There  are  those  who  believe  that  there  have  been  several  successes  with  childrens’ 
programs  that  offer  rapid  and  rhythmical  presentation  speed  (Anderson,  1979;  Anderson,  1983;  Anderson,  1988;  Pearl,  1982). 
Moreover,  it  appears  that  there  exists  a direct  link  between  one’s  ability  to  comprehend  televised  messages  and,  in  fact,  overall 
academic  achievement  and  one’s  innate  cognitive  (i.e.  personal)  tempo  (Flowers,  1995;  Shaffer,  2000,  August  19;  Snow,  1965; 
Wagely,  1978).  Others  contend  that  presentation  speed  in  instruction  may  add  interest  to  otherwise  uninspiring  content  (Canelos, 
1986,  January;  Edgar,  1997;  Hawkins,  1997;  Hill,  1993). 

Still  others  have  looked  into  presentation  speed  as  its  own  construct,  comparing/combining  it  to/with  arousing  content  (Lang, 
1999;  Lang,  2000).  Further,  Lang  (1999;  2000)  discussed  the  effect  of  adding  interesting  and  arousing  content  as  having  a 
positive  effect  on  cortical  arousal  and,  therefore,  recall  and  recognition.  In  additional  studies,  Lang  (2000)  alluded  to  future 
research  that  should  continue  to  probe  the  shape  of  the  relationship  between  presentation  rate  and  recall  and  test  even  faster  rates 
of  edits  to  determine  whether  there  is  a point  at  which  memory  begins  to  decline.  Her  studies  suggest  that  producers  who  want 
their  messages  to  be  remembered  should  create  arousing  messages  that  are  slow  or  medium  paced,  or  calm  messages  that  are 
medium  or  fast  paced.  She  concluded  that  producers  should  not  create  messages  that  are  either  calm  and  slow  paced  or  arousing 
and  fast  paced  (Lang,  2000).  The  current  study  will  look  into  the  effect  of  integrating  increased  message  presentation  speed 
directly  into  instructional  messages  in  an  educational  environment  whose  content  might  be  considered  by  some  students  as  less 
than  arousing.  It  should  also  be  noted  that  Lang  (2000)  considered  cuts  to  be  fast  if  they  changed  at  a top  rate  of  eleven  to  twelve 
per  thirty-second  segment  (i.e.  one  every  2-3  seconds).  While  these  studies  are  important  in  their  own  right,  perhaps  their  speed 
of  presentation  may  not  be  fast  enough,  considering  an  observation  by  this  author  that  current  trends  in  television  editing 
techniques  tend  to  present  images  almost  ten  times  as  fast. 

Lack  of  Correlation  in  Educational  Settings 

These  investigations  into  the  effects  of  casual  television  viewing  have  not  translated  too  well  to  the  educational  setting 
(Salomon,  1994).  As  Kozma  (1986,  p.  14)  had  stated,  "viewership  should  not  be  confused  with  learning".  However,  once  one 
delves  deeper  into  these  studies,  four  possible  reasons  for  this  lack  of  generalizability  become  apparent.  First,  the  reputation 
commercial  television  has  held  for  being  nothing  more  than  an  entertainment  device  has  certainly  hurt  its  reputation  in 
educational  circles  and  has  caused  detrimental  pre-conceived  notions  about  how  viewers  are  to  be  properly  introduced  to 
televised  content  (Wetzel,  1994).  Second,  many  of  the  schema  used  in  commercial  television  are  considered  by  some  to  be 
nothing  more  than  prototypical,  trite,  and  overly  familiar  formulas  that  reduce  the  attention  and  concentration  because  they  have 
been  over- learned  (Anderson,  1979;  Langer,  1979).  In  these  cases,  encounters  with  overly  familiar  information  formats  “lead 
one  to  revert  to  a mindless  routine  in  which  the  material  is  ignored  or  receives  a low  level  of  attention”  (Wetzel,  1994,  p.  169). 
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This  alleged  over- familiarity  with  format  has  lead  many  educators  to  believe  that  viewers  will  have  difficulty  responding 
appropriately  to  educational  televised  presentations,  unless  some  form  of  intervention  is  used  (Wetzel,  1994).  Third,  in  many 
studies  into  casual  viewing,  there  has  been  a tendency  to  lump  all  viewers  into  a single  category  (Lang,  1998;  Lang,  1999;  Lang, 
2000;  McLuhan,  1964;  Neuman,  1976;  Tyner,  1998;  Walma  van  der  Molen,  2000).  Classical  instructional  models  tend  to  validate 
the  value  of  segregating  learners  and  classifying  them  by  their  individual  differences  (Gentry,  1998;  Joyce,  2000). 

The  fourth,  and  possibly  most  important  reason  for  the  lack  of  correlation  between  studies  of  casual  viewing  and  those 
performed  in  an  educational  setting  is  that  the  learning  environment  is  thought  to  present  a different  set  of  circumstances  - a 
different  view  if  it  were.  This  is  based  on  the  importance  placed  on  the  medium  to  be  evaluated  as  to  its  unique  ability  to  bring 
about  some  type  of  alteration  of  intellectual  behavior  or  thinking  process.  Although  many  of  the  symbolic  (i.e.  intellectual) 
combinations  found  in  non-verbal  endeavors  such  as  music,  painting,  and  dance  can  be  displayed  directly  on  television,  there  has 
been  some  question  as  to  the  extent  to  which  transformations  in  the  thinking  process  are  created  in  viewing,  and  whether  any 
changes  that  might  occur  are  of  any  significance  (Ide,  1974).  In  other  words,  television  has  not  been  given  credit  for  yielding  any 
new  intellectual  construct  of  its  own.  Previous  studies  into  the  value  of  using  television  as  a medium  for  intellectual  change  have 
demonstrated  mostly  negative  progress  (i.e.  television  does  not  actually  interfere  with  learning,  nor  is  it  no  less  effective  than 
other  forms  of  media)  (Thompson,  1996;  Wetzel,  1994). 

Rapid  Presentation  as  a Construct 

Although  there  appears  to  be  a dirth  of  studies  looking  into  the  conflict/benefit  of  rapidly -presented  visual  images  in 
educational  settings,  there  does  exist  some  research  that  clearly  demonstrates  the  potential  instructional  value  of  video  that  is 
aided  by  a systematic  variation  of  presentation  speed  of  as  a valid  instructional  strategy  (Comstock,  1978).  Intraub’s  (1999) 
experiments  into  individuals’  ability  to  understand  and  remember  briefly  glimpsed  images  dealt  with  pictures  that  were  not  or 
only  very  loosely  related.  The  use  of  montage  implies  that  the  pictures  included  are  at  least  conceptually  related.  Intraub  (1999) 
indicated  that  subjects  might  be  able  to  hold  more  than  one  picture  at  a time  in  a conceptual  buffer,  so  long  as  the  “series  was  not 
too  long”  (p.  57),  and  the  notion  that  the  included  pictures  were  related  to  one  another.  It  appears  that  humans  may  have  the 
ability  to  construct  meaning  from  these  types  of  presentations  through  the  use  of  interpretive  coding  (i.e.  the  process  by  which 
meanings  are  put  together  from  specific  parts  of  visual  communications).  In  describing  his  research  in  teaching  Native  Americans 
how  to  use  film  to  communicate  meaning,  Sol  Worth  (1997)  noted  that  the  process  of  coding  has  been  neglected  in  the  study  of 
most  of  the  fine  arts,  including  film.  His  comments  appear  to  be  alluding  to  the  fact  that  the  form  of  a medium  might  be  what 
carries  meaning.  In  an  earlier  attempt  t o extending  this  notion  to  television.  Pearl  (1982)  discussed  the  relationship  between  form 
and  content  and  admitted  that  it  is  the  form  (that  is,  the  way  it  uses  verbal  and  linguistic  codes),  not  the  content,  of  television  that 
is  unique.  However,  she  also  cautioned  that  form  and  content  cannot  always  be  distinguished  - “no  more  than  grammar  and 
meaning  in  any  verbal  language  can”  (p.  24).  However,  she  went  one  to  say  (Pearl,  1982)  that  some  forms  are  unique  to  the 
medium  and  apply  syntactical  meaning  only  in  the  context  of  that  medium.  For  example,  slow  motion  is  not  real  and  its  meaning 
must  be  learned.  However,  once  learned,  studies  have  shown  (Barnett,  2000)  that  these  formats  become  generally  used  by  p eople 
in  their  own  thinking  (i.e.  when  one  speaks  of  applying  slow  motion  to  a video  message,  it  generally  carries  the  same  contextual 
or  emotional  connotation).  The  current  study  tries  to  apply  the  same  logic  to  fast  cuts  and  video  montage  to  see  if  this 
presentation  format  can  be  interpreted  in  such  a way  so  as  carry  the  same  or  different  implied  meanings  than  slower  paced 
messages. 

Previous  research  into  using  presentation  speed  and  movement  in  educational  multimedia  has  been  the  subject  of  controversy 
in  the  literature  (Downs,  1989).  Downs  indicated  that  there  might  be  a new  wrinkle  on  evaluating  the  findings:  one  must  try  to 
“determine  if  children  are  attending  to  motion  but  not  expressing  it”  (p.  97).  She  alluded  to  possible  future  studies  that  should 
include  additional  cueing  strategies  to  determine  their  effect  on  learning.  Under  very  broad  interpretation,  the  speed  of  message 
presentation  of  motion  and  edits/cuts  may  be  considered  a form  of  cueing  (Lang,  2000).  Previous  research  also  referred  to 
presentation  speed  in  a similar  way,  referring  to  it  as  a message’s  domain  “attribute”  (Downs,  1989,  p.  3).  Salomon  (1979) 
acknowledged  that  media  attributes  are  that  “within  the  mediated  stimulus,  possibly  shared  to  some  extent  with  other  (forms  of 
(sic))  media,  and  makes  the  presented  information  more  comprehensible  or  better  memorized  by  learners  of  particular 
characteristics”  (p.  5-6).  Under  Salomon’s  definition,  symbols  include  “most  objects,  marks,  events,  models,  or  pictures”  (p.  29). 
It  is  assumed  in  this  study  that  the  rhythmic  patterns  afforded  by  fast -cuts  are  an  event.  Where  the  current  study  varies  from 
Downs’  is  that  it  takes  the  interrogation  of  symbols  and  attributes  to  another  level.  The  current  study  aims  to  show  that  rapid 
presentation  speed  (also  referred  to  as  fast-cuts)  may  be  considered  an  invaluable  attentional  attribute  of  media  and  is,  therefore, 
capable  of  being  studied  separately  to  discover  its  contribution  to  learning.  In  short,  this  study  aims  to  determine  to  what  extent 
the  (rapid)  presentation  speed  of  video  images  either  aids  or  interferes  with  learning,  considering  the  changes  that  appear  to  be 
taking  place  in  an  ever-increasing  mediacentric  society.  Another  consideration  is  the  way  in  which  the  effect  on  learning  is  to  be 
measured.  Mayer  (1996)  indicated  that  words  might  not  serve  as  proper  feedback  format  to  visual  motion  cues.  Archer  (1965) 
proposed  that  other  evaluative  vehicles  be  designed  that  are  more  closely  aligned  with  the  visual  processing  because  something 
may  be  lost  in  the  translation  to  verbal.  The  current  study  will  utilize  a combination  of  verbal  and  non-verbal  methods  to  evaluate 
its  results  that  are  more  closely  attuned  to  the  visual  perceptual  process. 
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New  Media,  a new  View  of  their  Impact 

Changes  in  media  technologies  create  the  need  for  taking  a new  look  at  it  potential  impact  on  education.  Child  development 
psychologists  like  Robert  Dorman,  Jr.  (1984)  have  noted  that  it  is  no  accident  that  tottlers  learn  more  in  their  first  five  years  than 
in  any  other  time  of  their  lives.  The  fact  is  that  they  perceive  and  process  images  very  rapidly  - at  a rate  that  equals  or  exceeds  a 
new  one  every  100-300  milliseconds.  These  observations  beg  the  question  as  to  what  happens  to  this  rapid  learning  process  just 
about  the  time  that  a child  enters  his  or  her  formal  educative  years.  What  do  current  educational  strategies  do  to  slow  the  learning 
rate  down?  Has  previous  research  been  hampered  by  a lack  in  technology  to  implement  and  subsequently  evaluate  rapid 
presentation  as  an  instructional  strategy?  Some  have  expressed  this  opinion  in  the  past.  Loftus  (1982)  found  only  one  doctoral 
dissertation,  and  no  published  work  had  taken  place  up  to  the  time  of  publication  of  his  review  of  the  literature  in  1984.  The 
current  author  has  found  one  more  in  a very  recent  research  of  the  literature,  and  that  one  also  took  place  before  Loftus’  work. 
The  truth  is,  almost  no  research  has  been  done  in  this  area  since  then. 

On  the  other  hand,  recent  contemporary  theories  about  fuzzy  trace  memory  (Brainerd,  1990;  Brainerd,  1994;  Reyna,  1994) 
might  be  the  key  to  looking  at  rapid  presentation  as  an  instructional  strategy.  Research  has  uncovered  that  as  a child  progresses 
through  developmental  stages,  s/he  begins  to  loose  specific  details  of  episodic  memory,  but  general  details  of  contextual 
information  last  much  longer  and  actually  become  the  sole  extent  to  which  a person  recall  previous  events.  Research  into  this 
child  development  phenomenon  has  alternately  given  rise  to  new  thinking  about  human  memory  in  educational  settings.  In  short, 
fuzzy  trace  proposes  that  there  might  be  two  distinct  memory  tracks,  one  that  deals  with  verbatim  information,  and  another  that 
tracks  essential  contextual  (i.e.  gist)  information  that  aids  in  the  process  of  synthesis  and  analysis.  These  studies  have  indicate 
that  both  verbatim  and  gist  memory  can  be  stimulated  with  very  short  bursts  of  visual  stimuli.  While  verbatim  memory  might  be 
subject  to  masking  limitations,  gist  memory  has  been  found  to  be  much  more  durable.  It  is  obvious  that  more  work  in  this  newly 
created  field  needs  to  be  done  before  any  specific  changes  in  educational  strategies  can  be  made.  However,  successes  in 
experiments  dealing  with  numbers  (Brainerd,  1994)  and  text  (Reyna,  1994)  are  promising. 

Summary 

In  spite  of  previous  experiments  into  memory  associated  with  rapidly -presented  visual  messages,  the  time  might  be  right  to 
revisit  this  field.  We  appear  to  be  living  in  an  age  where  information  overload  might  be  in  vogue  . Previous  experiments  were 
hampered  by  the  lack  of  any  stable  technology  to  recreate  for  testing  purposes  in  a consistent  manner  rapidly-presented  visual 
imagery.  That  no  longer  appears  to  be  problem.  According  to  some,  today’s  youth  who  live  in  this  world  not  only  appear  to  be 
attracted  to  the  rapid  pace,  but  siso  might  have  learned  how  to  cope  with  it  and  utilize  it  as  a more  efficient  way  to  take  in 
information.  Child  psychologists  studying  right-brain  activity  (Shichida,  1993)  have  known  for  years  that  image  training  in 
infants  produces  remarkable  mathematical  and  reasoning  capabilities.  In  fact,  the  right  brain  can  process  cognition  without  any 
fixed  memory.  The  right  brain  functions  beyond  one’s  consciousness.  Processing  rapid  montage  appears  to  be  a right  brain 
activity.  In  short,  rapid  montage  video  may  be  able  to  play  a significant  role  in  the  learning  process  because  it  matches  three 
essential  elements  of  perception  and  learning: 

• Frequency 

• Intensity 

• Duration 

The  rapid  changes  in  visual  stimuli  have  been  shown  to  gain  viewers’  attention,  and  cause  viewers  to  concentrate  longer. 
Educational  research  may  well  benefit  from  discovering  whether  visual  stimuli  presented  at  an  increased  delivery  pace  can 
provide  similar  instructional  outcomes  in  a more  efficient  and  stimulating  manner  (Cobb,  1997).  This  is  something  that  might  be 
a more  practical  outcome  measurement  for  using  video  media  than  achievement  alone. 
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Abstract 

Reflection  involves  active,  persistent,  and  careful  consideration  of  any  belief  or  practice.  It  promotes  understanding  of 
underlying  beliefs  and  application  of  new  knowledge  to  new  situations.  Problem-Based  Learning  (PBL)  provides  the 
instructional  mechanisms  for  prompting  learner  reflective  thinking.  This  study  found  that  young  students  perceived  three  factors 
as  most  important  in  supporting  their  reflection  in  PBL  lessons:  learning  environment,  teacher,  and  scaffolding  tools.  Reflective 
activities  are  described  and  implications  for  designing  PBL  discussed. 

Introduction 

Modem  society  is  becoming  more  complex,  information  is  becoming  available  and  changing  more  rapidly  prompting  users  to 
constantly  re-think,  switch  directions,  and  change  problem-solving  strategies.  Thus,  it  is  increasingly  important  to  help  young 
students  develop  keen  reflective  thinking  capabilities  during  learning  that  help  them  construct  strategies  for  applying  new 
knowledge  to  complex  situations  in  their  day-to-day  activities.  Reflective  thinking  helps  learners  develop  higher-order  thinking 
skills  by  prompting  learners  to  a)  relate  new  knowledge  to  prior  understanding,  b)  think  in  both  abstract  and  conceptual  terms,  c) 
apply  specific  strategies  in  novel  tasks,  and  d)  understand  their  own  thinking  and  learning  strategies  (Hmelo  & Ferrari,  1997). 
PBL  provides  learners  with  instructional  mechanisms  that  can  increase  their  reflective  thinking  while  exploring  authentic  and  ill- 
structured  problems,  participating  in  social  interactions,  and  receiving  coaching  from  peers  and  teachers  (Albanese  & Mitchell, 
1993;  Donahuse,  1999;  Hmelo  & Ferrari,  1997).  This  mindful  stance  toward  learning  is  essential  for  efficient  development  of 
reflective  thinking  and  ultimately  knowledge  construction.  However,  the  research  on  factors  that  may  affect  reflection  during 
PBL  is  limited. 

Previous  studies  have  sought  to  identify  factors  that  influenced  reflection  by  looking  at  the  activities  in  which  learners  engage 
during  the  PBL  process  (van  den  Hurk,  et  al.,  1999).  These  studies  have  identified  factors  that  may  encourage  reflective  thinking 
but  it  is  still  unclear  which  factors  the  learner  feels  prompts  valuable  reflection.  Answering  questions  such  as  how  do  we  support 
reflective  thinking  in  a PBL  environment  requires  identifying  both  the  factors  that  might  prompt  reflective  thinking  and 
examining  learner  perceptions  about  those  factors.  Therefore,  the  purpose  of  this  study  was  to  identify  factors  that  learners 
perceived  as  important  in  facilitating  their  own  reflections  during  learning  activities. 

Examining  research-based  factors  for  prompting  reflective  thinking  in  a PBL  environment  would  be  helpful  in  several  areas. 
First  it  would  simplify  the  further  analysis  of  factors  prompting  reflective  thinking  in  PBL  by  reducing  the  number  of  variables. 
Second  it  would  provide  a meaningful  and  useful  framework  for  discussing  design  factors  that  support  learners’  reflective 
thinking  when  participating  in  a PBL  lesson  in  a classroom. 

Theoretical  Framework 

What  is  reflective  thinking?  John  Dewey  introduced  the  concept  “reflective  thought.”  Dewey's  most  basic  assumption  was 
that  learning  improves  to  the  degree  that  it  arises  out  of  the  process  of  reflection.  Dewey  (1933)  defined  reflective  thinking  as 
“active,  persistent,  and  careful  consideration  of  a belief  or  supposed  form  of  knowledge  on  the  grounds  that  reflective  thinking 
supports  the  belief  or  knowledge  and  the  further  conclusions  one  can  draw  about  it.  This  cycle  is  determined  by  the  production  of 
changes  one  finds  on  the  whole  satisfactory  or  by  the  discovery  cf  new  features  which  give  the  situation  new  meaning  and 
change  the  nature  of  questions  to  be  explored”. 

Moon  (1999)  believed  that  reflective  thinking  is  a chain  of  ideas  that  is  aimed  at  a conclusion  and  is  more  than  a stream  of 
consciousness;  whereas.  Canning  (1991)  believed  that  reflective  thinking  was  a behavior  that  involves  active,  persistent,  and 
careful  consideration  of  any  belief  or  practice  that  promotes  understanding  of  underlying  beliefs  and  applying  newly  gained 
knowledge  to  new  situations.  These  studies  agree  that  reflective  thinking  includes  the  process  of  analyzing  and  making  judgment 
about  what  has  happened.  Reflective  thinking  experiences  are  associated  with  increased  motivation,  willingness  to  take  risks, 
enhanced  self-esteem  and  independence. 

Why  is  it  important  to  support  reflective  thinking  in  a PBL  lesson?  We  can  find  an  answer  in  the  characteristics  of  PBL.  PBL 
provides  an  environment  where  learners  encounter  ill  structured  problem  situations.  In  reality,  PBL  takes  place  in  settings  that  are 
characterized  by  a great  deal  of  ambiguity,  complexity,  variety,  and  conflicting  values  that  make  unique  demands  on  the  learner's 
skills  and  knowledge.  As  a result,  learners  in  a PBL  lesson  are  constantly  making  choices  about  the  nature  of  practice  problems 
and  how  to  solve  them.  Learners  must  be  able  to  change  ill-defined  practice  situations  into  those  in  which  they  are  more  certain 
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about  the  most  appropriate  course  of  action  to  pursue.  Therefore,  the  ability  to  reflect  while  acting  is  necessary  to  maintain  the 
essence  of  effective  practice  in  a PBL  lesson. 

Prompting  reflective  thinking  is  especially  important  to  young  students  because  the  students  of  this  age  are  experiencing 
many  developmental  transits.  Reflective  thinking  doesn’t  occur  in  middle-level  students  spontaneously.  According  to  King  and 
Kitchener  (1994),  reflective  thinking  has  seven  developmental  stages.  Reflective  judgment  is  in  the  seventh  stage  and  the  term 
that  they  apply  to  the  most  advanced  stage  in  their  model.  People  in  the  seventh  stage  who  have  reflective  judgment  can 
acknowledge  that  there  is  no  right  answer  and  experts  may  disagree  as  to  the  best  solution  of  a dilemma.  King  & Kitchener 
describe  reflective  judgment  as  similar  to  ‘wisdom’  that  adults  usually  have.  However,  middle  school  students  are  in  a different 
developmental  stage  than  adults.  The  National  Middle  School  Association  reports  that  middle  level  students  are  in  a transition 
period  from  concrete  thinking  to  abstract  thinking.  Therefore,  they  need  some  supportive  activities  to  prompt  their  reflection  in 
order  for  them  to  make  learning  meaningful  and  active.  Therefore,  it  is  necessary  to  prompt  reflective  thinking  for  middle-level 
students  who  are  in  their  concrete  thinking  stage. 

How  then,  do  we  prompt  middle  school  students’  reflective  thinking  in  a PBL  lesson?  Research  suggests  that  various 
elements  in  PBL  are  related  to  prompting  students’  reflective  thinking.  Previous  research  indicates  that  tasks,  teachers, 
instructional  environments,  and  reflective  thinking  tools  are  key  elements  that  support  reflective  thinking  in  PBL  (Andrusyszyn, 
1997;  Lin,  1999;  Moon,  199;  Barrow,  1998).  First,  ill-structured,  authentic,  and  complex  tasks  are  known  to  promote  reflective 
thinking.  These  features  of  the  task  help  students  think  reflectively  because  they  come  from  real-world  experiences,  have  no 
single  formula  for  conducting  an  investigation  to  resolve  the  problem,  and  require  more  information  to  understand  the  problem 
situation  (Stepien  & Pyke,  1997).  Second,  the  role  of  teacher  is  important  in  prompting  reflective  thinking  during  PBL. 
According  to  Virtanen  et  al.  (1999),  both  facilitating  teachers  and  traditional  teachers  are  effective  in  a PBL  environment.  A 
teacher  who  prefers  facilitative  activities  may  help  learners  by  asking  reflective  questions  while  a traditional  teacher  explains  or 
directs  important  reflective  concepts  to  students  (Moon,  1999;  Virtanen  et  al,  1999).  Third,  flexible  and  active  learning 
environments  are  also  important  in  prompting  reflective  thinking  during  PBL.  Effective  PBL  requires  a relaxed  atmosphere  that 
can  promote  cooperative  and  collaborative  learning  and  is  conducive  to  students  and  teachers  exploring  misunderstandings 
together  (Michale  & Susan,  1998).  Finally,  the  scaffolding  tools  are  important  in  prompting  reflective  thinking  during  PBL. 
Andrusyszyn  & Daive  (1997)  and  Kinchin  & Hay  (2000)  posit  that  there  are  three  main  types  of  tools  that  scaffold  reflective 
thinking:  reflective  journals,  guiding  questions,  and  concept  maps. 

However,  incorporating  these  factors  into  PBL  may  or  may  not  enhance  reflective  thinking.  Learners  may  perceive  different 
factors  in  the  environment  as  important  in  promoting  their  own  reflection.  Therefore,  understanding  how  learners  perceive  the 
importance  of  each  factor  in  prompting  their  thinking  about  their  learning  is  important  in  designing  effective  PBL  environments, 
which  has,  thereby,  prompted  the  following  research  questions; 

1 . What  are  the  factors  that  students  perceive  as  prompting  reflective  thinking? 

2.  Which  factor  is  perceived  as  the  most  important  for  prompting  learners’  reflective  thinking? 

3.  Is  there  a significant  difference  between  the  derived  factors? 

4.  Which  elements  or  characteristics  prompt  reflective  thinking  within  the  derived  factors? 

Method 

Subjects 

One  hundred  and  forty-four  sixth  through  eighth  grade  students  attending  three  different  middle  schools  in  rural  Pennsylvania 
participated.  Students  were  from  6 different  classrooms;  including  82  boys,  59  girls,  and  3 who  did  not  identify  their  gender. 

Instrument 

A survey  questionnaire  for  measuring  the  perceived  factors  related  to  reflective  thinking  in  PBL  was  designed  by  the  authors 
based  on  the  literature  of  reflective  thinking.  The  instrument  consisted  of  1 0 items  that  were  scored  on  a 5-point  Likert  scale  from 
strongly  agree  (5)  to  strongly  disagree  (1).  The  survey  was  reviewed  for  content  and  face  validity  and  then  tested  with  a small 
sample  of  middle  school  children  to  establish  readability.  The  Cronbach  alpha  reliability  of  the  final  survey  was  .890. 

Data  Source  and  Analysis 

Quantitative  data  on  the  perception  questions  were  collected  prior  to  participation  in  Problem-based  Learning  lessons.  Data 
were  collected  over  a five-month  period,  between  October  2000  and  March  200 1 . Maximum  likelihood  extraction  and  varimax 
rotation  method  were  conducted  for  the  factor  analysis,  using  the  SPSS/PC+  statistical  package. 

Results 

In  response  to  the  first  research  question,  what  are  the  factors  that  students  perceive  as  prompting  reflective  thinking;  three 
factors  emerged  from  the  data  based  using  an  Eigen  value  of  1 .0.  Based  on  the  literature  review  about  design  attributes  required 
for  reflective  thinking,  these  findings  were  encouraging.  (See  Table  1.)  Three  items  (teacher  explanation,  teacher  question,  and 
authentic  task)  were  in  the  same  factor  1 , five  items  (having  freedom  in  class,  working  with  a partner,  working  with  an  ill- 
structured  task,  having  time  to  think,  and  drawing  pictures)  were  in  the  same  factor  2,  and  two  items  (answering  questions  and 
writing  about  my  understanding)  clustered  in  the  same  factor  3.  The  main  characteristics  of  factor  I appear  to  relate  to  the  teacher 
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variable,  those  of  factor  2 appear  to  relate  to  the  student  learning  environmental  variable,  and  those  of  factor  3 appear  to  relate  to 
the  tool  variable  for  prompting  reflective  thinking 


Table  1.  Factor  loading  of  elements  that  prompt  reflective  thinking  in  PBL 


Item 

Item  content 

Factor  1 

Factor  2 

Factor  3 

3 

When  my  teacher  explains  how  to  solve 
difficult  tasks  it  helps  me  think  more  about 
what  I am  studying. 

.909 

,104 

,217 

4 

When  my  teacher  asks  me  how  to  solve 
difficult  tasks  it  helps  me  think  more  about 
what  I am  studying. 

.566 

,297 

.398 

2 

Working  on  activities  in  class  related  to  real 
problems  on  earth  or  in  our  society  helps  me 
think  more  about  what  I am  studying. 

J88 

,385 

.208 

7 

Having  freedom  in  class  to  explore  topics  I am 
interested  in  helps  me  think  more  about  what  I 
am  studying. 

,181 

.600 

5 

Working  with  partners  during  classroom 
activities  helps  me  think  more  about  what  I am 
studying. 

.122 

.491 

1 

Working  on  activities  in  class  that  have  many 
different  answers  helps  me  think  more  about 
what  I am  studying. 

.247 

.475 

.360 

6 

Having  time  to  think  about  a question  before 
answering  helps  me  think  more  about  what  I 
am  studying. 

.428 

.460 

.116 

8 

Drawing  pictures  to  illustrate  my 
understanding  of  a topic  helps  me  think  more 
about  what  I am  studying. 

.451 

,225 

10 

Answering  questions  about  a topic  helps  me 
think  more  about  what  I am  studying. 

,172 

.137 

.782 

9 

Writing  about  my  understanding  of  a topic 
helps  me  think  more  about  what  I am 
studying. 

.182 

.600 

In  response  to  the  second  and  third  research  questions,  which  factor  is  perceived  as  the  most  important  for  prompting 
learners*  reflective  thinking  and  is  there  a significant  difference  between  the  derived  factors,  the  highest  ranked  factor  mean  was 
the  student  learning  environment  factor  (Factor  2,  M=  3,87),  followed  by  the  teacher  factor  (factor  1,  M=  3,62)  and  tool  factor 
(Factor  3,  M=  3,21),  See  Table  2,  A paired  sample  t-test  analysis  was  carried  out  to  compare  the  factor  means  scores  in  three 
factor  groups.  The  paired  sample  t-tests  indicated  that  there  were  significant  differences  between  the  factors.  The  mean  score  of 
factor  2 is  significantly  higher  than  that  of  factor  1 or  factor  3 (p<  ,01),  The  mean  score  at  factor  1 is  also  significantly  higher 
than  that  of  factor  3 (p<  ,01),  This  result  shows  that  students  perceive  the  student  learning  environment  factor  (factor  2)  as  the 
most  significant  factor  to  help  think  reflectively.  The  student  learning  environment  fictor  included  student -centered  attributes 
such  as  more  flexible  atmosphere,  time,  and  tasks,  peer  tutoring  activities,  and  bursts  of  activities  incorporating  a drawing 
Therefore  students  perceive  a student -centered  learning  environment  as  prompting  more  reflective  thinking  than  a teacher- 
centered  environment  that  provides  questions  and  explanations  from  teachers  or  a simple  supportive  learning  environment  that 
includes  reflective  thinking  tools  such  as  questions  or  writing. 

In  response  to  the  fourth  research  question,  what  elements  or  characteristics  prompt  reflective  thinking  within  the  derived 
factors,  the  highest  ranked  elements,  both  of  a social  nature  and  loaded  to  factor  2,  were  having  freedom  to  explore  topics  in  class 
(x=  4,10)  and  working  with  partners  (x=  4,05).  See  table  3, 

Table  2,  Paired  samples  t-test  for  factors 


Mean 

Std.  Deviation 

t 

df 

Sig  (2-tailed) 

FI 

3.62 

.80 

-4.209 

140 

.000 

F2 

3.87 

.62 

F2 

3.87 

.61 

8.471 

142 

.000 

F3 

3.21 

.92 

FI 

3.62 

.80 

5.243 

140 

.000 

F3 

3.21 

.93 
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Table  3,  Mean  and  standard  deviation  of  question  lists 


Item 

Item  content 

Mean 

SD 

1 

Working  on  activities  in  class  that  have  many 
different  answers  helps  me  think  more  about 
what  I am  studying. 

3,706 

,93 

2 

Working  on  activities  in  class  related  to  real 
problems  on  earth  or  in  our  society  helps  me 
think  more  about  what  I am  studying. 

3,63 

,99 

3 

When  my  teacher  explains  how  to  solve  difficult 
tasks  it  helps  me  think  more  about  what  I am 
studying. 

3,71 

,90 

4 

When  my  teacher  asks  me  how  to  solve  difficult 
tasks  it  helps  me  think  more  about  what  I am 
studying. 

3,52 

1,03 

5 

Working  with  partners  during  classroom 
activities  helps  me  think  more  about  what  I am 
studying. 

4,05 

,94 

6 

Having  time  to  think  about  a question  before 
answering  helps  me  think  more  about  what  I am 
studying. 

3,90 

,82 

7 

Having  freedom  in  class  to  explore  topics  I am 
interested  in  helps  me  think  more  about  what  I 
am  studying. 

4,10 

,94 

8 

Drawing  pictures  to  illustrate  my  understanding 
of  a topic  helps  me  think  more  about  what  I am 
studying. 

3,59 

1,07 

9 

Writing  about  my  understanding  of  a topic  helps 
me  think  more  about  what  I am  studying. 

2,94 

1,13 

10 

Answering  questions  about  a topic  helps  me 
think  more  about  what  I am  studying. 

3,48 

.99 

Conclusions,  Limitations,  and  Importance  to  Instructional  Design 

Students  perceived  three  major  factors  as  most  important  in  prompting  their  reflective  thinking;  student  learning 
environment,  teacher,  and  tools,  ranked  respectively.  Of  further  importance  was  that  the  social  activities  within  the  environment 
were  ranked  as  most  important,  demonstrating  the  importance  of  social  learning  to  students.  Previous  research  on  the 
collaborative  learning  activities  support  the  importance  of  social  learning  (Koschman,  Kelson,  Feltovich,  & Barrows,  1996), 
Although  the  kids  “liked”  the  collaborative  activities,  their  teachers  found  them  to  be  inexperienced  in  group  decision-making 
and  collaborative  learning.  Further  research  is  needed  to  investigate  how  to  scaffold  students  for  successAil  participating  in 
collaborative  activities. 

Further  research  is  also  needed  to  examine  how  these  three  factors,  student  learning  environment,  teacher,  and  tools,  interact 
with  each  other.  Given  that  the  student  learning  environment  emerged  as  the  most  important  factor,  it  is  important  to  further 
refine  and  test  the  attributes  of  this  factor.  This  process  will  help  to  determine  whether  there  are  other  specific  components  in  the 
learning  environment  that  student  perceive  as  prompting  their  reflective  thinking. 

The  findings  of  this  study  have  important  implications  on  the  design  of  problem-based  learning  environment  so  that  they  will 
prompt  reflective  thinking.  Student -centered  environments  that  have  a more  flexible  atmosphere,  and  provide  many  venues  for 
social  learning  may  have  a stronger  impact  on  learner’s  perception  on  reflective  thinking.  This  research  raised  implications  about 
students’  perceptions  of  factors  that  prompt  their  thinking  and  learning.  Designing  PBL  that  prompts  reflective  thinking  and 
ultimately  deeper  learning  can  be  achieved  by  better  understanding  learner  perceptions  about  factors  that  prompt  their  reflective 
thinking. 
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Older  Adults  Eager  to  Explore  Cyberspace 


Dianne  Ford  Lawton 
Valdosta  State  University 


Abstract 

This  study  compared  two  methods  of  computer  instruction  for  older  adults.  Elder  Computer  Instruction  was  systematically 
designed  and  developed  according  to  criteria  established  by  both  theory  and  research  in  andragogy.  This  instructional  design 
took  into  consideration  identified  cognitive  and  physical  changes  that  accompany  the  aging  process.  Traditional  Computer 
Instruction  consisted  of  generic  computer  instruction  commonly  used  with  adults  of  all  ages.  A checklist,  the  Criteria  Checklist 
for  Andragogical  Principles,  was  developed  to  help  instructional  designers  and  educators  assess  the  adherence  of  instructional 
materials  to  andragogical  principles.  The  systematic  process  for  instructional  design  detailed  in  the  study  should  be  of  use  to 
those  involved  in  the  design,  development,  and  delivery  of  instruction  for  older  adults.  The  study  investigated  the  effects  of  both 
types  of  instruction  on  older  adults’  computer  attitudes,  frequency  of  computer  use,  and  types  of  computer  tasks  performed. 
Participants  in  both  groups  completed  the  Attitudes  Toward  Computers  Questionnaire  that  assessed  six  dimensions  of  attitudes 
toward  computers.  Participants  receiving  Elder  Computer  Instruction  had  significantly  more  positive  attitudes  in  the  dimensions 
of  efficacy,  interest,  and  utility.  The  Computer  Task  Frequency  Survey  was  given  as  a pretest/posttest.  Both  groups  increased  in 
their  frequency  of  computer  use  and  types  of  computer  tasks  performed  following  instruction.  There  was  a significant  difference 
between  the  two  groups  in  the  e-mail  task,  with  those  who  received  Elder  Computer  Instruction  sending  more  e-mail  than  those 
who  received  Traditional  Computer  Instruction.  This  study  underlines  the  importance  of  designing  instruction  to  meet  the 
specialized  learning  needs  of  older  adults. 

Older  Adults  Eager  to  Explore  Cyberspace 

As  people  age,  once-active  adults  often  find  their  world  gradually  shrinking  as  they  become  less  mobile  due  to  physical 
impairments  that  affect  walking  and  driving  (Coughlin,  1999).  The  Internet,  however,  offers  older  adults  new  opportunities  for 
communication  and  car-less  access  to  religious  services,  cultural  activities,  and  educational  opportunities  (Whelen,  1998). 
According  to  Czaja  and  Sharit  (1998),  if  older  adults  are  successful  in  acquiring  basic  computer  skills,  they  will  realize  that  their 
daily  lives  can  be  enhanced  and  enlivened  by  using  technology.  Because  most  older  adults  did  not  learn  to  use  computers  in 
school  (Dunnett,  1998),  or  in  the  workplace,  those  who  now  want  to  use  them  must  find  sources  for  instruction. 

Computer  use  among  older  adults  is  increasing  rapidly.  According  to  the  U.  S.  Census  Bureau  (2000),  27.6%  of  adults  over 
the  age  of  55  live  in  a home  with  Internet  access.  A survey  by  International  Data  Corporation  reports  that  adults  over  the  age  of 
55  account  for  more  than  12  million  Internet  users,  an  increase  of  106%  over  users  in  the  same  age  group  in  1999.  By  2004,  this 
age  group  is  expected  to  comprise  20%  of  all  new  Internet  users  (Hoffman,  2000). 

Although  computers  are  becoming  less  expensive  and  easier  to  use,  older  adults  still  face  the  problem  of  acquiring  basic 
computer  skills.  Morrell,  Mayhom,  and  Bennett  (2000)  surveyed  1 15  Web  users  and  266  Web  nonusers  between  the  ages  of  40- 
92.  More  than  81%  of  the  Web  users  reported  that  they  had  taught  themselves  to  use  the  Web.  The  two  primary  reasons  given  by 
the  nonusers  for  not  using  the  Web  were  the  lack  of  computer  access  and  lack  of  knowledge  about  using  the  Web.  Both  groups 
indicated  that  they  would  like  to  have  simple  instructional  materials  to  show  them  how  to  use  the  different  features  of  the  Web. 
Morrell  and  Echt  (1997)  point  out  that  older  adults  who  are  interested  in  learning  about  computers  often  have  to  search  for 
training  opportunities  or  have  to  teach  themselves  to  use  computers.  The  instructional  materials  they  use  are  often  designed  for 
much  younger  people. 

Research  indicates  that  adults  learn  best  when  the  learning  goal  is  articulated  clearly  and  when  they  can  apply  their  learning 
to  real-life  problems  (Redding,  Eisenman,  & Rugolo,  1998).  The  necessity  for  taking  into  account  the  learner's  readiness  to  learn, 
life  experiences,  self-direction,  intrinsic  motivation,  and  problem-solving  ability,  as  well  as  the  immediate  value  to  the  learner,' 
forms  the  basis  of  the  adult  learning  theory  known  as  andragogy.  One  of  the  strengths  of  this  theory  is  its  adaptability  to  the 
uniqueness  of  learners  and  to  various  learning  situations.  Because  the  principles  of  andragogy  can  be  applied  to  a variety  of  adult 
learning  situations  (Knowles,  Holton,  & Swanson,  1998),  it  is  an  ideal  conceptual  framework  on  which  to  base  computer 
instruction. 

The  practice  of  teaching  adults  could  be  said  to  have  begun  with  philosophers  and  religious  leaders  such  as  Confucius, 
Socrates,  Plato,  and  Jesus.  It  was  not,  however,  until  the  second  half  of  the  twentieth  century  that  Malcolm  Knowles  integrated 
various  adult  teaching  methods  into  an  adult  learning  theory  that  is  now  called  andragogy.  Knowles  (1990)  found  that  andragogy 
was  especially  pertinent  to  computer  instruction.  He  points  out  four  characteristics  of  adult  learners  that  should  inform  the  design 
of  computer  instruction  for  adults.  First,  because  adults  have  a deep  need  to  know  why  they  should  learn  something  before  they 
invest  their  time  and  energy,  computer  instructors  should  explain  the  purpose  of  specific  computer  functions.  Next,  instructors 
should  start  with  adult  learners  interests  because  adults  learn  best  those  things  that  they  must  know  in  order  to  perform  tasks  that 
are  relevant  to  them.  Instructors  should  ask  learners  to  write  their  personal  goal  for  computer  use.  Third,  the  instructor  should 
find  out  the  background  experiences  of  the  learners  in  order  to  give  them  choices  based  on  their  prior  experiences.  Instructors 
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should  administer  a pretest  survey  to  determine  the  computer  tasks  the  learners  have  performed  prior  to  the  instruction.  Finally, 
adults  are  self-directing  and  dislike  having  decisions  imposed  on  them.  For  this  reason,  teachers  should  allow  adults  to  figure 
things  out  for  themselves.  Teachers  who  become  facilitators  rather  than  directors  of  learning  create  a nurturing  learning 
environment. 

Researchers  such  as  Czaja  and  Sharit  ( 1 998)  and  Jay  and  Willis  ( 1 992)  found  that  experience  with  computers  increases  older 
adults’  feelings  of  comfort  with  technology,  competence  with  computers,  and  feelings  that  computers  are  useful.  Redding, 
Eisenman,  and  Rugolo  (1998)  note  that  adults  learn  best  when  the  learning  goal  is  articulated  clearly  and  when  they  can  apply 
their  learning  to  real-life  problems. 

In  addition  to  the  characteristics  of  the  individual  adult  learner,  the  effects  of  group  experience  are  also  relevant.  Because 
learning  is  a social  practice  (Knights,  1993),  the  communal  dimension  of  learning  should  not  be  undervalued,  Knights  points  out 
that  groups  can  exert  a powerful  influence  to  advance  learning,  Dixon  and  Gould  (1996)  found  tfiat  when  older  adults 
collaborated  with  others  in  a problem-solving  situation,  their  cognitive  performance  was  enhanced.  According  to  Gaboon  (1996), 
when  members  of  a group  learn  computer  skills  together,  they  can  share  their  skills  and  knowledge  as  they  solve  computer 
problems  through  informal  interactions  with  other  group  members. 

Statement  of  the  Problem 

As  America's  population  ages,  it  has  become  increasingly  important  that  older  adults  have  access  to  computer  instruction  that 
will  enable  them  to  feel  more  included  in  our  technological  society  (Morris,  1 994),  Furthermore,  computer  instruction  should  be 
designed  specifically  to  meet  their  needs.  This  study  focused  on  the  need  to  design  instructional  materials  and  to  develop  both 
instructional  and  grouping  strategies  that  were  theoretically  grounded  in  the  principles  of  andragogy.  This  instruction  took  into 
consideration  identified  cognitive  and  physical  changes  associated  with  aging.  These  changes  include  changes  in  vision,  hearing 
loss  (White  et  al,,  1999),  and  decline  in  working  memory,  which  is  the  process  of  storing  information  that  is  necessary  in  order  to 
perform  certain  cognitive  tasks  (Salthouse  & Babcock,  1991), 


Purpose  of  the  Study 

This  study  focused  on  the  systematic  design,  development,  implementation,  and  evaluation  of  instruction  to  address  the  needs 
of  older  adults.  The  purpose  of  this  study  was  to  compare  two  methods  of  computer  instruction  for  older  adults.  The  study 
investigated  the  effects  of  both  types  of  instruction  on  older  adults'  computer  attitudes,  frequency  of  computer  use,  and  types  of 
computer  tasks  performed.  In  addition,  the  study  considered  the  effects  of  grouping  on  the  computer  attitudes  of  older  adults. 

Elder  Computer  Instruction 

A needs  assessment  was  conducted  among  a representative  group  of  54  older  adults  (Lawton,  1 999),  The  results  indicated 
that  this  population  perceived  themselves  to  have  the  greatest  performance  discrepancy  with  regard  to  computer  competencies 
when  they  compared  themselves  to  other  age  groups.  Next,  computer  instruction  for  older  adults  was  designed.  The  instructional 
materials  were  tested  in  a field  trial  and  then  refined  and  revised  based  on  feedback  from  participants  in  a posttest  survey  and  a 
focus  group. 

Elder  Computer  Instruction  was  systematically  designed  computer  instruction.  It  was  based  on  criteria  established  from  both 
theory  and  research  in  andragogy  and  cognition  and  was  developed  using  a process  called  instructional  systems  design. 
According  to  Seels  and  Rchey  (1994),  instructional  systems  design  is  an  organized  procedure  that  includes  the  steps  of 
analyzing,  designing,  developing,  implementing  and  evaluating  instruction.  Analyzing  is  the  process  of  defining  what  is  to  be 
learned;  developing  is  the  process  of  authoring  and  producing  the  instructional  materials;  implementing  is  actually  using  the 
materials  and  strategies  in  context,  and  evaluating  is  the  process  of  determining  the  adequacy  of  the  instruction.  The  Dick  and 
Carey  (1996)  model  was  chosen  as  the  instructional  design  framework  because  it  provides  a strong,  fundamental  process  of 
instructional  design  that  incorporates  learning  theory  research  and  practical  application.  The  strength  of  this  model,  which 
provides  a step  -by-step  process  to  design  instruction,  is  that  all  design  components  work  together  systematically  to  produce 
effective  instruction  and  evaluation.  The  components  of  a learning  system  are  made  up  of  the  learners,  the  instructor,  the 
instructional  materials,  and  the  learning  environment.  The  focus  of  a systematic  instructional  design  is  on  what  the  learner  is  to 
know  at  the  conclusion  of  the  instruction  (Dick  & Carey),  Design  decisions  based  on  data  are  particularly  important  in  this 
model,  thus  the  emphasis  on  needs  assessment,  formative  evaluation  and  field  trials. 

Traditional  Computer  Instruction 

Traditional  Computer  Instruction  is  generic  computer  instruction  commonly  used  with  adults  of  all  ages.  It  was  not 
specifically  designed  for  older  adults.  Lectures  were  used  to  teach  the  history  of  computers,  the  purposes  of  computer  systems, 
the  functions  of  computer  hardware,  the  importance  of  a comfortable  working  environment,  and  the  use  of  computer  terminology. 
Hands-on  activities,  beginning  with  lower  level  skills,  were  used.  Activities  were  modeled,  but  in  some  instances  learners  were 
not  given  time  to  practice  the  activity.  The  computer  manual  contained  the  same  information  as  the  lecture  and  was  uniform  for 
all  adult  age  groups.  Most  of  the  instructions  were  given  orally.  No  memory  aids  in  the  form  of  written  step-by-step  directions 
were  provided.  Learners  were  instructed  to  use  the  "help"  function  on  the  task  bar  if  they  forgot  a computer  application. 

The  two  instructional  methods  compared  in  this  study  shared  the  same  terminal  goal,  that  of  enabling  older  adults  to  develop 
basic  computer  skills.  The  differences  in  both  the  instructional  content  and  the  teaching  strategies  in  the  two  methods,  however, 
were  considerable. 
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Methodology 

A quasi-experimental  design  was  chosen  as  the  framework  for  this  empirical  study.  According  to  Huck  and  Cormier  (1996), 
the  most  frequently  used  quasi-experimental  design  is  the  nonequivalent  control  group  design.  The  group  using  Elder  Computer 
Instruction  was  the  treatment  group;  the  group  using  Traditional  Computer  Instruction  served  as  the  comparison  (control  group). 
This  study  utilized  the  nonequivalent  control  group  design  because  of  the  availability  of  pretest  data  and  the  fact  that  participants 
were  not  randomly  assigned  to  the  comparison  group. 

Instructional  Environment 

The  six  computer  workshops  for  the  treatment  group  (Elder  Computer  Instruction)  were  held  in  the  same  computer  lab 
located  in  the  Educational  Technology  Training  Center  at  a local  university.  Each  workshop  was  conducted  on  a Saturday  from 
9:00  a.m.  until  3:00  p.m.  with  an  hour  for  lunch.  In  addition,  a morning  and  an  afternoon  break  were  included  in  the  schedule. 
Participants  were  encouraged  to  take  any  extra  breaks  that  were  necessary.  Every  effort  was  made  to  ensure  that  participants  were 
comfortable.  In  addition  to  the  instructor,  at  least  two  facilitators  were  present  in  each  workshop  to  provide  individual  instruction 
and  to  answer  participants’  questions.  Participants  were  encouraged  to  ask  questions  at  any  time  during  the  instruction.  The  lab 
was  well  lighted  with  adequate  space  and  was  furnished  with  adjustable  chairs  on  rollers.  Although  the  17  computers  in  the  lab 
were  located  in  close  proximity,  there  was  enough  space  at  each  workstation  for  participants  to  work  comfortably.  Computers 
were  networked  to  one  printer  and  equipped  with  Windows  2000.  All  computers  had  Internet  access.  The  lab  also  contained  a 
teaching  station  and  a smart-board  projection  system.  Each  participant  was  provided  with  a computer  manual  and  a diskette. 

The  comparison  group  (Traditional  Computer  Instruction)  attended  computer  classes  held  in  a computer  lab  at  a local 
technical  college.  The  class  met  for  two  hours  on  Tuesdays  and  Thursdays  for  two  weeks.  Total  instructional  time  was  eight 
hours.  The  computer  lab  at  the  technical  college  was  spacious  and  well  lighted.  The  18  computers  in  the  lab  were  located  on  wide 
tables,  and  the  adjustable  chairs  were  equipped  with  rollers.  A white  board  at  the  front  of  the  room  was  provided  for  the 
instructor’s  use.  The  lab  contained  a teaching  computer  that  could  be  projected  onto  the  screen  at  the  front  of  the  room.  The 
computers  were  networked  to  one  printer  and  equipped  with  Windows  2000.  The  computers  did  not  have  Internet  access.  The 
instructor  provided  both  group  and  individual  assistance,  as  there  were  no  facilitators  to  assist  in  the  instruction.  Each  participant 
was  provided  with  a computer  manual  and  a diskette. 

Participants 

The  93  participants  in  this  study  consisted  of  older  adults  who  ranged  in  age  from  55  to  85  years  of  age  with  a mean  age  of 
68.4  years.  There  were  25  males  and  68  females.  Participants  were  divided  into  three  groups:  an  existing  group,  a newly  formed 
group,  and  a comparison  group.  Thirty -seven  participants  were  volunteers  who  were  members  of  three  different  existing  groups. 
Two  groups  were  from  area  churches,  and  one  group  consisted  of  a sorority  for  professional  women.  This  group  consisted  of  9 
males  and  28  females  with  a mean  age  of  69.1  years.  Twenty -seven  were  Caucasian  and  10  were  African-American.  These 
participants  were  previously  acquainted  and  shared  commonalities. 

There  were  45  participants  in  the  newly  formed  groups.  They  were  acquainted  with  few,  if  any  of  the  other  participants  prior 
to  the  computer  workshop.  These  participants  were  recruited  from  advertisements  in  the  newspaper,  fliers  in  area  grocery  stores, 
advertisements  in  senior  centers,  advertisements  at  a local  hospital  that  provides  senior  activities,  and  word-of-mouth  referrals. 
This  group  consisted  of  12  males  and  33  females  with  a mean  age  of  67.8  years.  Thirty-five  were  Caucasian  and  10  were  African- 
American. 

The  participants  in  the  comparison  group  began  with  four  males  and  eight  females.  One  female  dropped  out  after  the  second 
session.  The  remaining  participants  had  a mean  age  of  68.6  years.  All  were  Caucasian.  These  participants  were  enrolled  in  a basic 
computer  class  through  a continuing  education  program  at  a local  technical  college.  All  volunteered  to  take  part  in  the  study.  The 
number  of  participants  was  determined  by  course  registration  with  the  intent  to  compare  groups  of  approximately  equal  size. 
Instructional  materials  designed  for  this  study  were  compiled  in  a computer  manual  entitled  Seniors  Surf  Into  the  Twenty-first 
Century.  These  materials,  first  used  in  a field  trial,  were  developed  following  the  model  in  The  Systematic  Design  of  Instruction 
(Dick  & Carey,  1996).  This  development  process  included  formative  and  summative  evaluation  that  was  used  to  revise  the 
instructional  materials,  making  them  more  efficient  and  effective.  A checklist,  the  Criteria  Checklist  for  Andragogical  Principles 
(Lawton,  2001),  was  also  designed  to  assess  the  adherence  of  instructional  materials  to  the  principles  of  andragogy  (Caffarella, 
1993;  Cross,  1981;  Knowles  et  al.,  1998;  Pratt,  1993).  This  yes/no  checklist  asks  evaluators  to  assess  three  major  components  of 
instructional  design,  instructional  strategies,  design  of  instructional  materials,  and  design  of  the  physical  learning  environment. 
Evaluators  then  cite  evidence  and/or  concerns  for  the  application  of  each  principle  in  the  space  provided  on  the  checklist. 

Data  Collection 

Quantitative  data  for  this  study  were  collected  in  the  following  ways.  At  the  beginning  of  the  instruction,participants  for  both 
the  treatment  goup  and  the  comparison  group  were  given  the  Background  Demographic  Survey.  Learners  were  given  the 
Computer  Task  Frequency  Survey  (Lawton,  2001)  as  a pretest/posttest  to  determine  differences  in  computer  frequency  and  types 
of  tasks  performed  between  the  groups.  Although  there  were  differences  in  the  administration  of  the  pretest  (which  was  print)  and 
the  posttest  done  via  phone  eight  weeks  following  the  instruction,  the  administration  was  the  same  for  both  the  treatment  group 
and  the  comparison  group.  Participants  in  both  the  treatment  group  and  the  comparison  group  completed  the  Attitudes  Toward 
Computers  Questionnaire  (ATCQ;  Jay  & Willis,  1992).  This  33-item  multidimensional  measure  assesses  six  dimensions  of 
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attitudes  toward  computers:  comfort,  efficacy,  control,  dehumanization,  interest,  and  utility.  Each  of  these  dimensions  is  assessed 
by  five  or  six  items  on  a 5-point  Likert  scale  format.  A seventh  dimension,  gender  equality,  was  not  used  in  the  present  study. 
Response  options  range  from  "strongly  disagree"  to  "strongly  agree."  The  ATCQ  has  been  used  in  prior  research  with  older 
adults  (Czaja  & Sharit,  1998;  Jay  & Willis,  1992). 

Data  Analysis 

Data  gathered  through  the  Computer  Task  Frequency  Survey  and  the  Attitudes  Toward  Computers  Questionnaire  were 
analyzed  by  a one-way  analysis  of  variance.  The  purpose  of  the  analysis  was  to  determine  whether  or  not  there  were  significant 
differences  at  the  p < .05  level  in  attitudes  toward  computers  and  types  and  frequency  of  computer  tasks  performed  between  the 
treatment  and  comparison  groups. 

Significant  differences  were  found  between  the  treatment  group  and  the  comparison  group  with  the  treatment  group 
performing  higher  on  three  of  the  six  dimensions:  efficacy,  interest,  and  utility.  In  the  dimension  efficacy,  differences  were 
statistically  significant,  F (1,92)  = 6.204,  p.  = .015.  Analysis  of  the  interest  dimension  revealed  a statistically  significant 
difference,  F (1,92)  = 7.904,  p = .006.  The  dimension  utility  also  had  a statistically  significant  difference,  F (1,92)  = 5.534,  p.= 
.021.  Analysis  of  the  dimensions  of  comfort,  control,  and  dehumanization  did  not  reveal  a statistically  significant  difference 
between  the  two  groups. 

When  the  attitudes  of  the  participants  in  the  existing  groups  and  the  newly  formed  groups  (both  of  these  groups  received 
Elder  Computer  Instruction)  were  compared,  there  were  no  significant  differences  found.  A comparison  of  the  computer  attitude 
of  participants  receiving  Elder  Computer  Instruction  according  to  age— the  young-olds  (ages  55-64)  and  the  older-olds  (65+)— 
revealed  no  significant  differences  in  attitudes. 

Two  questions  measured  the  frequency  of  computer  use.  Participants  were  asked  how  many  times  they  had  used  a computer 
for  any  reason  and  how  many  hours  per  week  they  used  a computer.  While  each  group  increased  their  frequency  of  use  from  the 
pretest  to  the  posttest,  the  differences  between  the  two  instructional  groups  were  not  statistically  significant. 

The  other  nine  questions  measured  the  different  types  of  computer  tasks  participants  performed:  using  the  mouse,  changing 
the  speed  or  size  of  the  mouse  pointer,  moving  desktop  icons,  playing  solitaire,  saving  a document  to  a disk,  sending  e-mail, 
opening  e-mail,  sending  an  e-mail  attachment,  and  using  the  Internet. 

Analysis  of  the  e-mail  computer  task  revealed  a statistically  significant  difference,  F (1,92)  = 6.067,  p.  = .016,  between  the 
treatment  group  and  the  comparison  group.  Participants  in  the  treatment  group  sent  e-mail  more  frequently  than  did  those  in  the 
comparison  group.  There  was  no  statistically  significant  difference  between  the  two  groups  in  the  other  types  of  computer  tasks 
performed. 

Discussion 

Results  of  data  analysis  support  the  benefits  of  systematically  designed  computer  instruction  for  older  adults.  The  findings 
indicate  that  the  treatment  group  receiving  Elder  Computer  Instruction  demonstrated  more  positive  attitudes  on  every  attitude 
dimension  than  did  those  receiving  Traditional  Computer  Instruction.  In  three  of  the  six  attitude  dimensions,  there  were 
significant  differences  between  the  two  groups.  These  three  dimensions-efficacy,  interest,  and  utility— have  a direct  relationship 
to  the  theory  of  andragogy.  The  participants  receiving  Elder  Computer  Instruction  evidenced  a significantly  more  positive 
attitude  in  the  area  of  efficacy.  According  to  the  theory  of  andragogy,  adults  learn  best  when  they  are  motivated  to  learn.  They 
believe  that  they  can  learn  new  material  (Knowles,  et  al.,  1998).  Elder  Computer  Instruction  participants  also  evidenced 
significantly  more  positive  attitudes  in  the  area  of  interest.  Another  adult  learning  principle  (Knowles,  1990)  is  that  adults  are 
interested  in  learning  what  they  need  to  know  to  perform  a task.  Finally,  data  analysis  revealed  a significantly  more  positive 
attitude  in  the  dimension  utility  for  participants  receiving  Elder  Computer  Instruction.  The  core  principle  of  andragogy  is  that 
adults  need  to  know  why  they  should  learn  something  before  they  engage  in  learning  (Knowles  et  al.,  1998).  When  adults  realize 
how  computers  can  enrich  and  enliven  their  lives,  they  will  understand  the  usefulness  of  learning  computer  skills.  The 
participants  receiving  Elder  Computer  Instruction  appeared  to  have  a greater  appreciation  of  the  usefulness  of  computers. 

Conclusions 

The  distinguishing  feature  of  this  study  is  its  focus  on  the  importance  and  benefits  of  systematically  designing  computer 
instruction  for  older  adults.  The  statistically  significant  differences  found  in  attitudes  were  between  the  groups  receiving  different 
instruction.  Neither  group  membership  nor  age  appeared  to  have  an  effect  on  computer  attitudes.  The  type  of  instructional 
method  received  by  the  participants  appears  to  be  a more  important  factor  in  shaping  computer  attitudes.  It  was  observed  during 
the  instruction  that  members  of  newly  formed  groups  receiving  Elder  Computer  Instruction  evidenced  interaction  and 
collaboration  with  other  participants  similar  to  that  observed  in  the  existing  groups  who  were  previously  acquainted.  Many 
participants  who  were  not  previously  acquainted  worked  together  to  perform  computer  tasks.  Participants  who  learned  tasks 
quickly  often  checked  to  be  sure  that  slower  participants  kept  up  with  the  instruction.  During  the  e-mail  activity,  many  exchanged 
e-mail  addresses  and  e-mailed  each  other.  New  acquaintances  chatted  with  each  other  during  breaks  and  often  went  to  lunch 
together.  With  the  exception  of  two  married  couples,  participants  receiving  Traditional  Computer  Instruction  were  not  previously 
acquainted.  These  participants  also  worked  together  to  perform  computer  tasks,  helped  each  other  with  tasks,  and  became 
acquainted  with  each  other  during  the  instruction. 

Knights  (1993)  notes  that  learning  is  a social  practice,  and  group  members  often  a-e  concerned  for  each  other.  Findings 
suggest  that  older  adults  appear  to  be  more  aware  of  the  advantages  of  collaboration  and  are  often  more  willing  to  interact  with 
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others  for  assistance  or  support  in  learning  a task  than  are  younger  learners  (Dixon  & Gould,  1 996;  Kazemek,  1 997).  It  may  be 
that  when  older  adults  have  a shared  sense  of  purpose  such  as  learning  computer  skills,  previous  group  affiliation  becomes  less 
important.  Prior  research  also  indicates  that  older  adults  often  feel  at  a disadvantage  when  learning  computer  skills  in  a fast-paced 
class  with  younger  learners  (Dunnett,  1998;  Redding  et  al.,  1998;  Timmerman,  1998).  Both  Elder  Computer  Instruction  and 
Traditional  Computer  Instruction  were  limited  to  adults,  ages  55  and  above.  Adults  in  both  existing  groups  and  newly  formed 
groups  expressed  that  they  preferred  computer  classes  made  up  of  older  adults. 

This  study  describes  the  step  -by -step  process  used  to  design  instructional  materials  according  to  learning  theory  research  and 
practical  application.  The  research  also  provides  new  information  regarding  the  relationship  between  the  instructional  materials 
used  for  older  adults  and  their  resulting  attitudes  toward  computer  use.  When  older  adults  receive  computer  inkruction  that  is 
designed  uniquely  for  their  needs,  they  appear  to  develop  more  positive  attitudes  toward  computers. 

At  the  conclusion  of  one  of  the  workshops  in  which  the  participants  were  members  of  an  existing  group,  one  participant 
confessed  that  she  had  once  walked  out  of  a computer  workshop.  "The  instructor  gave  a detailed  lecture  on  the  inner  workings  of 
the  computer.  I went  to  the  workshop  to  learn  how  to  use  the  computer.  I really  did  not  care  how  it  worked,  so  I left.  I stayed  for 
this  entire  workshop  because  it  was  practical.  Now  I believe  that  I can  use  a computer." 

JoAnn,  another  participant,  came  to  the  workshop  to  learn  to  e-mail.  She  had  traveled  in  China  and  discovered  that  she  could 
have  ©.mailed  her  family  from  public  computers  at  a nominal  cost  had  she  known  how  to  set  up  a hotmail  account.  After  the 
workshop,  she  traveled  in  Europe.  She  reported  that  not  only  had  she  used  the  hotmail  account  that  she  learned  to  set  up  during 
the  class,  she  had  taken  her  computer  manual  with  her  and  taught  a fellow  traveler  how  to  use  e-mail. 

Monira,  originally  from  Lebanon,  owned  a computer  prior  to  the  workshop  and  regularly  exchanged  e-mail  with  her  family  in 
Lebanon.  However,  she  had  to  wait  for  a relative  or  friend  to  come  to  her  house  and  open  her  e-mail  so  that  she  could  read  it,  and 
then  send  her  responses  for  her.  At  the  workshop,  she  learned  to  e-mail  independently. 

Loretta,  another  participant,  told  about  her  experiences  in  teaching  English  to  a woman  who  had  recently  moved  to  the 
United  States  from  South  America.  In  addition  to  learning  English,  the  woman  wanted  to  learn  basic  computer  skills  in  order  to 
e-mail  her  family  in  South  America.  Loretta  gave  her  a copy  of  the  manual  used  in  the  workshop.  Because  it  was  written  clearly 
and  simply,  a person  with  limited  English  could  use  it  successfully. 

One  of  the  basic  principles  of  andragogy  is  that  adults  learn  best  those  things  that  they  must  know  in  order  to  perform  tasks 
that  are  relevant  to  them  (Knowles,  1990).  At  the  beginning  of  one  work  shop,  an  80-year-old  participant  confided  that  her 
grandchildren  had  asked,  “Grandma,  what’s  wrong  with  you?  You  can’t  e-mail.”  On  the  posttest  survey  eight  weeks  after  the 
instruction,  she  reported  that  she  was  now  e-mailing  her  grandchildren. 

Since  this  study  was  the  first  to  compare  systematically  designed  computer  instruction  for  older  adults  with  more  generic 
computer  instruction  commonly  used  with  older  adults,  it  provides  ample  opportunity  for  further  research.  There  are  still  many 
older  adults  who  need  access  to  training  facilities  (Galusha,  1998).  All  interested  older  adults  need  the  opportunity  to  explore  the 
world  afforded  them  by  technology -not  the  least  to  facilitate  those  e-mails  to  their  grandchildren. 
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Abstract 

Two  types  of  self-efficacy  were  investigated  in  this  study:  self-efficacy  for  course  content  and  self- efficacy  for  online  learning 
technologies.  Specifically,  we  examined  how  these  two  types  of  self-efficacy  change  throughout  a semester.  Secondly,  we 
examined  whether  students’  self-efficacy  is  predictive  of  their  satisfaction  and  course  performance.  Three  hypotheses  were  tested: 
(a)  self-efficacy  for  both  course  content  and  online  learning  technologies  change  across  a semester;  (b)  self  efficacy  is  predictive 
of  student  satisfaction  with  course;  (c)  self-efficacy  is  predictive  of  course  performance.  Participants  were  undergraduate 
students  who  enrolled  in  an  online  course  at  the  University  of  Central  Florida.  In  an  attempt  to  longitudinally  gauge  the  student’s 
continuing  self-efficacy,  a self  efficacy  survey  for  the  course  content  and  online  learning  technologies  was  administered  every 
three  weeks  four  times  across  a semester).  At  the  end  of  the  semester,  students’  perceived  degree  of  satisfaction  with  the  online 
course  was  measured  and  students’ final  course  scores  were  obtained  from  the  instructor.  Results  indicated  that  both  self-efficacy 
for  course  content  and  self-efficacy  for  online  technologies  increased  during  the  semester.  In  addition,  while  initial  self-efficacy 
for  course  content  was  a significant  predictor  (p<0.05)  of  students*  satisfaction  with  the  course,  neither  self  efficacy  with  course 
content  nor  self  efficacy  with  online  technologies  was  significant  predictors  of performance. 

Introduction 

The  development  of  telecommunications  technology  since  the  early  90 ’s  has  led  to  an  explosive  growth  in  the  World-Wide 
Web  (WWW).  According  to  the  Ipsos-Reid  (2001)  and  CyberAtlas’s  (2001)  reports,  as  many  as  400  million  people  are  already 
on  the  Internet,  and  nearly  one  billion  people  (about  15%  of  the  world's  population)  will  be  connected  to  the  Internet  by  2005. 
Along  with  the  astonishing  growth  of  Internet  usage,  an  increasing  number  of  educational  institutions  are  now  offering  their 
courses  via  the  Web,  namely  Web-based  (online)  education.  International  WHERE  + HOW  (2001)  lists  more  than  55,000  online 
courses  that  are  provided  by  higher  educational  institutions  and  training  corporations.  The  benefits  of  online  education  are  two- 
fold: individuals  are  given  an  opportunity  to  enhance  their  professional  development  and  expand  career  opportunities  while 
juggling  responsibilities  between  family  and  work,  and  online  education  provides  educational  institutions  a means  to  reach  a 
greater  student  population  resulting  in  an  increase  in  revenue. 

Although  online  education  is  gaining  tremendous  popularity,  it  is  accompanied  with  a serious  problem:  a high  attrition  rate. 
Students  enrolled  in  distance  courses  are  more  likely  to  drop  out  than  their  traditional  counterparts.  According  to  Moore  and 
Kearsley  (1996),  the  attrition  rate  in  distance  education  courses  is  between  30  and  50%,  which  is  much  higher  than  in  traditional 
course  settings.  This  results  in  some  negative  implications  for  students,  as  well  as  for  institutions.  For  students,  the  negative 
effects  of  dropout  include  loss  of  opportunity  for  personal  and  career  advancement,  lowered  self-esteem,  and  increased  likelihood 
of  future  disappointment  (Atman,  Egan,  Sebastian,  Welch,  & Page,  1991).  For  an  institution,  high  attrition  rate  results  in 
considerable  financial  loss  (Keegan,  1986). 

Background  of  the  Study 

A great  deal  of  research  has  been  conducted  to  investigate  factors  that  lead  to  student  attrition  (Baynton,  1992;  Coggins, 
1988;  Dille  & Mezack,  1991;  Eisenberg  & Dowsett,  1990;  Fjortoft,  1995;  Frew  & Weber,  1995;  Garland,  1993;  Keller,  1999* 
Miltiadou,  2000;  Parker,  1995;  Pugliese,  1994;  Zajkowski,  1993).  Among  those  factors  identified,  motivation  is  often  considered 
a strong  predictor  of  success  in  a distance  course  (Baynton,  1992;  Coggins,  1988;  Dille  & Mezack,  1991;  Fjortoft,  1995;  Garland, 
1993;  Keller,  1999;  Miltiadou,  2000;  Zajkowski,  1993).  Keller  (1999)  directly  attributes  attrition  to  a motivational  problem. 

While  learning  takes  place  at  a distance,  student  motivation  becomes  particularly  critical  because  distance  learning  places  the 
responsibility  of  learning  on  the  student  much  more  so  than  does  traditional  learning.  Cropley  and  Kahl  (1983)  stated: 

“distance  learners  are  thrown  back  upon  their  own  motivational  resources  to  a greater  extent  than  is  the  case  with  face-to- 

face  learners,  since  many  of  the  factors  which  provide  external  motivation  are  absent  or  present  only  in  an  indirect 

form  in  distance  education.  Internal  motivation  is  a highly  desirable  thing  in  face-to-face  education,  but  is  a necessary 

precondition  in  distance  education”  (p.32). 

In  numerous  studies  of  learning  motivation,  self-efficacy  has  been  identified  as  a significant  predictor  of  student  motivation 
(Bandura,  1997).  Specifically,  self-efficacy  is  predictive  of  academic  performance  and  course  satisfaction  in  traditional  face-to- 
face  classrooms  (Bandura,  1997)  and  online  courses  (Miltiadou,  2000;  Wang  & Newlin,  in  press).  Furthermore,  an  individuaFs 
self-efficacy  has  a significant  impact  on  his  or  her  (a)  actual  performance  (Locke,  Frederick,  Lee,  & Bobko,  1984;  Schunk, 
1981);  (b)  emotions  (Bandura,  Adams,  & Beyer,  1977;  Stumpf,  Brief,  & Hartman,  1987);  (c)  choices  of  behavior  (Betz  & 
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Hackett,  1981);  and  (d)  amount  of  effort  and  perseverance  expended  on  an  activity  (Brown  & Inouye,  1978).  Self-efficacy,  as 
defined  by  Bandura  (1997)  is,  “the  beliefs  in  one’s  capabilities  to  organize  and  execute  the  courses  of  action  required  to  produce 
given  attainments”  (p.  3).  According  to  Bandura  (1997),  four  major  sources  contribute  to  an  individual’s  self-efficacy:  enactive 
experiences,  vicarious  experiences,  verbal  persuasion,  and  physiological  indexes.  Bandura  (1982)  reviewed  a variety  of  self- 
efficacy  studies  in  various  settings,  and  found  that  self-efficacy  theory  has  significant  explanatory  potential.  He  found  that 
perceived  self-efficacy  accounts  for  a wide  variety  of  people’s  behaviors,  including  changes  in  coping  behavior  produced  by 
different  modes  of  influence,  levels  of  physiological  stress  reactions,  self-regulation,  achievement  strivings,  growth  of  intrinsic 
interest,  and  choice  of  career  pursuits.  Similarly,  Multon  et  al.  (1991),  in  a meta-analytic  review  of  thirty-nine  educational 
studies,  found  that  self-efficacy  beliefs  were  positively  related  to  student  persistence  and  academic  performance  across  a variety 
of  subject  areas,  experimental  designs,  and  grade-level.  Bandura  (1997)  also  found  that  self-efficacious  students  share  similar 
characteristics:  they  participate  more  readily,  work  harder,  persist  longer,  and  have  fewer  adverse  emotional  reactions  when  they 
encounter  difficulties  than  do  those  who  doubt  their  capabilities.  This  led  Bandura  to  conclude  that  it  is  one  thing  for  an 
individual  to  possess  the  necessary  knowledge  and  skills  to  perform  a task,  and  quite  another  to  embody  the  self-beliefs  in 
continuing  with  the  task  at  hand  while  facing  obstacles. 

In  light  of  the  research,  it  is  legitimate  to  infer  that,  in  order  to  be  motivated  and  successful  in  an  online  course,  students 
should  possess  high  self-efficacy  for  the  content  taught.  However,  if  examined  carefully,  having  adequate  self-efficacy  for  course 
content  might  not  be  enough  to  succeed  in  an  online  course.  Students  with  high  self-efficacy  for  course  content  might  not  feel 
confident  in  learning  in  the  online  educational  environment.  Participation  in  online  learning  requires  the  extensive  use  of  online 
technologies.  To  succeed  in  such  courses,  students  should  be  able  to  use  technologies  to  access  course  materials,  send  and  receive 
email,  browse  the  Internet,  and  perform  searches  to  locate  information.  Students  with  limited  or  inadequate  computer  experiences 
or  skills  do  not  feel  efficacious  enough  to  participate  in  online  learning  and  this  can  lead  to  computer  anxiety  (Loyd  & Gressard, 
1984).  While  students  are  experiencing  computer  anxiety,  more  effort  is  expended  learning  the  media  rather  than  the  subject 
matter  (Davie  8l  Wells,  1991).  Consequently,  if  an  individual  is  to  be  efficacious  about  learning  in  an  online  course,  he  or  she 
should  possess  two  types  of  self-efficacy.  One  is  self-efficacy  for  course  content  and  the  other  is  self-efficacy  for  online 
technologies.  Online  students  should  not  only  feel  efficacious  about  the  course  content,  but  they  should  also  feel  efficacious  in 
using  online  technologies. 

Currently  relatively  few  studies  have  been  conducted  to  investigate  self-efficacy  and  its  relationship  to  satisfaction  and 
performance  when  learning  takes  place  in  the  online  learning  environment  (Miltiadou,  2000;  Wang  & Newline,  in  press).  And, 
results  from  those  earlier  studies  concerning  the  effects  of  online  technologies  self-efficacy  are  inconclusive.  Online  technologies 
self-efficacy  was  negatively  related  to  student  performance  in  Miltiadou’ s (2000)  study,  whereas  it  was  positively  related  in 
Wang  and  Newline’s  study  (in  press).  In  addition,  those  studies  measured  self-efficacy  only  one  time  - at  the  beginning  of  a 
semester.  However,  self-efficacy,  especially  self-efficacy  with  online  technologies,  may  fluctuate  during  the  course  of  a semester. 
Previous  research  on  computer  self-efficacy  has  indicated  that  positive  past  experience  with  computers  increases  computer  self- 
efficacy  beliefs  while  negative  experience  with  computers  lowers  computer  self-efficacy  beliefs  (Ertmer,  Everbeck,  Cennamo,  & 
Lehman,  1994;  Hill,  Smith,  & Mann,  1987;  Torkzadeh  & Koufteros,  1994).  Similarly,  in  a web-based  course,  students’  success 
or  failure  mastering  online  technologies  contributes  to  the  fluctuation  of  their  efficacy  expectation.  Because  self-efficacy 
regarding  online  technologies  can  fluctuate  through  the  semester,  measuring  self-efficacy  one  time  can  reduce  the  accuracy  of 
predicting  the  impact  of  self-efficacy  on  student  learning. 

Purpose  of  the  Study 

In  an  effort  to  provide  an  in-depth  understanding  of  student’s  self-efficacy  and  its  effects  on  student  satisfaction  and 
performance,  this  study  examined  (1)  whether  students’  self-efficacy  regarding  course  content  and  online  technologies  change 
throughout  a semester;  (2)  whether  self-efficacy  for  course  content  and  online  technologies  are  predictive  of  student  satisfaction 
and  performance.  Specifically,  answers  to  the  following  questions  were  sought: 

1 . Does  a student’s  self-efficacy  for  course  content  change  across  a semester? 

2.  Does  a student’s  self-efficacy  for  online  technologies  change  across  a semester? 

3.  Does  self-efficacy  of  course  content  and  online  technologies  serve  as  a significant  predictor  of  student  satisfaction 
and  performance  within  a web-based  course? 

Methods 

Participants 

A total  of  sixteen  students  attending  the  University  of  Central  Florida  (UCF)  at  Orlando  participated  in  this  study.  These 
students  were  enrolled  in  an  undergraduate  course.  Introduction  to  Educational  Technology  (EME  2040).  This  course  was  offered 
through  WebCT  (WebCT  is  the  course  management  tool  used  at  the  University  of  Central  Florida). 

Instruments 

• Self-Efficacy  Instrument 

The  Self-Efficacy  Instrument  measures  two  components  of  self-efficacy  beliefs  - self-efficacy  for  course  content  and 
self-efficacy  for  online  technologies.  A total  of  27,  5-point  Likert-scaled  items  were  developed.  The  first  three  items 
measuring  course  content  self-efficacy  were  generated  based  on  Eccles  and  Wigfield’s  (1995)  7-point  Likert-scaled 
items.  The  last  24  items  measuring  online  technologies  self-efficacy  were  developed  based  on  Miltiadou  and  Yu’s  (in 
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press)  Online  Technologies  Self-efficacy  Scale  (OTSES).  Each  statement  is  preceded  by  the  phrase  “I  feel  confident...” 
For  each  item,  students  are  asked  to  indicate  their  attitude  from  “Strongly  Disagree”,  “Disagree”,  “Neural”,  “Agree”,  to 
“Strongly  Agree.” 

The  instrument  was  tested  by  a pilot  study  conducted  with  thirty -two  students  during  Spring  2001.  Reliability  andysis 
(Cronbach’s  coefficient  alpha)  showed  that  the  reliability  was  .87  for  the  first  three  items  measuring  content  self- 
efficacy  and  .90  for  the  rest  24  items  measuring  online  technologies  self-efficacy. 

• Student  Satisfaction  Instrument 

In  this  study,  student  satisfaction  with  the  course  was  measured  by  an  attitude  questionnaire  that  was  administered 
during  the  last  week  of  Summer  2001.  This  satisfaction  instrument  consists  of  19  items  measuring  students’  self-r 
reported  level  of  satisfaction  with  the  online  course.  Specifically,  the  attitudes  instrument  measured  students’  self- 
reported  level  of  satisfaction  with  the  course  materials,  instructor,  and  the  online  technologies.  For  each  item,  students 
were  asked  to  indicate  their  attitude  from  “Strongly  Disagree”,  “Disagree”,  “Neural”,  “Agree”,  to  “Strongly  Agree.”  In 
a pilot  study  conducted  with  thirty -two  students  during  Spring  2001,  reliability  analysis  (Cronbach’s  coefficient  alpha) 
indicated  that  the  reliability  was  .93  for  these  19  items  in  this  instrument. 

Procedures 

Participating  students  were  asked  to  take  an  online  survey  (Self-Efficacy  Instrument)  at  four  intervals  during  the  course  of  the 
Summer  2001  semester.  Every  three  weeks,  students  were  asked  to  fill  out  an  online  survey  measuring  their  course  content  self- 
efficacy  and  online  technologies  self-efficacy.  Along  with  the  fourth  survey  (the  last  survey),  a satisfaction  survey  (Student 
Satisfaction  Instrument)  was  administered  to  measure  student’s  degree  of  satisfaction  with  this  online  course.  In  addition,  with 
students’  permission,  their  final  course  scores  were  obtained  from  their  course  instructor. 

Two  statistical  analyses  were  employed  in  this  study.  First,  a doubly  multivariate  repeated  measures  analysis  of  variance  was 
used  to  examine  whether  self-efficacy  for  both  course  content  and  online  learning  technologies  changed  across  a semester. 
Second,  multiple  linear  regression  was  used  to  determine  if  course  content  self-efficacy  and  online  technologies  self-efficacy 
could  predict  satisfaction  and  performance. 

Results  and  Discussion 

The  results  of  this  study  showed  that  there  is  a statistically  significant  change  (Fg,  io  = 4.4,  p = .02)  in  self-efficacy  for  course 
content  and  self-efficacy  for  online  technologies  during  the  semester.  Almost  73%  (72.5%)  of  the  change  in  combined  self- 
efficacy  can  be  accounted  for  by  the  time  interval.  Both  self-efficacy  for  online  technologies  (F3^  45  = 7.72,  p<  .01)  and  self- 
efficacy  for  course  content  (F3  45=  5.06,  p<  .01)  changed  over  time.  Within  subjects  contrasts  showed  that  self-efficacy  for  online 
technologies  increased  significantly  (Fi,  15  =5.64,  p=  .03)  during  the  first  three  weeks  of  the  semester  (between  time  1 and  time  2). 
Although  the  respondents’  confidence  level  with  online  technologies  increased  after  the  second  time,  neither  the  increase  from 
the  second  time  to  the  third  time  nor  the  increase  from  the  third  time  to  the  fourth  time  was  statistically  significant  (p<0.05).  On 
the  other  hand,  within  subjects  contrasts  using  self-efficacy  for  course  content  showed  a non-significant  decrease  (p>.05)  from 
time  1 to  time  2,  a statistically  significant  increase  from  time  2 to  time  3 (Fj^  15=  5.4,  p=  .04)  and  a statistically  significant  increase 
from  time  3 to  time  4 (Fi^  15  = 6.67,  p=  .02).  These  results  are  displayed  in  Figures  1 and  2. 

Figure  1 Figure  2 

Self-Efficacy  for  Online  Technologies  Self-Efficacy  for  Course  Content 
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The  results  suggest  that  self-efficacy  for  both  course  content  and  online  technologies  is  dynamic,  indicating  that  self-efficacy 
is  subject  to  change,  even  within  a relatively  short  period  of  time  (i.e.  three  weeks).  A considerable  change  found  in  this  study 
was  the  significant  increase  in  student’s  level  of  self-efficacy  for  online  technologies  during  the  first  three  weeks  of  the  semester. 
Commonsense  dictates  that,  when  students  have  more  experience  with  online  technologies,  they  feel  more  confident  in  using 
online  technologies,  and  their  self-efficacy  increases,  accordingly. 

In  predicting  student’s  satisfaction  with  a course  using  the  initial  self-efficacy  measurements,  only  course  content  self- 
efficacy  was  a statistically  significant  predictor  (t=2.77,  p<.02).  This  predictor  alone  accounted  for  32%  of  the  variance  in 
satisfaction.  When,  however,  a composite  of  self-efficacy  for  course  content  and  self-efficacy  for  online  technologies  was  used, 
the  initial  composite  was  a statistically  significant  (R=  .71,  F2,  13=  6.7,  p=  .01)  predictor  of  student  satisfaction.  The  resulting 
equation.  Satisfaction  — -2.89  + .4  (Online  Tech)  4-  3.49  (Course  Content),  indicated  that  as  self-efficacy  increased  for  either 
online  technologies  or  course  content  there  was  a resulting  increase  in  satisfaction.  Although  initial  self-efficacy  for  online 
technologies  was  not  a significant  predictor  of  satisfaction,  more  than  50%  of  the  variance  in  final  course  satisfaction  could  be 
explained  by  the  initial  composite  of  online  technologies  and  course  content  self-efficacy.  By  the  third  time  period,  self-efficacy 
for  online  technologies  was  a statistically  significant  predictor  of  satisfaction  but  self-efficacy  for  course  content  was  not  (R=  .74, 
F2, 13=  7.9,  p<  .01).  Overall,  these  results  indicated  that  when  these  two  types  of  self-efficacy  were  compounded,  the  possibility  of 
predicting  student  satisfaction  was  increased.  Consequently,  we  concluded  that  these  two  types  of  self-efficacy  play  substantial 
roles  in  predicting  satisfaction. 

Finally,  the  findings  of  this  study  showed  that  neither  self-efficacy  for  course  content  nor  self-efficacy  for  online  technologies 
were  statistically  significant  predictors  of  student  performance  until  the  fourth  (last)  time  period.  The  resulting  equation. 
Performance  = 114.4  - .48  (Online  Tech)  + 2.99  (Course  Content),  indicated  that  as  self-efficacy  for  online  technologies 
increased,  performance  decreased;  and  as  self-efficacy  for  course  content  increased,  performance  increased.  The  linear  composite 
of  self-efficacy  for  course  content  and  for  online  technologies  explained  40.1%  of  the  variance  in  performance  (R=  63  F2  13= 
4.4,  p<  .05).  ’ ’ 

Contrary  to  previous  studies  (Miltiadou,  2001;  Wang  & Newlin,  in  press),  this  study  showed  that  initial  self-efficacy  for 
course  content  and  online  technologies  was  not  a statistically  significant  predictor  of  student  performance.  The  absence  of  a 
relationship  between  initial  self-efficacy  and  performance  might  be  due  to  the  small  sample  size.  However,  in  the  last  time 
period,  a relationship  between  self-efficacy  for  online  technologies  and  performance  did  appear.  It  is  noticeable  that  the 
relationship  was  negative,  indicating  that  students  who  were  not  efficacious  with  online  technologies  perform  better  than  those 
who  were  efficacious.  This  finding  corresponds  with  Miltiadou’s  (2001)  study.  A possible  explanation  for  this  phenomenon  is 
that  when  online  technologies  were  perceived  as  difficult  and  students  were  not  confident  in  learning  via  this  media,  they  were 
more  likely  to  be  cognitively  engaged.  However,  when  online  technologies  were  perceived  as  easy,  students  seemed  to  expend 
less  effort.  Consequently,  those  who  were  efficacious  with  online  technologies  earned  lower  grades  than  those  who  were  not 
efficacious.  This  phenomenon  also  appeared  in  Salomon’s  (1984)  study,  where  the  students  considered  learning  from  watching 
television  easier  than  learning  from  reaching  printed  text.  These  students  exerted  less  effort,  resulting  in  poor  performance. 

Although  self-efficacy  for  online  technologies  was  negatively  related  to  performance,  a positive  relationship  was  found 
between  self-efficacy  for  course  content  and  performance.  This  was  consistent  with  research  findings  from  previous  studies 
conducted  in  traditional  classrooms  as  well  as  online  courses  where  self-efficacy  was  positively  related  to  student  achievement 
(Bandura,  1997;  Locke,  Frederick,  Lee,  & Bobko,  1984;  Miltiadou,  2000;  Schunk,  1981;  Nicholls  & Miller,  1994;  Pajares  & 
Kranzler,  1995;  Wang  & Newlin,  in  press). 

Conclusions  and  Suggestions  for  Further  Research 

With  the  growing  popularity  of  online  education  and  the  urgent  need  to  curb  the  online  attrition  rate,  understanding  and 
fostering  online  students’  motivation  is  imperative.  This  study  investigated  self-efficacy,  a critical  element  of  motivation,  and 
identified  its  effects  on  satisfaction  and  performance.  Although  the  small  sample  size  of  this  study  limits  its  general izability  to  the 
larger  population,  the  findings  of  this  study  revealed  some  important  points:  (a)  self-efficacy,  both  for  course  content  and  online 
technologies,  changed  over  time  in  a web-based  course,  (b)  the  initial  composite  of  self-efficacy  for  course  content  and  online 
technologies  was  identified  as  a significant  predictor  of  satisfaction,  (c)  the  final  measure  of  self-efficacy  with  online 
technologies  was  identified  as  a significant  predictor  of  performance  (with  a negative  coefficient),  and  (d)  the  final  measure  of 
self-efficacy  with  course  content  was  identified  as  a significant  predictor  of  performance. 

Given  the  findings  of  this  study,  there  appear  to  be  several  implications  for  researchers  and  instructors  in  the  field  of  online 
education.  First,  this  study  showed  that  self-efficacy  is  dynamic  and  changeable  within  the  course  of  a semester.  However,  in 
previous  studies  (Miltiadou,  2000;  Wang  & Newlin,  in  press),  the  dynamic  nature  of  self-efficacy  was  often  overlooked  and  was 
measured  only  one  time  in  order  to  predict  satisfaction  and  performance.  This  approach  does  not  provide  a comprehensive  picture 
of  self-efficacy  and,  accordingly,  reduces  the  researcher’s  ability  to  explain  the  impact  of  self-efficacy  on  learning.  In  light  of 
this,  a genuine  examination  of  the  effect  of  self-efficacy  on  learning  requires  repeated  measures  of  self-efficacy.  When  multiple 
measurement  of  self-efficacy  is  not  feasible,  it  is  imperative  to  specify  the  point  in  the  semester  when  self-efficacy  is  measured. 

Second,  in  past  studies  on  self-efficacy,  attention  was  predominately  paid  to  efficacy  expectation  regarding  the  content 
learned,  and  yet  participants’  efficacy  with  learning  tools  or  instruction  delivery  systems  was  often  neglected.  Nevertheless,  in  a 
web-based  learning  environment,  participants’  efficacy  expectation  with  online  technologies  can  no  longer  be  ignored.  As  shown 
in  this  study,  while  predicting  satisfaction  and  performance  in  a web-based  course,  participants’  self-efficacy  with  online 
technologies  was  as  critical  as  self-efficacy  with  course  content.  More  attention  should  be  paid  to  students’  efficacy  expectations 
while  teaching  or  designing  a web-based  course. 
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Third,  this  study  showed  an  unexpected  fnding  regarding  the  negative  relationship  between  self-efficacy  with  online 
technologies  and  performance.  Students  who  were  confident  with  online  technologies  appeared  to  erroneously  overestimate  their 
abilities  to  deal  with  learning  tasks,  and  thus  exert  less  mental  effort,  which  led  to  poor  performance.  To  avoid  the  pitfall  of  faulty 
assumptions,  students  should  be  informed  that  no  matter  how  proficient  they  are  with  the  online  technologies,  participating  in 
online  learning  requires  no  less  effort  than  traditional  classes. 

The  limitation  of  this  study  comes  from  its  small  sample  size.  In  order  to  generalize  the  findings  of  this  study,  it  is 
recommended  that  this  study  be  replicated  with  a larger  sample  and  with  different  types  of  classes  in  different  academic  settings 
(i.e.  high  schools,  community  colleges,  and  universities).  In  addition,  the  negative  relationship  between  self-efficacy  with  online 
technologies  and  performance  should  be  further  explored.  It  is  suggested  that  further  studies  look  at  whether  students  who  are 
confident  with  online  technologies  do  exert  less  mental  effort  in  their  online  learning  activities.  This  could  unveil  the  mechanism 
that  contributes  to  the  negative  relationship  between  online  technologies  self-efficacy  and  performance.  Other  studies  could  look 
at  other  possible  predictors  of  student  satisfaction  and  performance,  such  as  task  value,  which  has  been  identified  as  a significant 
predictor  of  both  student  learning  achievement  and  satisfaction  in  traditional  classes  (Hammann  & Stevens,  1998;  Townsend  & 
Hicks,  1995;  Velayo  & McKeachie,  1994).  These  findings  could  shed  additional  light  into  the  prediction  of  student  achievement 
and  satisfaction  in  web-based  courses. 
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Introduction 

This  investigation  examined  whether  or  not  the  professional  trainers  could  benefit  from  a needs  analysis  course  in  which  real 
cases  from  various  organizations  were  used  and  all  the  learning  activities  and  materials  were  geared  closely  to  the  learning 
expectations  and  preferences  of  adults.  Specifically,  this  study  attempted  to  find  out  whether  or  not  the  use  of  five  prominent 
adult  learning  preferences  could  enable  the  trainers  to  improve  their  (a)  post-test  achievement,  (b)  attitudes  toward  these 
preferences,  and  (c)  performance  in  conducting  individual  needs  analysis  project.  These  preferences,  which  will  be  detailed  in 
the  methodology  section,  were  derived  from  related  literature  on  adult  learning  and  included  well-written,  well-organized  texts, 
lectures  and  handouts,  and  well-planned  class  discussions,  reflections,  and  case  studies. 

Research  questions  of  the  study  included:  (a)  were  the  use  of  the  five  adult  learning  preferences  effective  in  helping  the 
trainers  comprehend  and  recall  the  needs  analysis  knowledge  and  therefore,  perform  better  on  the  post-test?  (b)  were  the  use  of 
the  five  adult  learning  preferences  effective  in  helping  the  trainers  apply  the  needs  analysis  knowledge  and  skills  and  therefore, 
be  able  to  implement  the  individual  needs  analysis  project  correctly  and  effectively?  and  (c)  what  were  the  trainers’  attitudes 
toward  each  of  the  learning  preferences?  In  the  following  sections,  related  literature  on  adult  learning  characteristics  and 
preferences  is  reviewed,  which  serves  to  provide  a theoretical  context  for  the  study  and  to  justify  the  rationale  and  the 
significance  of  the  study.  Then,  the  methodology  of  the  study  is  detailed,  and  finally,  the  findings  of  the  study,  together  with  the 
implications  of  the  findings,  are  discussed. 

Adult  Learning  Characteristics  and  Preferences 

Most  literature  on  adult  learning  is  theoretical,  and  theorists  of  adult  learning  believe  that  “forms  of  reasoning,  thinking,  and 
judging”  of  adults  are  qualitatively  different  from  those  characteristics  of  adolescents  and  children.  This  means,  unlike 
adolescents  and  children,  adults  have  more  Bfe  experiences  and  are  capable  of  building  up  a kind  of  situational  reasoning  to 
interpret  their  experiences  and  guide  their  actions  during  learning. 

In  providing  a more  comprehensive  view  concerning  adult  learning,  Knowles  (1980)  pointed  out  that,  adulthood  should  be 
defined  as,  when  an  individual  is  essentially  responsible  for  his  or  her  own  life  and  is  performing  some  types  of  acceptable  social 
roles.  Accordingly,  adults,  while  assuming  the  role  of  learners,  are  different  from  child  learners  and  possess  distinct  learning 
characteristics,  which  are  qualities  or  traits  of  a person’s  learning.  These  distinct  learning  characteristics  of  adults  include:  (a) 
experience — ^adults  often  have  rich  life  experience  and  are  eager  to  find  connections  between  new  information  and  their 
experiences;  (b)  self-direction — adults  enjoy  directing  their  own  learning  and  prefer  to  have  freedom  to  choose  their  learning 
experiences  based  on  their  interests  and/or  needs;  (c)  readiness  to  learn — adults  learn  better  when  they  are  ready  to  or  need  to 
learn,  and  anticipate  that  the  learning  experience  will  match  their  expectations;  (d)  orientation  to  learning — adults  prefer  life- 
centered  or  task-centered  type  of  learning  versus  subject-centered  courses  and  want  to  apply  what  they  learn  in  the  classroom  to 
real-life  situations.  Knowles  (1984)  also  believed  that  adults  are  more  motivated  to  learn  by  internal  factors,  such  as  increased 
self-esteem  and  confidence,  than  by  external  rewards  like  pay  raises  and  job  promotions. 

In  the  past  two  decades,  Knowles  ‘s  theory  has  been  widely  applied  by  other  practitioners  to  discuss  issues  related  to  the 
learning  preferences  of  adults.  Learning  preferences  refer  to  the  selection  or  choice  of  certain  learning  activities,  situations  or 
climates  of  an  adult  learner  (Loesch  & Foley,  1988).  For  example,  Rosemary  and  Caffarella  (1994)  elaborated  Knowles’  theory 
and  explained  that  adults  not  only  have  the  need  to  examine  and  reflect  on  their  past  experiences  and  prior  knowledge,  but  they 
can  call  upon  these  experiences  and  knowledge  in  formulating  learning  activities  and  in  using  them  as  learning  resources.  They 
believed  that  experiential  learning  activities,  such  as  reflective  journals,  critical  incidents,  and  portfolio  development  could  be 
used  to  provide  opportunities  to  help  adults  integrate  their  past  and  current  experiences  into  the  learning  events.  Rosemary  and 
Caffarella  further  pointed  out  that,  additional  experiential  activities,  including  field-based  learning,  small  and  large  group 
discussion,  role  play,  storytelling,  metaphor  analysis,  case  study  analysis,  and  simulation  are  all  effective  in  encouraging  adults  to 
engage  in  learning  and  communication  with  peers. 

Other  practitioners,  including  Charlton  (1995),  Collins  (1999),  Cross  (1981),  Dinmore  (1997),  Ference  and  Vockell  (1994), 
Johnson  (1995),  Slusarski  (1994),  and  Zemke  and  Zemke  (1995),  all  provided  suggestions  for  the  use  of  various  adult  learning 
preferences  to  enhance  learning.  For  instance,  while  presenting  new  information  to  adults.  Cross  (1981)  suggested  that:  (a)  one 
idea  should  be  presented  at  a time  to  avoid  overburdening  the  short-term  memory  of  adults;  (b)  new  information  should  be 
presented  in  an  organized  way  that  allows  for  mastery  of  the  information  and  for  the  creation  of  relations  between  the  new  and 
previously  learned  information;  and  (c)  frequent  review  and  summarization  should  be  made  available  to  adults  for  assisting  in 
retention  and  recall. 
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For  enhancing  abstract  conceptualization  among  adults,  Johnson  (1995)  recommended  the  use  of  a well-thought-out,  well- 
presented  lecture  followed  by  a focused  question  and  answer  section.  As  for  the  design  of  learning  situations,  Collins  (1999) 
suggested  that  the  provision  of  structure  and  direction  at  the  beginning  of  new  activities  are  helpful  to  give  adults  a clear  sense  of 
what  to  expect  from  the  learning  journey.  Charlton  (1995)  indicated  that  offering  adults  an  interactive,  performance-centered  type 
of  learning  environment  would  be  effective  in  helping  adults  integrate  the  new  learning  with  their  prior  knowledge.  Dinmore 
(1997)  also  pointed  out  that  interdisciplinary  courses  that  allow  for  the  integration  of  knowledge  derived  in  formal  and  informal 
environments  are  beneficial  to  adults.  Ference  and  Vockell  (1994)  emphasized  that  adults  like  to  take  charge  of  their  learning 
and  are  more  self-reliant  (1994,  p.  25).  They  stated,  “adult  learners  have  often  acquired  their  most  successful  skills  through 
concrete,  hands-on  experience.  They  often  prefer  to  continue  this  practice  of  learning  by  doing  rather  than  by  listening”. 

As  for  increasing  adults’  learning  motivation,  Zemke  and  Zemke  (1995)  suggested  to:  (a)  make  the  content  appeal  to 
personal  growth  and  gain;  (b)  describe  the  immediate  and  long-term  relevance  of  the  content  to  their  lives;  (c)  stimulate  curiosity 
about  the  subject  matter;  (d)  ensure  low  risk  for  learner;  and  (e)  explore  the  learners’  positive  and  negative  expectations.  Finally, 
to  promote  adult’s  self-directing  abilities,  Slusarski  (1994)  agreed  with  other  practitioners  (Brockett  & Hiemstra,  1991;  Hiemstra 
& Sisco,  1990;  Knowles,  1980,  1990;  O’Donnell  & Caffarella,  1990)  that  learning  contracts  can  be  effective  in  promoting  self- 
direction  among  adults.  Learning  contracts  that  permit  learners  to  “ indicate  what  they  will  learn,  how  they  will  learn  it,  and  how 
the  learning  will  be  evaluated”  allow  adult  learners  to  plan  subsequent  learning  activities  in  a more  systematic  way  and  therefore, 
grant  them  more  control  over  their  own  learning. 

In  summary,  theorists  of  adult  learning  believe  that  adults  have  distinct  learning  characteristics  and  preferences  due  to  their 
rich  life  experiences.  Adults  learn  when  they  are  ready  to  learn  and  take  pleasure  in  having  self-direction  during  learning.  They 
tend  to  build  the  newly  learned  information  on  their  past  experiences  and  prior  knowledge.  They  are  motivated  by  being  able  to 
see  the  relevance  of  the  learning  content  and  desire  to  link  the  new  learning  experience  with  their  personal  growth  and  gain. 
They  want  to  have  flexibility  in  choosing  learning  activities  that  are  suitable  for  their  learning  needs  and  expectations.  Finally, 
they  prefer  learning  in  a progressive  manner  and  favor  learning  activities  that  are  realistic,  well-organized,  and  interactive. 

Method 

Research  Design  and  the  Participants 

This  study  used  the  pre-test-post-test  research  design  involving  53  trainers  from  an  instructional  analysis  course  at  a north- 
eastern state  university  of  the  US.  Among  the  53  trainers,  17  were  from  the  section  of  Fall  II,  1998,  21  from  Spring  I,  1999  and 
15  from  Spring  II,  1999.  These  trainers  were  all  working  full-time  in  local  corporations,  businesses,  and  educational 
organizations  and  were  all  enrolled  as  part-time  graduate  students.  From  the  pre-assessment  questionnaire,  it  was  revealed  that 
there  were  41  females  and  12  males  with  ages  ranging  from  23  to  50.  Most  of  the  participants  had  at  least  one  to  three  years  of 
training  experience  and  had  taken  a course  on  the  use  of  symptomatic  models  for  training  design  and  development. 

Three  major  steps  were  included  in  the  study.  At  the  beginning  of  the  class,  in  each  section,  the  participants  were  given  a 
pre-assessment  questionnaire  to  gather  data  on  their  demographic  characteristics,  background  in  training  and  needs  analysis,  and 
their  expectations  in  taking  the  course.  A pre-test  was  then  administered  to  measure  their  prior  knowledge  of  needs  analysis. 
After  the  pre-test,  the  participants  then  engaged  in  learning  the  knowledge  and  skills  of  needs  analysis  by  using  the  five 
prominent  learning  preferences  of  adults.  Throughout  the  learning  process,  the  participants  were  expected  to  apply  the  content 
being  studied  to  an  individual  needs  analysis  project.  Successful  completion  of  the  project  was  required  for  passing  the  needs 
analysis  segment.  Upon  completion  of  the  project,  the  participants  were  asked  to  take  the  post-test  and  the  attitudinal 
questionnaire  to  measure  their  post-test  performance  and  attitude  toward  the  learning  preferences.  The  participants  were  not 
informed  about  the  tests  and  the  attitude  questionnaire  before  taking  them. 

Learning  Content  and  the  Learning  Preferences 

The  learning  content  included  four  human  performance  theories  and  models  and  five  different  data  collection  and  analysis 
methods  for  needs  analysis  purposes.  These  theories  and  models  included:  (a)  Allison  Rossett’s  (1987)  theory  that  seeks 
information  to  bridge  the  gap  between  the  optimal  and  the  actual  performance,  to  examine  feelings  of  performers  or  significant 
others,  to  identify  causes  of  problems,  and  to  propose  solutions  to  the  problem;  (b)  Robert  Mager  and  Peter  Pipe’s  (1970)  Human 
Performance  Model  to  examine  performance  discrepancies,  skill  deficiencies,  and  performance  punishment  issues;  (c)  Ron 
Zemke  and  Thomas  Kramlinger’s  (1982)  model  to  inspect  the  major  human  and  organizational  factors  that  affect  people’s 
performance  in  an  organization;  and  (d)  Thomas  Gilber’s  (1978)  theory  and  formula  to  calculate  the  ratio  of  exemplary 
performance  to  typical  performance  and  to  determine  the  potential  for  improving  performance.  The  data  collection  and  analysis 
methods  included  the  design  and  conduction  of  interviews,  focus  groups,  observations,  questionnaires,  and  critical  incident  and 
document  reviews. 

The  five  adult  learning  preferences  used  were:  first,  an  easy-to-follow  textbook  geared  specifically  toward  needs  and  task 
analysis  knowledge  and  skills,  titled,  “Figuring  Things  Out:  A Trainer’s  Guide  to  Needs  and  Task  Analysis”  by  Ron  Zemke  and 
Thomans  Kramlinger  (1982).  Second,  multiple  sets  of  well-organized  handouts  that  corresponded  to  the  text  and  highlighted 
critical  points  of  the  learning  contents.  Both  the  text  and  handouts  provided  the  participants  with  an  organized  means  to  learn  the 
new  and  unfamiliar  theories  and  skills  of  needs  analysis.  They  also  set  directions  for  the  learning  sequences  and  allowed  the 
participants  to  conduct  frequent  previews  and  reviews  of  the  learning  content. 

Third,  well  thought-out  and  well-paced  lectures  were  used  to  facilitate  participants’  conceptual  understanding  of  the 
aforementioned  theories  and  data  collection  methods.  Also  covered  by  lectures  were  real  examples  of  needs  analysis  studies 
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conducted  in  various  organizations  and  how  the  knowledge  and  skills  learned  in  the  course  could  be  applied  both  personally  and 
professionally.  Fourth,  following  lectures,  class  discussions  were  encouraged  for  the  participants  to  ask  pertinent  questions,  to 
share  ideas  with  peers,  and  to  stimulate  critical  reflections  on  the  learning  content.  During  discussions,  all  the  participants  were 
ensured  a safe  climate  in  which  to  exchange  constructive  information  and  prior  experiences  in  the  areas  of  training  design,  and 
needs  analysis. 

Fifth,  a training  performance  related  case  in  a business  organization  was  used  for  the  participants  to  analyze  by  applying  the 
newly  learned  theories.  The  participants  in  each  section  were  divided  into  four  groups  and  each  group  was  provided  with 
detailed  information  about  the  case.  Then,  each  group  was  assigned  to  use  one  of  the  four  aforementioned  theories  to  analyze  the 
needs  of  the  case.  Upon  completing  the  needs  analysis  for  the  case,  each  group  had  to  present  the  analysis  process,  the 
conclusion,  and  the  solutions  for  the  situation.  Each  group  was  encouraged  to  provide  reflections  on  the  practice.  This  practice 
granted  the  participants  to  have  an  experiential  activity  that  required  the  application  of  the  newly  learned  theory  to  analyze 
training  and  performance  issues  in  a real  situation.  It  also  permitted  the  participants  to  work  in  a group  setting  that  had  a specific 
goal  to  reach,  was  task-based,  and  needed  to  be  interactive  in  order  to  complete  the  assignment. 

More  cases  were  created  for  the  participants  to  have  concrete,  hands-on  experience  to  collect,  analyze  and  interpret  the 
meanings  of  relevant  data.  Specifically,  in  this  activity,  the  participants  of  each  section  were  divided  into  two  groups  to  analyze 
if  there  were  performance  related  gaps  or  problems  in  the  library  and  the  computer  lab  of  the  university  that  they  currently 
attended.  Each  group  acted  as  if  it  were  a focus  group  discussing  its  experiences  in  using  the  services  provided  by  either  the 
library  or  the  computer  lab.  The  discussion  evolved  around  (a)  if  there  was  a gap  between  the  service  they  received  and  the 
optimal  service  they  would  like  to  have  had;  and  (b)  if  there  was  a gap,  what  caused  it?  Based  on  the  results  of  the  discussion, 
each  group  then  designed  a brief  observational  tool  for  conducting  needs  analysis  in  the  library  or  the  computer  lab  and 
developed  a brief  interviewing  agenda  with  which  to  interview  the  staff  of  the  library  or  the  computer  lab  to  gather  more 
information.  It  was  suggested  that  the  groups  try  to  create  a situation  or  situations  that  would  allow  the  group  to  observe  the 
service  provided  by  the  library  or  the  computer  lab  personnel.  Based  on  the  observational  tool  and  interviewing  agenda,  each 
group  gathered  the  needs  analysis  data.  Finally,  each  group  had  to  analyze  the  data  collected  from  both  the  observation  tool  and 
interview,  and  recommend  a solution  based  on  the  findings. 

Research  Materials 

Research  materials  included  a pre-test,  a post-test,  and  an  attitudinal  questionnaire.  All  the  research  materials  were 
developed  by  the  researcher  of  the  study  and  further  reviewed  by  a colleague  of  hers,  who  had  a Ph.D.  in  Instructional  Design 
and  Technology  and  had  more  than  ten  years  of  experience  in  conducting  adult  training.  The  pre-test  contained  eight  open-ended 
questions,  corresponding  to  the  eight  learning  objectives  for  the  needs  analysis  section.  The  wording  was  changed  only  slightly 
to  present  the  objective  in  question  format.  Four  of  the  questions  covered  the  four  aforementioned  theories  for  needs  analysis, 
and  the  other  four  included  data  collection  methods.  The  post-test  was  identical  to  the  pre-test  except  that  the  questions  were 
presented  in  a different  order. 

The  attitudinal  questionnaire  was  designed  to  gather  the  students*  perception  of  how  helpful  the  used  learning  preferences 
were  in  learning  and  applying  the  course  content,  as  well  as,  how  much  they  enjoyed  them.  The  participants  were  asked  to  rank 
them  one  through  seven,  with  one  being  of  the  least  assistance  or  least  liked  and  seven  being  of  the  greatest  assistance  or  the  most 
liked.  To  avoid  bias  in  the  participants*  responses  and  obtain  a clearer  picture  of  the  participants*  attitudes,  the  questionnaire  also 
included  questions  on  all  activities  used  in  the  course  including  the  text,  lectures,  handouts,  class  discussions  and  reflections, 
group -based  case  studies  and  presentations,  final  project,  and  a combination  of  all  these  activities. 

In  addition,  the  participants  were  asked  about  their  perception  regarding  which  activity  was  most  crucial  in  helping  them 
achieve  certain  learning  outcomes.  Sp  ecifically,  these  outcomes  were:  to  recall  the  learning  content,  to  explain  the  content  to 
someone  else,  to  analyze  the  content,  to  apply  the  content,  to  synthesize  the  content  in  a meaningful  way,  and  to  evaluate  the 
value  and  usefulness  of  the  content. 

Data  Collection  and  Analysis 

The  pre-and  post-tests  were  scored  using  a predetermined  answer  key.  A set  number  of  points  were  assigned  to  each 
question  based  on  the  scope  and  difficulty  of  the  question  with  the  total  possible  points  equal  to  1 00.  Answers  were  broken  down 
into  sections,  and  partial  credit  was  given.  A paired  t-test  was  performed  to  determine  if  the  results  of  the  two  tests  were 
significantly  different  from  each  other. 

For  the  needs  analysis  project,  it  is  necessary  that  a needs  analysis  be  based  on  a real  performance  or  training  related  issue  of 
an  organization.  Written  guidelines  detailing  the  purpose,  procedure  and  expected  outcomes  of  the  final  project  were  made 
available  to  the  participants  prior  to  the  implementation  of  the  project.  There  were  five  criteria  used  to  evaluate  each  project  and 
20  points  were  assigned  to  each  criteria.  These  criteria  were;  (a)  the  analyzed  situation  of  an  organization  was  clearly  and 
completely  described;  (b)  an  attempt  was  made  to  analyze  the  situation  by  applying  appropriate  theory(ies);  (c)  a hypothesis  was 
formed  and  effective  data  collection  method(s)  was  (were)  used  to  test  the  hypothesis;  (d)  an  objective  or  scientific  reporting  of 
the  collected  data  was  included  and  data  was  accurately  analyzed  and  interpreted;  and  (e)  effective  solutions/recommendations  to 
bridge  the  performance  gap  or  solve  the  problem  in  the  situation  was  proposed. 

Two  analyses  of  variance,  ANOVA,  were  first  performed  for  ranking  (a)  how  much  the  activities  helped  the  participants  in 
learning  the  content,  and  (b)  how  much  the  participants  liked  the  activities.  Once  a significant  F ratio  was  revealed  by  an 
ANOVA,  the  Tukey*s  test,  with  the  total  error  rate  of  a=0.05,  was  performed  to  determine  which  ranking  means  differed 
significantly  from  one  another.  Percentages  of  responses  concerning  how  the  learning  preferences  helped  in  reaching  each  level 
of  learning  (recall,  comprehension,  application,  analysis,  synthesis,  and  evaluation)  were  also  calculated. 
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Results  I 

The  Pre-  and  Post-tests 

A paired  t-test  was  first  performed  to  determine  if  there  was  a significant  difference  between  the  pre-test  and  post -test 
performance.  The  results  revealed  that  the  participants  performed  significantly  better  in  the  post -test  than  in  the  pre-test, 
t(52)=34.19,  p<0.0001.  The  following  Table  presents  the  means,  standard  deviations,  and  differences  of  the  t-test  results  for  the 
pre-and  post-tests. 


Test 

N 

Mean 

StDev  T 

P -Value 

Pre-test 

53 

3.00 

6.19 

Post-test 

53 

79.64 

16.58 

Difference 

♦ 

76.64 

16.32 

34.19 

0.0000* 

*p<0.0001 


The  Project  Performance 

All  the  participants  performed  very  well  in  conducting  the  individual  needs  analysis  project  (mean=89.972,  median=94.00, 
and  St.  Dev.=  13.42).  Table  2 lists  the  means  and  standard  deviations  of  the  participants’  performance  in  reaching  each  criteria  of 
the  project. 


Criteria 

Mean* 

Median 

St.  Dev. 

Situational  Description 

18.214 

19.000 

2.521 

Theoretical  Analysis 

18.337 

20.000 

2.706 

Data  Collection 

18.393 

20.000 

3.240 

Data  Analysis  and 
Interpretations 

17.661 

20.000 

4.231 

Recommendation  of 
Solutions 

17.321 

18.500 

17.800 

Overall 

89.972 

94.000 

13.42 

Discussion 

This  study  revealed  a significant  performance  improvement  from  the  pre-test  to  the  post -test  and  an  outstanding  project 
accomplishment,  which  lend  increasing  support  to  the  use  of  adults’  learning  preferences.  These  results  suggest  that  the  factors, 
which  impact  the  professional  trainers’  learning  and  the  application  of  the  needs  analysis  knowledge  and  skills,  are  mutually 
dependent.  It  may  be  that  a good  text  provided  a systematic  description  of  the  covered  content.  Well-organized  handouts  called 
the  trainers’  attention  to  the  most  important  aspects  of  the  content.  Well-thought-out  and  well-paced  lectures  further  interpreted 
and  analyzed  the  meaning  of  the  content.  Group -based  discussion  allowed  the  trainers  to  share  their  expertise  and  related 
experiences  among  themselves,  and  further  increased  their  incentive  to  team.  Realistic  case  studies  set  up  scenarios  for  the 
trainers  to  approach  and  analyze  the  case  and  therefore,  invoked  deeper  understanding  of  all  the  important  factors  involved  in 
each  case.  A combination  of  these  reasons  may  have  accounted  for  the  trainers’  success  in  increasing  post-test  scores,  in  doing  a 
great  job  for  the  individual  needs  analysis  project,  and  in  developing  a more  positive  attitude  toward  the  combined  use  of  the  five 
learning  preferences. 

Also,  from  the  results,  case  studies  were  perceived  by  some  trainers  to  be  helpful  in  comprehending  and  recalling  the  content. 
One  reason  for  this  may  be  that  case  studies  demanded  the  trainers  to  put  the  newly  learned  information  into  practice  and, 
therefore,  empowered  them  to  make  a connection  between  the  knowledge  and  skills  gained,  and  their  application.  In  addition, 
the  project  work  was  cited  by  most  trainers  to  be  effective  in  helping  them  apply,  synthesize,  and  evaluate  the  learning  content.  It 
may  be  that  a purposeful,  individual  project  was  effective  to  enhance  self-initiation  and  the  self-directing  abilities  of  these 
trainers.  With  these  abilities,  they  were  able  to  derive  more  meaning  from  the  process  of  accomplishing  the  project,  thereby 
increasing  the  ability  to  apply,  synthesize,  and  evaluate  the  content.  Another  reason  is  that  these  learning  preferences  were 
arranged  in  a progressive  manner.  The  participating  trainers  were  permitted  to  build  their  learning  progress  hierarchically  due  to 
this  manner.  That  is,  by  the  time  these  trainers  were  engaged  in  flie  project  work,  they  had  acquired  all  the  necessary 
competencies  in  conducting  the  needs  analysis  project.  Because  of  this,  they  felt  more  confident  in  finishing  the  project  and  felt 
that  the  project  work  was  most  beneficial  in  helping  them  accomplish  most  of  the  learning  objectives. 

Furthermore,  a high  level  of  learning  motivation  and  professional  interest  held  by  these  trainers  may  have  contributed 
to  their  positive  learning  results.  The  participating  trainers  demonstrated  their  motivation  to  learn  in  many  ways,  such  as 
attending  classes  regularly,  engaging  in  group  discussions,  and  participating  actively  in  case  studies.  Most  of  them  expressed 
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concerns  of  conducting  needs  analysis  studies  on  a trial -and-error  basis  in  the  past  and  were  eager  to  acquire  a formal  education 
on  the  subject.  While  learning  the  subject,  they  truly  appreciated  the  pragmatic  nature  of  the  subject,  were  very  enthusiastic 
about  the  strong  connection  from  theory  to  practice  provided  by  the  course;  and  were  excited  about  the  possibility  of  applying  the 
acquired  knowledge  to  their  jobs.  Therefore,  the  desire  to  learn  came  from  within,  and  such  a desire,  eventually,  made  these 
trainers  outperform. 

Despite  the  positive  results  of  the  study,  there  were  a couple  of  limiting  factors  involved  in  the  study.  First,  the  study  did  not 
provide  an  opportunity  for  the  participating  trainers  to  reveal  how,  and  why,  the  combined  use  of  the  five  learning  preferences 
worked  for  them.  For  example,  in  what  way  did  the  five  learning  preferences  help  them  have  a successful  construction  of  new 
concepts  about  needs  analysis?  To  what  extent  did  the  use  of  realistic,  open-ended  case  studies  stimulate  these  trainers  to  apply 
the  newly  learned  knowledge  and  skills?  Second,  this  study  did  not  investigate  the  long  term  effects  of  these  preferences. 
Questions  that  need  to  be  answered  include:  will  the  positive  learning  results  acquired  by  these  trainers  be  sustained  over  a long 
period  of  time?  Will  these  trainers  continue  to  achieve  successful  transfer  of  the  needs  analysis  knowledge  and  skills? 

Accordingly,  for  future  research,  it  is  necessary  to  use  think-aloud  interviews  for  the  participants  to  discuss,  in  detail,  how 
and  why  the  use  of  adult  learning  preferences  assist  them  to  construct,  internalize  and  apply  the  knowledge  and  skills  of  needs 
analysis.  Researchers  also  need  to  determine  the  long  term  impact  of  using  the  preferences  of  adults  to  learn  subjects  with  a 
pragmatic  nature.  In  addition,  questions  regarding  which  adult  characteristics  and/or  preferences  should  be  integrated  into  the 
learning  of  different  skills  including  psychomotor,  verbal,  cognitive,  or  attitudinal  need  to  be  answered.  The  examination  of 
other  factors  such  as  levels  of  learning  motivation,  professional  interests  and  needs,  and  prior  knowledge  and  experiences,  which 
all  have  impact  on  adult  learning,  is  necessary. 
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Teachers’  Perceptions  of  Technology: 
Four  Categories  of  Concerns 

HeeKap  Lee 
Indiana  University 


Introduction 

Today,  many  educational  institutions  have  been  challenged  to  integrate  technology  into  their  work  settings.  Technology  is  a 
mean  of  supporting  goals  related  to  increased  student  involvement  with  complex,  authentic  tasks  within  classrooms  and  schools 
(Scheingold,  1991).  However,  successful  technology  integrations  are  sometimes  confronted  with  several  difficulties,  which 
include  clients’  resistance  to  change  (Conner,  1992;  Collins,  1991)  or  the  lack  of  cooperation  of  the  part  of  the  people  involved  in 
that  change  (Kemp,  1 996). 

Recently,  researchers  argue  that  an  innovation  without  considering  clients’  needs  or  concerns  usually  resulted  in  resistance  to 
change  (Ertmer,  1996;  Hall  & Hord,  1978;  Dormant,  1986).  The  reason  for  this  problem  is  a lack  of  attention  to  the  clients’ 
attitude,  perceptions,  and  concerns  that  people  form  toward  innovation.  These  concerns  play  an  important  role  in  the  innovation 
process  as  well  as  in  the  inherent  quality  of  the  proposed  change  (Pershing,  An,  & Lee,  2000).  Hence,  identifying  and  addressing 
concerns  and  perceptions  are  an  essential  task  of  change  agents  during  the  whole  innovation  process.  In  this  article,  I will 
introduce  a framework  for  analyzing  clients’  concerns  and  perceptions  based  on  an  information  technology  project  that  has  been 
implemented  in  a seminary  setting  during  the  last  five  years.  While  conducting  interviews  with  the  teaching  faculty  members 
who  participated  in  the  information  technology  project  at  a seminary,  I identified  four  categories  of  concerns.  In  this  article,  I will 
explain  the  characteristics  of  and  interventions  in  each  category  of  concern. 

Context  of  the  Case 

In  the  mid  1990s  a seminary  in  the  Midwestern  United  States  was  awarded  an  externally  funded  grant  for  a technology 
initiative,  which  included  developing  instructional  computing  capabilities  throughout  the  school  (Saint  Meinrad,  1995).  The 
seminary  hired  two  instructional  interns  to  provide  computer  training  to  the  seminary  faculty  and  staff  At  the  beginning  phase  of 
the  initiative,  the  interns  conducted  a training  needs  analysis.  The  main  focus  of  the  analysis  was  to  gather  information  about  the 
kinds  of  training  programs  faculty  and  staff  members  would  need.  Through  the  analysis,  however,  several  concerns  surfaced 
(Saint  Meinrad,  1998).  For  example,  the  faculty  members  did  not  seem  to  think  that  computer  technology  was  a tool  useful  for 
theology  education,  which  emphasizes  personal  interactions  within  small  groups.  Administrators  of  the  initiative,  however,  did 
not  pay  much  attention  to  this  perceived  concern.  They  proceeded  on  the  assumption  that  faculty  members  always  complain 
about  new  initiatives,  and  they  viewed  such  concerns  as  natural.  They  assumed  that  faculty  members  would  eventually  accept  and 
use  computers  provided  the  faculty  members  received  the  proper  training.  With  these  assumptions  in  place,  the  administrators 
put  effort  into  collecting  and  addressing  training  needs  information  while  ignoring  their  concerns. 

After  the  interns  had  provided  in-service  training  for  one  year,  they  began  to  make  informal  visits  to  the  classrooms, 
computer  labs,  library,  and  the  faculty  resource  center.  They  found  that  many  faculty  members  were  not  integrating  computers 
into  their  teaching.  According  to  the  project  implementation  plan,  almost  all  faculty  members  should  have  been  using  computers 
in  their  instruction  after  one  year,  since  all  the  necessary  facilities  and  training  had  been  provided. 

Faced  with  such  resistance,  the  administration  began  to  take  the  idea  of  concerns  seriously.  The  administrators  of  the 
seminary  learned  it  was  not  the  lack  of  facilities  or  training,  but  concerns  of  the  faculty  that  affected  the  success  of  the  initiative. 
With  this  realization,  they  asked  the  interns  to  conduct  a concerns  analysis.  One-on-one  interviews  and  document  analyses  were 
used  as  data  collection  methods.  All  twenty -six  teaching  faculty  and  several  administrators  and  staff  of  the  seminary  were 
interviewed  (Lee,  2001).  After  conducting  many  rounds  of  card  sorting,  the  interns  identified  four  categories  of  concerns  as 
below. 

Category  I:  Concerns  of  Individual  Incompatibility 

Faculty  perceived  that  the  project  was  not  compatible  with  their  theological  values  or  past  personal  experiences.  These 
concerns  had  a critical  influence  in  the  earlier  stages.  Some  faculty  who  understood  and  were  well  informed  about  the  project, 
however,  had  not  embrace  it  because  of  their  perception  of  technology  as  opposing  theological  pedagogy.  If  they  were  not 
persuaded  during  the  earlier  stages,  then  it  was  hard  to  accept  the  project.  Hence,  this  area  of  concern  was  critically  important  in 
the  earlier  stages  of  an  innovation  project,  but  its  importance  declined  in  the  following  stages.  The  following  are  several 
representatives  of  this  area  of  concerns. 

Conflicts  of  Needs  between  Institution  and  Faculty  Regarding  Technology  A definite  incongruence  existed  in  needs  between 
individual  faculty  and  the  seminary  as  a whole.  The  institution  had  emphasized  that  technology  was  the  only  tool  to  increase 
learning  effectiveness  in  the  information  age.  However,  to  the  individual  faculty,  using  technology  was  just  one  of  many  ways  to 
improve  learning  methodologies.  As  one  faculty  pointed  out,  to  accomplish  the  goal,  it  was  not  necessary  to  incorporate 


technology,  because  technology  was  not  the  only  way  to  accomplish  the  goals.  Furthermore,  overemphasis  on  technology  at  the 
beginning  stage  of  the  project  negatively  influenced  all  faculty  members. 

Skepticism  about  Technology  Ten  faculty  members  among  the  26  were  very  skeptical  about  technology.  This  resulted  from  their 
not  being  aware  of  the  worth,  potential  benefits,  or  value  of  technology.  Also,  they  were  not  convinced  that  technology  was 
indispensable  for  their  academic  setting  or  their  personal  lives. 

Paradigm  Paralysis  Fourteen  faculty  members  out  of  26  expressed  this  category  of  concern,  which  was  the  largest  barrier  to 
adopting  technology  among  the  faculty  members  of  the  seminary.  This  concern  can  be  broken  down  into  two  sub-categories;  the 
faculty’s  perception  of  technology  as  opposing  theological  pedagogy,  and  the  faculty’s  comfort  with  current  teaching  styles. 
Faculty  members  thought  that  technology  (or  the  information  technology  project)  was  basically  incompatible  with  their 
theological  context. 

Theology,  they  argued,  should  focus  on  nurturing  human  nature,  which  is  only  possible  with  human  interactions  between 
instructors  and  students.  In  this  point,  they  felt  that  their  particular  theological  discipline  could  not  adapt  itself  to  technology 
because  theological  educators  are  suspect  of  the  value  of  technology  beyond  the  basics  of  classroom  pedagogy.  The  other  reason 
is  that  they  believed  their  current  teaching  style  had  worked  well  for  several  decades,  so  there  was  little  desire  to  take  the  time  to 
change  it. 

Fear  of  Technology  Nine  faculty  members  expressed  fear  of  technology  in  two  forms:  fear  of  the  unknown  and  fear  of  the  new. 
Some  faculty  were  afraid  something  would  go  wrong  with  the  computers.  Most  faculty  said  that  they  had  not  grown  up  in  the 
technology  culture.  To  them,  technology  was  a foreign  area.  Even  adopters  of  the  innovation  expressed  concerns  that  technology 
had  advanced  so  drastically  that  keeping  pace  with  the  advancement  of  technology  seemed  impossible. 

Laggard  Syndrome  Nine  faculty  members  perceived  themselves  as  being  far  behind  in  using  technology.  One  faculty  who  used 
technology  in  his  classroom  setting  even  expressed  that  he  was  at  the  knowledge  stage,  still  trying  to  find  out  the  benefits  of 
technology. 

False  Information/  Irrational  Belief  Seven  faculty  members  sympathized  strongly  with  the  criticism  that  technology  is  not  a 
learning  tool  proper  for  the  seminary.  Some  faculty  mentioned  that  technology  is  a deterrent  to  human  learning  and 
communication.  Those  arguments  were  not  based  on  scientifically  proven  facts  but  were  based  on  personal  feelings  or  subjective 
reflections.  However,  these  feelings  have  not  allowed  them  to  see  the  potential  benefits  of  the  technology. 

What  are  the  effective  interventions  should  be  needed  to  address  this  category  of  concerns?  Rogers  (1995)  indicated  that 
person-to-person  communication  is  important  to  address  this  area  of  concerns.  Dormant  (1986)  also  suggested  that  change  agents 
should  be  counselors  who  draw  out  concerns,  and  listen  to  and  clarify  the  adoption  units’  needs  and  interests.  Hence,  individual 
persuasion  is  a useful  strategy  to  address  this  area  of  concerns  by  providing  counseling  and  consultation  sessions.  The  seminary 
realized  that  persuasion  on  an  individual  basis  was  the  best  strategy  after  noticing  faculty’s  resistance  to  the  innovation. 

The  seminary  recognized  that  a core  group  was  very  skeptical  about  technology  even  after  several  years  had  passed  since  the 
innovation  started.  To  identify  their  concerns,  the  seminary  conducted  one-on-one  interviews  with  faculty  members  to  become 
aware  of  the  many  issues  that  related  to  this  area.  The  seminary  stressed  that  Instructional  Service  staff  were  not  attempting  to 
change  faculty’s  teaching  styles,  but  to  enhance  their  teaching  styles  with  the  use  of  technology.  Also,  the  seminary  published  a 
monthly  technology  newsletter,  both  in  print  and  on  the  intranet,  featuring  articles  on  the  individual- incompatible  area  of 
concerns.  Several  faculty  members  wrote  articles  mentioning  their  successful  experiences  with  technology  in  their  teaching 
settings.  The  seminary  provided  opportunities  for  faculty  members  to  visit  other  advanced  technology  -driven  education  institutes 
or  learning  opportunities  to  familiarize  them  with  the  practical  applicability  of  technology  in  the  seminary  context.  Also,  more 
than  ten  faculty  members  attended  technology -related  seminars,  conferences,  and  workshops. 

Category  II:  Concerns  of  Unknown 

Even  when  the  value  of  an  innovation  is  compatible  with  the  target  audience’s  values,  the  individuals  of  the  adoption  units 
may  not  accept  the  proposed  innovation  as  planned  for  several  reasons,  including  fear  of  the  unknown  and  lack  of  information  or 
knowledge  required  to  implement  the  innovation.  In  the  earlier  stages,  the  individual  faculty  usually  felt  fear  of  the  unknown  or 
fear  of  lacking  required  knowledge  or  skills.  The  following  are  the  typical  examples  of  this  category  of  concern. 

More  Work  To  eight  faculty  members,  technology  was  one  more  burden  that  they  had  to  learn.  Technology  adds  or  creates 
another  task.  Even  faculty  who  used  technology  in  the  classroom  expressed  this  concern  most  often.  To  learn  technology  was 
becoming  increasingly  stressful  and  time  consuming  for  the  faculty  members. 

Lack  of  Detailed  Information  about  the  Project  The  vision  of  the  project  was  not  address  well  to  all  faculty  members.  The  lack 
of  vision  also  made  it  difficult  to  set  up  the  details  for  diffusing  the  innovation.  Five  faculty  confessed  that  they  were  not  aware  of 
the  detailed  tasks  in  relation  to  the  innovation.  They  expressed  concerns  about  how  technological  innovations  were  to  apply  to  the 
particular  learning  environment. 
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Teaching/Mentoring  Concerns  Providing  individual  teaching  or  mentoring  was  an  effective  means  of  adopting  the  technology  by 
faculty  members.  Individual  training  was  preferred  over  group  sessions  by  the  school  faculty.  Several  reasons  were  expressed. 
The  difficulty  of  finding  a common  time  among  faculty  members  and  consideration  of  individual  pacing  were  major  reasons. 

Time  Conflict  Eight  faculty  mentioned  that  time  was  one  of  the  most  important  concerns  in  their  not  adopting  technology.  They 
said  that  technology  was  not  a priority  to  them,  for  their  primary  responsibility  was  to  prepare  a class  or  preach.  Some  faculty  had 
not  even  tried  to  learn  technology  because  they  worried  about  how  much  time  would  be  spent. 

Students'  Unfavorable  Attitudes  toward  Technology  Students’  unfavorable  attitudes  toward  technology  were  also  mentioned  by 
two  faculty  members.  Ironically,  faculty  members  who  had  unfavorable  attitudes  toward  technology  mentioned  that  students 
showed  the  same  phenomena.  Two  faculty  expressed  that  students  did  not  say  that  the  technology  helped  them.  That  made  some 
faculty  not  use  much  technolo©'  in  classrooms. 

Lack  of  Information  about  Good  Applications  Four  faculty  members  said  that  it  was  hard  to  find  someone  who  had  applied 
technology  very  well.  There  was  no  easy  way  to  identify  other  faculty  members  on  campus  who  had  already  begun  to  use 
technology  effectively  in  their  teaching.  And  for  most  theological  disciplines  in  particular,  there  was  no  comprehensive,  easy-to- 
fiind  source  of  information  about  relevant  instructional  applications  of  technology.  While  the  number  of  locally  successful  models 
of  educational  uses  of  technology  continued  to  increase,  access  to  good  descriptions  of  those  models,  training  for  them,  and 
reports  of  their  strengths  and  weaknesses  were  not  easy  to  find. 

The  major  strategy  for  addressing  this  category  of  concerns  is  learning,  because  usually  these  concerns  can  be  overcome  by 
providing  well-organized  training  programs,  job  aids,  and  consultation  programs.  Also,  providing  correct  information  in  a timely 
manner  is  another  useful  strategy  to  address  this  area  of  concern.  However,  the  faculty’s  learning  focus  changed  from  general  and 
basic  issues  of  technology  to  more  elaborate  and  complicated  issues,  such  as  transferring  or  applying  the  technology  in  more 
specific  contexts  in  this  case.  This  is  why  the  learning  format  changed  over  time  from  the  general  group -based  to  the  individual 
customized  format. 

To  identify  issues  in  this  area  and  set  up  learning  interventions,  the  seminary  conducted  a needs  analysis  project  by 
conducting  interviews  with  each  faculty  member  as  well  as  mailed  surveys  (Saint  Meinrad,  1998).  Based  on  these  phenomena, 
several  learning  interventions  were  arranged  in  the  seminary.  First  of  all,  an  individual  learning  road  map  for  each  faculty 
member  was  developed.  According  to  the  road  map,  the  well-organized  technology  training  programs  were  provided  for  the 
following  year.  The  interns  had  been  working  on-site  on  a weekly  basis.  After  taking  these  programs  for  one  year,  the  faculty 
improved  their  computer  competencies  from  2.5  out  of  5 on  the  Likert  scale  to  3.1  in  the  same  survey  (Saint  Meinrad,  1998).  As 
the  faculty  moved  deeper  into  the  innovation,  the  focus  of  learning  shifted  to  more  individualized  consulting  and  one-to-one 
training  sessions.  Also,  remote  consulting  was  offered  by  using  electronic  communication  channels  between  the  faculty  and  the 
outside  interns. 

To  address  time  concerns,  the  seminary  formed  a committee  to  reorganize  teaching  loads.  The  recommendation  of  the 
committee  was  that  the  eclectic  courses  could  be  cancelled  if  few  students  enrolled,  so  the  faculty  could  be  learning  at  that  time 
instead  of  teaching  the  course.  Also,  lack  of  time  to  learn  was  the  most  crucial  factor  in  this  category  of  concerns  in  this 
seminary.  To  address  this  concern,  the  seminary  developed  a training  schedule  that  was  flexible,  meeting  at  different  hours  of  the 
day,  even  evenings,  so  that  the  faculty  could  best  take  advantage  of  the  offerings. 

Category  111:  Concerns  of  Organizational  Support 

The  organizational-compatible  concerns  were  salient  factors  to  he  considered  at  the  middle  stage  of  the  project  at  the 
seminary.  Faculty  who  understood  the  benefits  of  the  innovation  did  not  adopt  it  because  there  were  no  organizational 
encouragements  to  do  so.  Many  faculty  members  expressed  concerns  about  the  lack  of  organizational  supporting  systems  and 
motivational  systems.  The  following  remarks  are  the  typical  expressions  of  this  category  of  concern.  The  following  are  several 
representative  examples  of  this  category  of  concern. 

Equipment  and  Maintenance  Problems  Several  concerns  were  expressed  in  this  category  by  seven  faculty  members.  First,  the 
faculty  experienced  difficulties  when  servers  went  down,  especially  after  hours  or  on  weekends.  Some  buildings  were  not 
equipped  with  technology.  Not  much  software  was  installed  in  the  Faculty  Resource  Center  (FRC)  or  the  Educational 
Technology  Center  (ETC),  which  made  programs  hard  to  access  when  needed.  Services  from  maintenance  persons  were  hard  to 
find  or  untimely  when  computing  problems  occurred.  Students  may  not  have  had  support  from  the  seminary  to  fix  their 
computers  if  the  computers  had  problems. 

Students’  Limited  Access  to  Equipment  and  Support  Services  Three  faculty  members  of  the  26  mentioned  that  students’ 
opportunities  to  access  technology  were  limited.  Not  all  the  students’  rooms  and  classrooms  were  wired.  Some  students  could  not 
access  the  technology,  so  electronic  communication  was  sometimes  impossible.  Also,  the  computers  in  the  student  production 
center  were  so  old  that  students  could  not  use  advanced  software. 

Students  not  Involved  Three  faculty  members  mentioned  that  students  were  not  involved  in  the  innovation  from  the  start.  It  was 
directed  to  the  faculty  group  only  and  it  began  without  asking  how  students  would  learn  or  use  the  technology. 
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Lack  of  Organizational  Benefits  Lack  of  organizational  benefits  and  motivational  factors  were  mentioned  by  seven  faculty 
members  out  of  26.  The  institution  did  not  recognize  the  adopters  of  the  innovation.  Two  faculty  mentioned  they  might  have 
adopted  it  if  the  institution  had  offered  some  motivation  or  incentives,  such  as  monetary  benefits.  Suggestions  of  non-monetary 
benefits  were  also  mentioned,  such  as  vacations  and  training  opportunities,  and  to  lessen  the  teaching  burden. 

Distrust  and  Poor  Communication  Among  Stakeholders  There  was  little  communication  and  coordination  among  stakeholders 
during  the  innovation  diffusion  process.  The  innovation  initiators  did  not  even  try  to  gather  ideas  from  the  three  constituent 
groups,  faculty  members,  staff,  and  students,  in  order  to  make  the  most  effective  uses  of  technology,  new  approaches  to  teaching 
and  learning,  and  other  available  resources  in  the  seminary  setting.  Six  faculty  of  the  26  expressed  problems  with  innovation  in 
this  category. 

This  area  of  concerns  is  relatively  easy  to  measure  and  to  eliminate  if  addressed  carefully  and  in  a timely  fashion  during  the 
innovation  diffusion  process  (Fisher,  Wilmore,  & Howell,  1994).  Traditionally,  most  change  scholars  have  overlooked  these 
concerns  at  the  beginning.  However,  in  order  to  lead  a successful  innovation  project,  the  plan  has  to  be  reviewed  regularly  during 
the  innovation  process.  Raising  money,  allocating  resources,  and  providing  technical  and  administrative  support,  including 
incentives  or  motivational  systems,  are  essential  elements. 

These  concerns  can  be  eliminated  by  acquiring  resources  and  equipment,  providing  timely  technical  and  administrative 
support,  providing  incentives  or  benefit  systems,  and  maintaining  equipment.  Foa  (1993)  pointed  out  that  incentives,  support,  and 
reward  structures  are  needed  in  order  to  make  the  efforts  of  the  individuals  more  widespread  and  their  results  used  more 
comprehensively.  Major  problems  for  the  seminary  lay  in  the  institution’s  failure  to  provide  motivation  or  incentives  to 
encourage  faculty  members’  active  usage  of  the  innovation.  Many  faculty  suggested  both  monetary  and  non-monetary  benefits 
and  motivators,  such  as  vacations,  training  opportunities,  and  a lessened  teaching  burden.  While  not  providing  any  monetary 
benefits,  the  seminary  provided  many  forms  of  non-monetary  benefits,  such  as  providing  training  programs  and  visiting  other 
technologically  advanced  schools.  The  director  of  the  Academic  Computing  Department  became  a member  of  the  president’s 
cabinet,  a group  of  advisors  to  the  president,  and  thus  was  directly  involved  in  developing  a new  master  plan,  which  included 
major  renovations  of  several  buildings  over  the  next  five  years. 

Category  IV:  Concerns  of  Organizational  Incompatibility 

Last  category  of  concerns  is  related  to  the  organizational  incompatibility.  Faculty  expressed  that  the  innovation  was  not 
compatible  with  the  seminary  culture.  The  seminary  culture  was  oriented  toward  more  human  interaction,  and  focused  on 
formation-building.  Furthermore,  the  seminary  was  isolated  geographically,  as  well  as  divided  by  disciplines.  They  also 
expressed  their  isolation  regarding  the  innovation.  The  innovation  was  initiated  in  a top-down  manner.  They  did  not  receive 
information  in  a timely  fashion.  Clear  goals  and  directions  for  the  innovation  were  not  given  to  the  faculty.  Furthermore,  faculty 
tended  to  work  individually  rather  than  in  teams.  Every  faculty  member  understood  the  innovation  differently.  Hence,  they 
perceived  that  two  incongruent  innovation  diffusion  tracks  existed  in  the  seminary:  the  individual  faculty  track  and  that  of  the 
institution.  This  concern  began  to  increase  in  importance  after  addressing  the  individual-incompatible  concerns,  but  increased 
strongly  in  importance  at  the  implementation  stage  during  the  diffusion  process.  The  following  are  several  examples  of  this 
category  of  concerns. 

Isolated  Culture  The  cultural  characteristics  of  the  seminary,  4 faculty  argued,  were  not  compatible  with  technology.  First,  in 
preparing  people  for  the  ministry,  the  top  priority  of  the  seminary  is  fundamentally  different  than  preparing  people  to  teach  in 
other  higher  educational  institutions.  The  use  of  technology  can  be  maximized  mainly  in  the  latter  setting.  Religious 
organizations  such  as  the  seminary  must  snphasize  the  value  of  forming  and  building  relationships,  which  does  not  embrace 
technology.  Hence,  some  faculty  mentioned  that  technology  was  not  a driving  force  at  the  seminary.  Second,  faculty  had  not 
grown  up  in  a technology  culture.  Some  faculty  mentioned  that  the  European  learning  model,  which  mainly  uses  lecture  format  in 
classrooms,  had  influenced  the  faculty  members  who  had  studied  in  Europe,  who  were  the  majority  in  the  seminary.  Third,  the 
individuality  of  faculty  members  was  another  cultural  characteristic.  Most  of  the  faculty  pursued  different  disciplines  and 
different  areas  of  interest.  That  was  the  major  reason  why  faculty  were  accustomed  to  working  individually  rather  than  in  teams, 
which  the  innovation  sometimes  required  them  to  do. 

Class  Characteristics  Another  reason  for  incompatibility  originated  from  the  class  contexts  that  were  small- group  class  setting 
and  technology  was  not  related  the  course  content.  Five  faculty  members  argued  that  technology  could  not  make  an  impact  in  a 
small  class.  Most  of  classes  were  populated  by  fewer  than  10  students.  In  this  situation,  technology  was  ineffective  for  increasing 
learning.  The  other  reason  why  the  faculty  did  not  utilize  technology  during  class  was  their  perception  of  the  inability  of  the 
course  to  embrace  technology. 

Sharing  and  Showing  Learning  technology  was  one  of  the  biggest  concerns  of  the  faculty  members.  Nine  faculty  members  of  the 
26  expressed  this  type  of  concern.  Sharing  information  about,  or  experiences  with,  technology  among  faculty  was  vital,  and  it 
could  have  been  a strong  influence  on  the  faculty  as  a whole.  Partly  owing  to  a lack  of  vision  for  the  innovation  and  to  a lack  of 
concrete  examples  of  how  to  apply  technology  in  a seminary  setting,  the  faculty  wanted  to  see  other  people’s  experiences  and  or 
knowledge. 
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Not  Having  a Clear  Image  of  the  Project  Eleven  faculty  out  of  26  expressed  a lack  of  vision  for  the  project  from  the  beginning. 
This  area  of  concern  was  the  second  largest  barrier  for  the  faculty.  They  argued  that  the  innovation  project  was  started  by  grant 
money  rather  than  a vision.  Without  serious  questions  about  why  this  innovation  was  needed  in  the  seminary,  the  institution 
started  the  innovation,  and  this  made  it  difficult  for  the  faculty  members  to  grasp  the  vision  or  purpose  of  the  innovation. 

Fragmented  Technology  Planning  Five  faculty  members  argued  that  the  innovation  was  started  without  considering  the  necessity 
of  information  technology  carefully  in  the  context  of  Catholic  pedagogy.  They  expressed  that  the  innovation  was  focused  on 
teaching  rather  than  learning,  and  focused  on  media  rather  than  methods.  Two  faculty  criticized  the  innovation  for  starting  in 
reverse  order,  selecting  media  (buying  computers)  without  considering  methods.  One  faculty  mentioned  that  this  project  had 
missed  one  critical  stage  in  the  beginning:  needs  assessment  or  values  clarification. 

Collaboration  was  the  most  useful  strategy  to  address  this  area  of  concern  in  the  seminary.  To  address  issues  of  this  area,  the 
seminary’s  geographical  isolation,  diverse  faculty  disciplines,  and  a top-down  diffusion  strategy,  collaborative  work  among  the 
faculty  was  essential.  For  example,  creating  a vision  statement  and  sharing  the  innovation-related  experiences  with  other 
individuals  in  the  adoption  units  were  helpful  tasks  in  the  seminary. 

The  seminary  formed  an  ad  hoc  committee  to  set  up  a clear  vision  for  technology  and  teaching  at  the  seminary.  The 
committee  developed  a vision  with  consensus  from  all  faculty  members  and  reported  their  findings  to  the  faculty.  Another 
intervention  was  to  arrange  several  learning  events  in  order  to  facilitate  collaborative  work  among  faculty  members  in  the 
seminary.  Through  these  events,  faculty  members  shared  their  ideas  with  other  faculty  members.  Sharing  among  faculty  was  the 
key  activity  for  changing  the  seminary  culture.  These  events  included  faculty  presentation  day,  faculty  learning  day,  small  group 
interests,  brownbag  lunches  and  learning  sessions.  Also,  through  the  funds  from  the  grant,  many  faculty  took  advantage  of 
conference  opportunities  to  gain  more  knowledge  about  the  appropriate  use  of  technology.  Furthermore,  the  seminary  developed 
contacts  with  other  schools  facing  the  same  issues  and  was  able  to  find  and  demonstrate  good  practices  in  technology  for 
theological  instruction. 

Concluding  Remarks 

Information  technology  is  an  effective  means  of  increasing  teaching  and  learning  effectiveness  in  higher  educational  settings 
including  seminaries.  However,  it  must  be  well  planned  and  organized  before  the  project  begins.  Identifying  clients’  concerns  and 
taking  care  of  them  are  an  important  task  of  change  agents  during  an  innovation  process.  Setting  up  a vision  statement, 
conducting  perception  analysis,  and  preparing  detailed  plans  for  the  project  would  guarantee  a successful  implementation  of  an 
information  technology  project  in  a higher  education  setting. 
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Introduction 

Universities  across  the  country  are  offering  a growing  number  of  courses  on  the  World  Wide  Web.  Many  faculty  and 
instructors  are  now  faced  with  the  challenge  of  teaching  at  a distance  in  a Web-based  environment  and  are  collaborating  with 
instructional  designers  on  how  to  effectively  design  Web-based  distance  education  courses. 

Teaching  at  a distance  requires  different  strategies  than  those  used  in  a traditional  teaching  environment.  In  traditional 
classes,  interactions  between  learner  aid  instructor  and  among  all  learners  in  a class  are  mostly  synchronous.  Often  a lack  of 
interaction  has  been  considered  a major  weakness  of  distance  education,  blamed  for  causing  feelings  of  isolation  in  learners  who 
are  situated  all  over  the  world  with  no  or  minimal  face  to  face  contact.  Interaction  between  the  learner  and  the  course  materials 
and  resources  can  be  hindered  by  new  and  unfamiliar  technology.  Nontraditional  learners  taking  courses  on  the  Web  may  not 
have  the  self-regulation  or  study  skills  to  perform  well  in  this  environment..  Interactions  on  each  of  these  levels  have  been 
identified  as  a critical  component  for  successful  online  learning  (Hirumi  & Bermudez,  1996;  Moore,  1989;  Rob  Iyer  & Ekhaml, 
2000;  Schrum  & Berge,  1997;  Vrasidas  & Mclsaac,  1999;  Wagner,  1994),  yet  it  can  be  extremely  difficult  for  instructors  to 
foster  interactions  in  a distance  environment. 

Much  of  the  literature  regarding  interactions  in  distance  education  has  focused  on  the  development  and  use  of  technology 
tools  that  permit  interactivity  or  have  focused  on  interactions  as  if  they  are  the  outcomes  of  instruction  rather  than  the  methods  of 
instruction  as  pointed  out  by  Wagner  (1997).  Little  research  has  focused  on  interactions  as  instructional  methods  rather  than 
outcomes. 

An  instructional  design  (ID)  theory  is  a “theory  that  offers  explicit  guidance  on  how  to  better  help  people  learn  and  develop” 
(Reigeluth,  1999).  ID  theories  are  prescriptive  and  design -oriented,  identifying  methods  of  instruction  (including  instructional 
strategies  and  tactics)  and  the  situations  in  which  these  methods  should  and  should  not  be  used,  based  on  the  instructional 
conditions  and  desired  outcomes. 

The  current  study  developed  and  formatively  evaluated  an  instructional  design  theory  to  guide  designers  in  selecting  when 
and  how  to  utilize  interactions  as  instructional  methods  in  a Web-based  distance  learning  higher  education  environment. 

Research  Questions 

1 . What  are  the  types  and  outcomes  of  interactions  between  participants  in  a Web-based  learningenvironment? 

2.  When  and  how  should  these  various  interactions  be  designed  into  a Web-based  learning  environment? 

Methods:  Phase  1 

Construction  of  New  Instructional  Design  Theory 

Phase  1 of  the  research  methodology  involved  construction  of  the  new  instructional  design  theory.  A type  of  formative 
research  methodology,  as  described  by  Reigeluth  and  Frick  (1999),  was  used  to  create  and  test  this  new  instructional  design 
theory.  Specifically,  we  designed  a particular  case  in  which  to  develop  the  new  theory.  We  chose  an  instructional  situation  (a 
Web-based  graduate  course)  that  we  wanted  our  theory  of  interaction  to  apply  to,  helped  to  design  the  case  (the  course),  and 
developed  a tentative  design  theory  in  parallel.  The  case,  therefore,  became  an  instance  of  the  theory. 

The  designed  case  was  that  of  a Web-based  distance  graduate  course  in  educational  psychology  at  a large  Midwestern 
university.  The  course  is  required  for  many  programs  in  the  school  of  education.  The  course  is  run  entirely  through  a course 
management  system  on  the  Web.  Both  researchers  were  involved  in  the  course  design  process,  as  recommended  by  Reigeluth  and 
Frick  (1999).  By  participating  in  the  course  design,  it  fosters  “intimate  familiarity  with  the  case  [which]  is  essential  for 
developing  good  grounded  theory”  (p.  644).  The  design  theory  was  developed  in  parallel  with  the  course. 

A five-member  design  team,  including  both  the  instructor  and  associate  instructor,  met  once  a week  for  approximately  three 
months  to  design  and  develop  the  course.  The  researchers  documented  the  design  decisions  and  rationales  as  they  developed  the 
theory.  This  included  documentation  of  intended  learning  outcomes;  selection  of  methods,  strategies  and  tactics;  documentation 
of  situational ities  which  influenced  design  decisions;  and  identification  of  the  values  behind  instructional  decisions. 

The  overall  learning  outcomes  for  this  graduate  course  were  for  students: 

• To  become  conversant  with  the  basic  assumptions,  concepts  and  principles  of  each  learning  theory 

• To  determine  the  possible  implications  of  each  theory  for  instructional  settings 

• To  be  able  to  compare  and  contrast  theories  and  their  usefulness  in  the  various  settings  of  interest  to  the  students  in  the 
course  and  other  education  professionals 
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• To  create  and  revise  a personal  theory  of  learning 

Throughout  the  design  of  the  graduate  course,  we  consistently  documented  our  rationales  for  the  selection  of  particular 
methods  for  particular  learning  outcomes,  particularly  as  they  related  to  use  of  interaction  methods.  We  referred  to  the  design 
notes  to  identify  what  the  intended  outcomes  were  for  the  method  and  extrapolated  the  instructional  conditions  and  instructional 
outcomes. 

The  following  procedure  was  used  (Reigeluth,  1983;  Nelson,  1998)  to  develop  the  theory  in  tandem  with  the  designed  case 
study: 

Define  the  Purpose  of  the  Theory 

The  purpose  of  this  theory  is  to  provide  guidance  for  designers  of  Web-based  graduate  courses  on  when  and  how  to  utilize 
learner  to  learner,  learner  to  instructor,  and  learner  to  resource  and  learner  to  self- interactions  as  instructional  methods. 
Interactions  as  an  instructional  method  contribute  to  the  outcomes  of  the  instruction  such  as  team  building,  clarifying 
understanding,  supporting  learner  control  and  enhancing  elaboration  and  retention  (Wagner,  1997). 

Define  the  Values  of  the  Theory 

According  to  Reigeluth  (1999),  values  “guide  selection  of  goals  and  selection  of  methods”  (p.  12).  The  values  that  guide  this 
theory  include: 

• A belief  that  adults  are  most  motivated  to  learn  when  they  feel  success,  volition,  value  and  enjoyment 

• Higher-order  thinking  skills  and  complex  cognitive  tasks  are  best  fostered  by  interacting  with  other  learners  to  socially 
constmct  meaning 

• Instruction  should  provide  variety 

• Instruction  should  foster  creativity 

• Instruction  should  be  authentic  and  relevant  to  learners’  lives 

• Instruction  be  related  to  previous  experiences  and  linked  to  prior  knowledge 

• Resources  should  be  available  to  help  learners  with  their  tasks 

• Activities  should  build  on  previous  work  done  in  the  course 

• Instruction  should  provide  social  as  well  as  cognitive  support 

• All  learners  should  have  the  chance  to  participate 

• Cognitive  overload  should  be  avoided 

• Learners  should  have  responsibility  for  their  own  learning 

Determine  the  Specific  Domain,  Situation  or  Scope  of  the  Theory. 

This  theory  addresses  learning  in  the  cognitive  domain.  It  is  applicable  to  learning  situations  with  adult  learners  in  higher 
education  settings,  particularly  Web-based  learning  environments.  The  scope  of  the  theory  is  specifically  to  provide  guidance  on 
when  and  how  to  utilize  interactions  as  instructional  methods  that  can  be  used  effectively  to  attain  cognitive  outcomes. 

Identify  an  Optimal  Participant  Interaction  Process  on  which  to  Model  the  Theory. 

The  development  of  an  optimal  participant  interaction  process  used  in  this  study  is  based  on  previous  experience  of  the 
researchers  both  as  students  and  as  researchers.  This  serves  as  a framework  for  the  synthesis  of  the  review  of  the  relevant 
literature.  The  review  of  the  literature  can  be  seen  as  an  initial  formative  research  of  the  developing  theory. 

Moore  (1993)  points  out  that  distance  education  is  not  simply  a geographic  separation  of  learners  and  teachers,  but  more 
importantly,  a pedagogical  concept.  It  is  a concept  describing  the  universe  of  teacher-learner  relationships  that  exist  when 
learners  and  instructors  are  separated  by  space  and  /or  by  time  (Moore,  1993).  From  this  separation,  special  patterns  of  learner 
and  teacher  behaviors  that  affect  both  teaching  and  learning  are  formed.  The  psychological  and  communications  space  of 
potential  misunderstanding  between  the  learners  and  instructors  is  called  the  transactional  distance.  The  three  sets  of  variables, 
which  define  the  extent  of  transactional  distance  in  an  educational  environment,  are  dialogue,  structure  and  learner  autonomy. 
‘Dialogue’  can  be  translated  as  interaction,  which  places  value  on  the  synergistic  nature  of  relationship  of  the  involved 
participants  (Moore,  1993).  Based  on  this  theory,  interactions  are  even  more  important  in  distance  environments  than  in 
residential  environments,  as  when  participant  are  separated  by  space  and  time  the  transactional  distance  is  more  difficult  to 
overcome. 

Vygotsky’s  (1978)  understanding  of  learning  as  a social  process  is  also  critical  to  the  discussion  of  distance  learning  theories. 
Various  technologies  and  tools  used  for  distance  education  create  the  context  and  setting  where  learning  can  occur.  In  Vygotsky’s 
concept  of  the  Zone  of  Proximal  Development  (ZPD),  social  interaction  is  crucial  to  the  development  of  the  new  patterns  of 
thoughts  and  behaviors. 

The  tools  and  signs  the  learner  is  exposed  to  play  a great  part  in  influencing  or  mediating  the  new  patterns  of  thought  and 
mental  functioning  (Werstch,  1991).  Under  the  socio-cultural  framework,  what  is  meant  by  knowledge  and  learning  is  a change 
in  perspective.  If  meaning  must  be  personally  constructed,  then  the  learner  is  central  to  the  learning  process  because  personal 
experience  determines  reality.  Hence,  there  is  a need  to  consider  instructional  design  from  a more  learner- centered  perspective. 
Bonk  and  Cunningham  (1998)  incorporated  the  framework  of  sociocultural  theory  and  principles  for  computer-supported 
collaborative  learning  (CSCL)  environments.  Compared  to  the  cognitive  constructivists’  focus  on  making  learning  more  relevant 
and  building  on  students’  prior  knowledge,  social  constructivists  emphasize  human  dialogue,  interaction,  negotiation,  and 
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collaboration  (Bonk,  Oyer  & Medury,  1995).  It  is  argued  that  social  interaction  and  dialogue  is  central  to  learning.  New  skills 
and  strategies  first  appear  in  a social  place  with  adults  and  more  capable  peers  in  one’s  learning  environment  and  are  later 
internalized  (Bonk,  Appleman  & Hay,  1 996). 

Thus,  interactions  among  others  in  the  learning  environment  are  important  to  the  learning  process.  Interaction  itself  can  be 
defined  as  “sustained,  two-way  communication  among  two  or  more  persons  for  purposes  of  explaining  and  challenging 
perspectives”  (Garrison,  1993,  p.l6)  or  as  “two-way  communication  among  two  or  more  people  within  a learning  context,  with 
the  purposes  as  either  task/instructional  completion  or  social  relationships  building”  (Gilbert  & Moore,  1998). 

Moore  (1989)  distinguishes  three  types  of  interaction  in  distance  education:  1)  Learner-content  interaction.  The  learners  are 
constructing  knowledge  through  a process  of  accommodating  new  understanding  into  their  cognitive  structure;  such  interaction 
with  content  is  one  way  to  restructure  knowledge.  2)  Learner-instructor  interaction  relates  to  the  assistance,  counsel,  organization, 
stimulation  and  support  that  the  instructor  provides  to  the  learner  in  constructing  new  understanding  of  the  content.  3)  Learner- 
learner  interaction  is  a kind  of  “interaction  between  one  learner  and  other  learners,  alone  or  in  group  settings,  with  or  without  the 
real-time  presence  of  an  instructor”  (Soo  & Bonk,  1998). 

In  addition  to  the  three  types  of  interaction  identified  by  Moore  (1989),  many  researchers  emphasize  learner-  to-self 
interaction  as  a fourth  very  important  component  in  learning:  4)  Learner-self  interaction:  participating  in  an  internal  dialogue 
with  oneself.  While  some  researchers  emphasize  the  importance  of  learner-learner  interaction,  minimizing  the  importance  of  the 
time  allocated  to  self-reflection  during  online  education  (Soo  & Bonk,  1998),  others  note  the  importance  of  the  inner-dialogue 
system,  suggesting  that  the  development  of  self-regulatory  skills  is  vital  to  independent  learning  and  instruction  (Savery  & Duffy, 

All  four  types  of  interaction  are  vital  for  distance  education,  although  each  type  may  be  most  appropriate  for  different  tasks 
and  for  learners  at  different  stages  of  development  (Moore  & Kearsley,  1996.)  An  instructional  design  theory  can  help  us  make 
this  determination. 

Based  on  the  literature  it  is  clear  that  interactions  have  instructional  value  and  that  there  are  many  ways  of  fostering  such 
interactions  in  distance  education  environments.  However,  this  literature  falls  short  of  being  prescriptive  in  nature  because  little 
attention  is  paid  to  the  situationalities  in  which  the  methods  may  be  more  or  less  effective.  Nor  does  the  literature  address  the 
relationship  between  the  various  types  of  interactions  and  outcomes.  Thus  our  theory  will  focus  on  addressing  situationalities  in 
which  the  methods  are  best  used. 

Determine  Goais/Outcomes. 

Wagner  (1997)  identified  thirteen  outcomes  of  interactions  that  are  relevant  to  the  design  of  this  theory.  These  include: 

• To  increase  participation  and  engagement  with  the  learning  process 

• To  increase  social  engagement  with  others  in  the  group.  To  develop  communication.  To  receive  feedback 

• To  enhance  elaboration  and  retention 

• To  support  learner  control/self-regulation 

• To  increase  motivation 

• To  negotiate  understanding.  To  build  a team 

• To  discover 

• To  explore.  To  clarify  understanding  intended 

• To  gain  closure 

The  development  of  our  theory  included  an  investigation  of  how  these  various  outcomes  were  attained  through  the  four  types  of 
interactions. 

Develop  Methods,  Strategies  and  Tactics. 

This  theory  focuses  on  the  use  of  four  methods  of  interactions:  learner  to  learner,  learner  to  instructor,  learner  to  content  and 
learner  to  self. 

Determine  Conditions/Situations. 

Situations  are  “aspects  of  the  instructional  context  that  influence  selection  of  methods”  (Reigeluth,  1999,  p.8).  If  any  element 
of  the  situation  changes,  the  instructional  methods  may  need  to  be  changed  as  well.  Situations  have  two  parts:  instructional 
conditions  and  the  desired  outcomes  of  instruction.  Instructional  conditions  include  the  nature  of  what  is  to  be  learned  (learning 
outcomes),  the  nature  of  learner,  the  nature  of  the  learning  environment,  and  the  nature  of  instructional  development  constraints. 

According  to  Reigeluth  (1999),  the  desired  instructional  outcomes  include  the  level  of  effectiveness  (how  well  learning  goals 
are  attained),  the  level  of  efficiency  (effectiveness  divided  by  cost  and  time),  and  the  level  of  appeal  (extent  to  which  students 
enjoy  the  instruction). 

Our  findings  from  the  formative  research  of  the  theory  enabled  us  to  identify  in  which  conditions  and  situations  the  methods 
of  interactions  worked. 


Methods:  Phase  2 

In  Vivo  Formative  Research  for  a Designed  Case 

Phase  2 of  the  study  involved  formative  research  of  the  theory.  Formative  research  was  done  by  a review  of  the  literature, 
expert  evaluation  and  through  a field  trial.  A review  of  the  literature  was  done  to  support  the  optimal  participant  interaction 
process  around  which  the  theory  was  developed.  Expert  evaluation  of  the  theory  was  done  by  the  instructor  of  the  course  used  as  ^ 
the  designed  case  study.  Finally,  a field  trial  was  conducted  with  the  course  as  it  was  taught  during  the  summer  of  2001. 

Formative  evaluation  of  the  designed  case  was  conducted  as  a field  trial.  The  purpose  of  the  formative  evaluation  was  to 
identify  and  remove  problems  with  the  instructional  design  theory  (Reigeluth  & Frick,  1 999;  Thiagarajan,  Semmel,  & Semmel, 
1974).  This  took  place  in  the  summer  of  2001  as  the  course  was  being  taught.  An  instrumental  case  study  approach  (Stake,  1995) 
was  used.  Stake  describes  the  purpose  of  the  case  examination  as  being  to  provide  insight  into  an  issue  or  to  redraw  a 
generalization.  Though  a case  study  focuses  on  a single  case,  the  researchers  can  construct  a hypothesis  for  generalization  by 
close  observation  of  the  case.  The  case  plays  a supportive  role,  and  it  facilitates  the  researcher’s  understanding  for  more  gpneral 
situations. 

Course  Context 

The  instructor,  an  associate  professor  in  the  department,  and  the  associate  instructor,  a doctoral  student,  participated  in  the 
design  and  development  as  well  as  the  implementation  of  the  course.  This  is  the  first  time  the  instructor  and  associate  instructor 
had  taught  a Web-based  course.  However,  both  were  very  familiar  with  the  content  of  the  course,  as  the  instructor  had  taught  the 
residential  version  of  the  course  many  times.  Nineteen  graduate  students  completed  the  fourteen-week  graduate  course  during  a 
summer  session.  Among  the  nineteen  students,  eight  students  were  part  of  a cohort  group  that  had  begun  a Web-based  Master’s 
degree  program  two  semesters  earlier.  Thirteen  students  had  taken  distance  courses  prior  to  this  one. 

The  fourteen-week  course  was  organized  into  eight  units  plus  an  orientation  unit.  Each  student  was  required  to  participate  in 
two  team  collaborative  activities  but  could  choose  whether  or  not  to  work  in  teams  for  additional  projects.  The  maximum  number 
of  team  activities  a student  could  participate  in  was  five.  The  students  were  given  seven  assignments  total:  they  were  allowed  to 
choose  to  work  on  the  assignments  either  individually  or  in  a group. 

Data  Collection 

Data  for  formative  evaluation  of  the  course  as  part  of  the  formative  research  on  the  instructional  design  theory  was  collected 
as  follows.  The  researchers  informed  course  participants  that  a new  instructional  design  theory  concerning  interactions  was  being 
used  for  the  course.  This  information  was  distributed  by  means  of  the  human  subjects  consent  forms  used  to  obtain  participant 
permission  to  have  their  online  data  be  used  in  the  study.  Sixteen  out  of  the  nineteen  students  in  the  course  agreed  to  participate 
in  the  study. 

Data  was  collected  from  multiple  sources  including  two  Web-based  surveys,  interviews  and  observations  via  document 
analysis  of  the  course  transcripts  of  communication,  learner  reflections  and  completed  assignments.  The  first  Web-based  survey 
was  distributed  after  the  first  half  of  the  course  and  the  second  after  the  conclusion  of  the  course.  Telephone  interviews  were 
conducted  after  the  conclusion  of  the  course  as  well.  Eleven  students  completed  the  first  survey  and  thirteen  completed  the 
second.  Eight  students  participated  in  telephone  interviews  after  the  course  ended.  Both  instructors  were  interviewed  half  way 
through  the  course  and  again  at  the  end  of  the  course.  In  addition  to  the  surveys  and  interviews,  transcripts  of  the  discussion 
forums,  class  e-mail,  course  assignments  and  feedback,  learner  reflections  and  chat  transcripts  from  the  Web  based  course  were 
downloaded  from  the  course. 

Data  Analysis 

Data  analysis  involved  data  reduction,  data  display,  and  conclusion  drawing  (Reigeluth  & Frick,  1999).  First,  the  researchers 
worked  independently  to  identify  themes  based  on  methods  and  strategies  of  the  instructional  design  theory.  They  then 
categorized  these  themes  by  the  instructional  method  or  strategy  for  which  it  was  intended  (English,  1992).  After  closely  reading 
the  data,  the  two  researchers  reviewed  the  categorized  data  to  identify  which  method  or  strategy  had  received  the  most  comments. 
These  comments  were  then  examined  by  both  researchers  to  identify  aspects  of  the  method  that  were  receiving  comments  by  the 
participants. 

The  findings  from  the  data  were  then  displayed  by  placing  the  categorized  data  into  a matrix,  which  illustrated  relevant 
situational  characteristics.  Finally,  conclusions  were  drawn  regarding  specific  recommendations  for  theory  improvement. 

Findings 

Interaction:  Learner  to  Self 

In  the  beginning  of  the  course,  the  learners  were  asked  to  share  their  expectations  of  the  course,  to  post  an  initial  personal 
theory  of  learning,  to  identify  criteria  for  usefulness  of  learning  theories  and  to  post  initial  reactions  to  general  statement  about 
learning.  The  goals  of  these  activities  were  to  support  learner  control  and  self-regulation  as  well  as  to  activate  prior  knowledge. 
The  data  revealed  that  these  adult  learners  have  rich  resources  of  prior  experiences  to  their  learning.  These  adult  learners  valued 


57 


248 


instruction  that  is  personally  relevant  to  them.  It  seems  to  be  very  important  for  the  instructor  to  identify  the  prerequisite  skills, 
knowledge  and  the  needs  of  individuals  before  and  during  the  instruction. 

As  learning  activities,  learners  completed  individual  as  well  as  team  thought  activities  requiring  application  of  the  learning 
theories  to  practice.  The  learners  were  also  asked  to  participate  in  whole  class  discussions  on  the  unit  topics.  The  goals  of  these 
activities  were  for  the  learners  to  discover,  explore,  clarify  and  negotiate  understanding  as  well  as  enhancing  elaboration  and 
retention.  Findings  showed  that  the  learners  were  interested  in  discussions  and  assignments  that  were  relevant  to  their  lives  and 
needs. 

For  reflection  on  learning,  the  learners  were  required  to  complete  self-reflections  after  each  project.  They  were  asked  to 
reflect  upon  what  resources  they  used  to  complete  the  projects  and  how  their  understanding  changed  during  the  unit  activity.  At 
the  end  of  course,  learners  revised  the  initial  personal  theory  of  learning  and  instruction  from  the  first  week  and  also  participated 
in  a final  reflective  discussion.  These  activities  supported  learner  control  and  self-regulation  as  well  as  providing  closure.  They 
provided  opportunities  to  create  a synthesis  framework  in  which  to  place  the  theories  and  the  self-reflections  were  helpful  in 
fostering  metacognitive  skill  development 

Interaction:  Learner  to  Learner 

The  learners  participated  in  many  activities  throughout  the  course  building  teams  and  increasing  social  participation  and 
engagement  with  learning  process.  For  orientation,  an  informal  icebreaker  activity  provided  a chance  for  learners  to  get  to  know 
each  other.  Everyone  in  class  including  the  instructors  was  required  to  post  eight  nouns  that  best  described  themselves.  The  three 
most  commonly  mentioned  words  in  this  activity  were  1)  teacher/educator,  2)  leamer/student  and  3)  parent/grandparent.  The  high 
frequency  of  these  words  revealed  the  important  characteristics  of  this  group  of  adult  learners:  many  of  them  were  involved  in 
authentic  situations  of  learning  and  instruction  in  their  everyday  lives.  The  class  was  also  asked  to  post  their  personal  profiles  and 
digital  photographs.  This  helped  building  a sense  of  community. 

One  of  the  main  activities  for  fostering  learner  to  learner  interaction  was  collaborative  team  tasks  for  the  unit  assignments.  By 
providing  choice  in  the  scheduling  of  collaborative  activities  as  well  as  the  number  of  group  activities  to  participate  in,  the  course 
supported  learner  control  and  self-regulation.  Because  these  adult  learners  had  extremely  busy  schedules,  the  learners  felt  that 
two  weeks  should  be  the  minimum  time  frame  for  collaboration.  Having  project  deadlines  on  Mondays  proved  to  be  helpful  for 
these  learners  since  it  provided  the  weekends  for  them  to  complete  the  tasks.  For  teamwork,  the  learners  were  required  to  provide 
feedback  on  the  contribution  of  team  members  after  the  completion  of  projects.  It  served  the  purpose  of  providing  feedback  and 
closure  as  well  as  building  the  sense  of  a team.  For  the  tools  the  team  used,  they  used  different  tools  for  different  purposes.  Most 
used  initial  chats  for  the  decision  making  process  and  found  them  very  useful.  Twelve  out  of  13  students  indicated  in  the  second 
survey  that  they  were  satisfied  with  the  amount  of  student -student  interaction  in  the  course.  Only  eight  out  of  13  indicated  they 
were  satisfied  with  the  amount  of  student-instructor  interaction.  Nine  out  of  13  chose  to  work  in  groups  again  for  the  second  half 
of  the  course.  Of  the  four  who  did  not  choose  to  work  in  groups,  two  participated  in  interviews.  They  indicated  that  their  reason 
for  not  working  in  groups  was  primarily  due  to  busy  work  schedules  and  that  they  wished  they  could  have  participated  more  in 
group  work. 

The  learners  were  also  asked  to  facilitate  the  whole  class  discussion  by  choosing  to  be  either  the  “facilitator”  or  the 
“wrapper”  for  the  unit  discussion.  This  activity  increased  the  engagement  with  the  learning  process  as  well  as  supporting  learner 
control.  Instead  of  having  instructors  responding  every  posting  to  the  discussion,  the  peer  facilitation  generated  a more  student- 
centered  atmosphere.  The  facilitator  role  presented  a new  perspective  each  week  and  some  of  the  facilitators  took  on  additional 
roles  in  the  discussion,  such  as  the  devil’s  advocate,  to  purposively  challenge  the  peers’  statements. 

Peer  to  peer  responses  were  also  encouraged  throughout  the  course.  The  learners  were  asked  to  comment  regularly  on  others’ 
postings.  Many  learners  felt  the  heterogeneity  of  experiences  in  the  class,  the  great  stories  and  advice  from  others’  experience 
was  tremendously  helpful.  They  said  the  multiple  perspectives  resulted  in  deeper  learning. 

Interaction:  Learner  to  Instructor 

The  primary  goals  of  learner  to  instructor  interaction  were  to  build  communication,  support  learner  control  and  self- 
regulation, and  to  provide  feedback.  They  also  appreciated  instructors  periodically  checking  in  on  teams’  progress  and  whole 
class  discussion.  At  times  when  the  topics  of  whole  class  discussions  would  drift  away  from  the  theory  being  discussed,  the 
instructors’  guidance  and  redirection  proved  to  be  helpful.  Learners  also  appreciated  instructors’  participation  for  the  sense  of 
“presence”.  The  learners  mentioned  that  they  benefited  from  instructor  feedback  that  focused  both  on  strengths  and  areas  of 
improvement.  In  Web-based  environments,  where  tone  is  not  apparent  in  writing,  increased  care  is  necessary  when  instructors 
formulate  criticisms. 

Interaction:  Learner  to  Resources 

Learner  to  resources  interactions  were  used  to  present  new  information,  increase  engagement  with  the  learning  process  as 
well  as  to  support  learner  control.  Because  learners  access  new  information  at  their  own  pace,  they  rely  heavily  on  the  course 
Website  and  resources  for  all  information.  In  an  online  environment,  it  proved  to  be  efficient  to  provide  additional  Web-based 
resources  to  supplement  the  text.  One  of  the  most  important  findings  was  that  learners  felt  that  having  models  or  examples  of 
expected  work  were  extremely  helpful  and  they  wanted  to  see  more  of  those.  They  pointed  out  that  they  really  liked  having 
examples  of  good  whole- class  discussion  postings  provided  at  the  start  of  the  course.  They  wanted  to  have  more  guidelines  for 
collaborative  assignments,  more  specific  grading  rubrics  and  more  advice  for  successful  online  collaboration.  The  findings 


showed  that  the  instructors  should  be  careful  not  to  make  many  assumptions  regarding  “Web-course  literacy”  on  the  part  of  the 
distance  students.  More  time  and  effort  were  required  than  in  residential  courses,  especially  for  the  first  time  distance  learners. 


Conclusion 

This  study  presents  the  guidelines  for  when  and  how  the  four  types  of  interaction  should  be  used  to  achieve  the  various 
learning  outcomes.  By  providing  a deeper  understanding  of  the  adult  learners  in  a distance  course,  this  research  indicates  that  the 
all  four  types  of  interaction  are  important  methods  of  instruction  in  a Web-based  learning  environment.  The  findings  showed  that 
the  adult  learners  appreciated  the  variety  of  assignments  and  a sense  of  learner  control  provided  by  a balance  between  flexibility 
and  structure  of  the  course.  Providing  instruction  relevant  to  the  prior  experiences  and  knowledge  of  these  adult  learners  also 
played  a great  role  in  fostering  interactions.  Given  the  appropriate  guidance  and  support,  the  learners  preferred  to  work  on  teams. 
When  the  experience  was  positive,  it  established  a sense  of  community  and  enabled  a deeper  level  of  situated  cognition. 

For  further  research,  it  would  be  useful  to  explore  the  dynamics  of  collaborative  works  in  Web-based  learning  environments: 
the  content  and  discourse  analysis  of  a successful  team’s  experience  as  well  as  criteria  for  good  participation  and  facilitation  in 
class  discussions.  Greater  insight  would  be  obtained  by  more  detailed  exploration  of  the  learning  outcomes  and  methods  of 
evaluation  specific  to  adult  learners  in  Web-based  learning  environments. 
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Abstract 

This  study  adopts  innovate  self-study  research  methods  (Bullough&  Pinnegar,  2001)  to  explore  the  effectiveness  of  teachers  ’ 
use  of  self-referenced  activities  designed  for  the  development  of  personal  teaching  portfolios.  Two  short  video  presentations, 
along  with  participant  and  instructor  workshop  guides  provided  necessary  instructions  and  materials.  Personalized  classroom 
design  and  photo  activities  were  completed  within  a teacher  education  workshop  aimed  at  developing  personal  portfolios.  Data 
collection  included  written/oral  activities  and  videotaped  group  discussions.  Four  criterion  selected  teachers  engaged  in 
activities  that  required  the  re-design  of  classroom  activities  and  the  use  of  personal  photos.  Results  indicated  that  all 
participants  were  able  to  successfully  redesign  conventional  classroom  activities  as  constructivist  classroom  activities.  Photo 
activities  were  found  to  be  motivating  and  relevant  to  all  participants.  Recommendations  offer  insights  into  future  constructivist 
learning  research,  the  potential  benefits  of  self-referenced  knowledge,  and  the  use  of  self-study  research  methods. 

Educational  reform  programmes  being  carried  out  throughout  North  America  are  aimed  at  offering  instruction  that 
accommodates  all  learners  and  highlights  lifelong  learning.  There  is  widespread  rejection  of  the  teacher  represented  as  one  who 
delivers  knowledge  to  passive  recipients.  Instead,  learning  is  viewed  as  a self-regulatory  process  involving  individual  meaning 
construction  and  processes  of  social  activity,  discourse,  and  debate,  (Twoney  and  Fosnot,  1996).  Current  reforms  derive 
theoretical  grounding  from  constructivist  learning  theory  (Lueddeke,  1999).  Based  on  constructivist  theory,  advocates  of 
educational  reforms  recommend  the  use  of  educational  portfolios  as  an  alternative  means  of  learner  evaluation  for  lifelong 
learning. 

Educational  portfolios  are  recognised  as  a multi-media  educational  tool  for  accommodating  all  learners  (Martin,  2000). 
Educational  portfolios  are  commonly  viewed  as  purposeful  collections  of  work  selected  by  learners  to  demonstrate  learning 
progress  and  achievements  over  some  period  of  time.  Portfolios  consist  of  on-going  work  efforts,  students*  reflections,  self- 
evaluations,  and  rationale  for  work  selected  that  make  it  up  (Cushman,  1999;  Daiker,  1992).  To  this  end  educational  portfolios 
have  been  identified  as  a useful  evaluation  tool  capable  of  informing  on  long-term  learning  growth  and  formative  evaluations. 
For  instance.  Durst,  et  al.  (1994)  consider  portfolio  evaluation  as  a **fluid  form  of  assessment  that  requires  negotiation  not  a 
calibrated  form  of  assessment  that  moves  away  from  absolute  judgements  about  writing  into  more  shaded,  nuance 
understandings  of  difference**  (p.  287).  This  view  of  assessment  captures  what  Cuba  and  Lincoln  (1989)  refer  to  as  'fourth 
generation  evaluation  ",  evaluation  which  explicitly  acknowledges  the  socio-political,  unpredictable,  and  constructivist  nature  of 
evaluation  as  a teaching  and  learning  process. 

Two  characteristics  of  educational  portfolios  highlight  their  widespread  appeal  to  the  educational  reform.  First, 
educational  portfolios  can  cater  to  a diversity  of  learner  styles  by  allowing  students  the  opportunity  to  document  learning  in  the 
media  they  are  most  comfortable  (Barrett,  1994).  Second,  portfolios,  specifically,  electronic  portfolios  allow  for  learners  to 
explore  multiple  media  (i.e.,  text,  graphics,  animation,  images,  audio  clips,  video  clips,  etc.).  This  has  importance  for  educators 
as  well  as  instructional  designers  because  of  the  educational  reform  *s  commitment  to  all  learners  and  the  educational  tools  that 
have  been  recommended  for  fulfilling  these  commitments. 

The  introduction  of  constructivist  informed  educational  portfolios  marks  a departure  from  conventional  educational 
methodology.  Informed  by  constructivist  theory,  this  study  is  framed  under  the  assumption  that  teachers  can  not  model  good 
educational  practice  when  they  have  never  had  the  opportunity  to  experience  and  relate  it  to  themselves.  The  rationalale  for  this 
study  is  structured  around  issues  in  multiple  media,  advances  in  image  and  art-based  inquiry,  and  the  signification  of  the  concept 
of  self-referencing  in  advancing  teacher  education. 

Literature  Review 

What  impact  has  media  research  had  in  education? 

Theoretical  grounding  for  multiple  media  (multi-media)  and  multiple  intelligences  comes  from  various  sources.  Symbol 
Systems  theory  posited  by  Salomon  (1974)  presented  arguments  based  on  media  research  to  explain  how  different  symbolic 
forms  of  representation  (e.g.,  pictorial,  verbal,  numeric,  graphic,  etc.)  require  that  individuals  use  different  mental  processes. 
Salomon  (1997)  adds,  “Being  part  and  parcel  of  the  information  itself,  media’s  symbolic  forms  of  representation  influence  the 
meanings  one  arrives  at,  the  mental  capacities  that  are  called  for,  and  the  ways  one  comes  to  view  the  world”.  In  other  words, 
each  medium  conveys  content  via  inherent  symbol  systems  which,  in  turn,  affects  the  meaning  derived. 

Research  on  multiple  media  in  education  covers  a wide  range  on  topics  with  relatively  focused  objectives  according  to 
Salomon  (1977).  These  objectives  include:  (a)  to  test  instructional  effectiveness  of  a medium  or  technology,  (b)  to  investigate 
psychological  effects  of  media  and  technology  on  individuals,  and  (c)  to  enhance  the  practice  of  education  by  providing  and 
evaluating  media  and  materials.  The  greatest  challenge  in  multiple  media  research  has  been  to  convert  media’s  potential  to  fulfill 
educational  purposes.  Criticisms  in  the  field  come  from  opposition  to  media-based  instructional  aims,  such  as  instruction  in 
visual  literacy  and  media  awareness  (Snow,  1970:  Cassidy  and  Knowlton,  1985).  One  potential  problem  of  media  research  is  the 
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lack  of  solid  evidence  of  how  symbolic  forms  can  be  used  to  enhance  diverse  educational  practices.  This  is  a problem  of 
association  in  my  view. 

In  a similar  vein,  Gardner’  s Multiple  Intelligence  theory  (1983)  asserts  that  different  intelligences  represent  not  only 
different  content  domains  but  also  learning  modalities.  Under  this  framework,  multiplicity  in  cognitive  processes  gives  rise  to 
multiple  forms  of  representation  that  influence  individuals’  understanding  of  phenomena.  Recognition  of  multiple  forms  of 
representation  has  been  connected  to  diversity  of  personal  learning  styles  (Gardner,  1993).  There  is,  however,  a challenge  when 
dealing  with  intelligences  that  are  difficult  to  represent  (i.e.,  affect)  and  are  considered  only  placeholders  in  contemporary 
educational  psychology  research.  One  potential  problem  is  a lack  of  research  that  succeeds  in  drawing  out  mental  processes 
theorised  about  but  never  observed.  This  is  essentially  a problem  of  representation  in  my  view. 

Problems  of  association  and  representation  call  into  question  whether  or  not  different  symbolic  forms  that  call  on  different 
cognitive  capacities  affect  the  way  individuals  represent  the  world  to  themselves  in  some  meaningful  and  lasting  way  that  could 
provide  leverage  for  advancing  education. 

Can  image  and  art-based  inquiry  make  a difference? 

Image  and  art-based  techniques  are  becoming  increasingly  popular  in  the  field  of  education.  Image  and  art -based  techniques 
initially  used  for  therapy  are  becoming  part  of  a recent  trend  in  educational  research  (Davis  & Butler-Kisber,1999;  Prosser, 
1999),  Art -based  research  has  been  used  to  enable  individuals  to  externalize  conflicts  and  remove  mental  blocks  (Davis  & Butler- 
Kisber’s,l999).  The  authors  treat  the  collage  as  a functioning  form  of  analytic  memo  to  complement  other  forms  of  representation 
by  providing  a means  to  self-critique.  Cartoons  and  other  types  of  visual  images  have  been  used  to  reconstruct  and  reify 
perceptions  of  public  narratives  through  analysis  and  interpretation  (Warburton,  1998),  Warburton’s  own  semiotic  approach  to 
cartoons  treats  cartoons  as  public  pictures  or  cultural  artefacts,  defining  meaning  according  to  how  the  image  is  produced  and  the 
intended  use  of  the  image.  This  type  of  work  has  typically  been  used  to  enrichen  educational  practices  but  could  contribute  as 
well  by  extending  the  breadth  of  representational  forms  beyond  what  is  presently  studied  in  educational  research. 

In  another  domain,  image-based  techniques  have  offer  a unique  capacity  to  inform  on  processes  difficult  to  access  by  other 
means.  Prosser  and  Schwartz  (1998)  discuss  the  use  of  photographs  within  qualitative  research  to  probe  for  personal  knowledge: 
Through  our  use  of  photographs  we  can  discover  and  demonstrate  relationships  that  may  be  subtle  or  easily 
overlooked.  We  can  communicate  the  feeling  or  suggest  the  emotion  imparted  by  activities,  environments, 
and  interactions.  And  we  can  provide  a degree  of  tangible  detail,  a sense  of  being  there  and  a way  of  knowing 
that  may  not  readily  translate  into  other  symbolic  modes  of  communication,  (p.  1 16). 

Image-based  psychoanalytic  techniques  and  other  forms  of  memory  work  employed  in  therapeutic  and  transformative 
practices  also  extend  the  depth  of  representational  forms  beyond  what  is  presently  studied  in  educational  research  by  providing 
researchers  and  educators  with  useful  insights  on  how  to  overcome  problems  of  representation  by  uncovering  mental  processes 
previously  hidden. 

What  is  self-referenced  knowledge  and  its  role  in  learning  and  research? 

Entire  branches  in  philosophy  are  directed  to  issues  of  knowledge  representation,  reference,  and  meaning  formation.  Without 
digressing  into  lengthy  discussion,  representing  objects  and  states  of  affair  is  considered  a part  of  the  mind’s  general  capacity  to 
relate  an  individual  to  the  world.  Consequently,  the  knowledge  representations  individuals  are  capable,  in  what  context  they 
occur,  and  how  they  occur  are  instrumental  in  informing  on  learning.  Equally  important  is  where  representations  point,  there 
referential  ity. 

This  study  puts  forth  the  notion  that  what  distinguishes  image  and  arts  based  research  in  the  domain  of  education  lies  largely 
in  their  self-referencing  quality.  Self-referencing  is  employed  uniquely  here  to  describe  the  relation  of  the  representation 
perceived  by  the  individual  who  conveys  it  to  him/herself.  Unlike  many  learning  contexts  which  involve  the  construction  of 
knowledge,  acquiring  self-referenced  knowledge  also  implicates  the  learner’s  self-concept,  resulting  in  greater  possibilities  for 
conceptual  associations  to  be  made  and  higher-order  thinking  to  be  achieved.  Self-referenced  knowledge  involves  being  able  to 
recognise  oneself  in  what  one  does.  This  is  exemplified  by  answering  questions  like:  Does  this  experience  relate  to  who  I am  and 
how  is  this  experience  important  to  me  personally? 

How  does  self-referenced  knowledge  make  a difference? 

Failed  attempts  to  integrate  constructivist  oriented  educational  reforms  and  educational  portfolios  is  one  area  where  the 
absence  of  self-referenced  knowledge  is  having  an  influence.  Where  teachers  have  little  or  no  personal  experience  with 
educational  portfolios  or  their  assessment,  portfolio  training  materials  have  been  implemented  for  teachers  that  focus  on  the 
development  of  professional  knowledge  (Ruskin-Mayher,  1999;  Silva,  2000).  Results  have  indicated,  however,  that  this  fails  to 
get  at  personal  knowledge  and  skills  development  (Au,  2001b;  Frederick,  McMahon,  & Shaw,  2000,  Silva,  2000),  Wadlington  & 
Partidge  (2000)  comment  “Before  teacher  educators  can  ask  preservice  teachers  to  use  journals,  self-assessment,  peer 
conferences,  portfolios,  observations,  and  so  forth,  they  must  first  model  these  techniques  themselves.”  Similarly,  Cushman 
(2000)  speculates,  “ What  if  educators  presented  portfolio  evidence  of  their  own  learning  and  growth?  What  if  they  tried  to  show 
in  concrete  ways  how  that  growth  affects  student  learning?  Doing  so,  many  are  coming  to  believe,  might  shed  new  light  on  some 
of  the  most  intractable  questions  in  the  current  debate  about  school  change.” 

Consequently,  there  are  a number  of  questions  that  have  not  been  flilly  explored.  First,  do  teachers  perceive  a need  for 
personal  development  to  be  able  to  successfully  adopt  this  new  form  of  classroom  instruction?  Second,  can  the  participation  of 
self-referenced  practices  advance  teachers’  understanding  and  comfortability  with  current  educational  reform  policies? 
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What  is  missing  from  teacher  education? 

The  study  posits  that  teacher  education  should  not  only  cultivate  professional  expertise,  but  also  self-referential  expertise. 
Understanding  ourselves  as  educators  is  of  fundemental  importance.  Being  able  to  explain  our  reasons  for  doing  something  or 
choosing  this  way  over  another  brings  additional  meaning  to  what  we  are  doing  (Connelly  & Clandinin,  1999,  p.  1 1).  Portfolio 
training  materials  do  exist  for  teachers  that  focus  on  the  development  of  professional  knowledge  (Ruskin-Mayher,  1999). 
Professional  knowledge  is  comprised  largely  of  step  ly  step  instructional  procedures  for  teaching  reflective  processes,  self- 
assessment,  and  peer  reviewing  within  a classroom  setting,  along  with  sample  activities,  and  evaluation  rubrics.  This  does 
articulate  the  importance  of  personal  experience  and  skills  development  considered  essential  to  effective  instruction  (Silva, 
2000).  How  can  teachers  engage  in  reflective  processes  and  self-evaluate  when  they  have  never  had  the  opportunity  to 
experience  it  themselves  in  their  own  learning?  How  can  teachers  model  good  educational  practices  when  they  have  not 
personally  experienced  what  they  are  attempting  to  model? 

Rationale 

The  rationale  for  conducting  the  following  study  stems  from  longstanding  work  in  image  and  art -based  clinical  research 
exploring  psychological  processes  not  accessible  by  other  research  methods  through  self-exploratory  techniques  (Weiser,  1993). 
Weiser  (1993)  demonstrates  the  success  of  photo-therapy  techniques  in  helping  individuals  to  represent  personal  knowledge  not 
directly  assessable.  Such  therapeutic  techniques  offer  innovative  avenues  of  exploration  through  their  power  to  use  images  to 
invoke  personal  knowledge.  In  addition,  art  and  image -based  approaches  to  learning  offer  the  potential  to  leverage  educational 
reform  efforts  by  contributing  research  knowledge  of  multiple  representational  forms.  To  this  end,  the  self-referenced  knowledge 
approach  has  been  developed  to  contribute  to  teacher  education  in  a way  consistent  with  advocated  reform  ideas.  This  innovation 
was  tested  by  evaluating  its  effectiveness  in  improving  teachers’  attitudes  and  abilities  to  apply  constructivist  concepts  and 
portfolio  processes  to  professional  practices.  The  self-referenced  dimension  of  learning  was  highlighted  throughout  the  workshop 
instruction  by  appealing  to  the  power  of  personal  meaning  formation,  and  self-reflection  through  personal  photo  use  and  its  role 
in  constructivist  informed  learning  practices. 

The  main  objective  was  to  identify  challenges  and  determine  whether  self-referenced  knowledge  can  be  an  effective  teacher 
education  intervention  for  advancing  expertise  and  improving  attitudes  of  constructivist  informed  educational  portfolios.  To  this 
end  photo-therapy  techniques  were  adapted  to  a teacher  education  workshop  aimed  at  developing  personal  portfolios  by:  1) 
appealing  to  the  power  of  personal  experience,  and  2)  exploring  the  use  of  self- referenced  knowledge  to  advance  teacher 
expertise. 

Method 

The  study  conducted  was  a qualitative  case  study  of  a teacher  workshop.  Merriam(  1 998)  defined  case  study  as,”  an 
intensive,  holistic  description  and  analysis  of  a single  instance,  phenomenon,  or  social  unit”  (p.  21).  The  study  focused  on  the 
experience  of  learning  from  the  integration  of  participants’  perspectives  and  is  based  upon  a constructivist  orientation  to 
qualitative  research.  The  guiding  premise  for  conducting  the  case  study  was  that  understanding  arises  most  meaningfully  through 
open-ended  activities  and  ongoing  exchange  between  all  participants.  Qualitative  methods  used  to  gather  information  follow  a 
constructivist  paradigm  aimed  at  reflecting  the  multiple  perspectives  constructed  by  those  involved  in  the  inquiry,  including  the 
researcher  (Lincoln  & Cuba,  1985).  This  method  was  chosen  for  its  capacity  to  provide  in  depth  evaluations  for  generating 
holistic  lifelike  descriptions,  illuminating  meaning,  and  communicating  tacit  knowledge  (Cuba  and  Lincoln,  1981,  p.  375).  This 
type  of  description  is  especially  useful  when  dealing  with  contextually  sensitive  image-based  data: 

Photographs  get  meaning,  like  all  other  cultural  objects,  from  their  context.  Even  paintings  or  sculptures, 
which  seem  to  exist  in  isolation,  hanging  on  the  wall  of  a museum,  get  their  meaning  from  a context  made  of 
what  has  been  written  about  them,  either  in  the  label  hanging  beside  them  or  elsewhere,  other  visual  objects, 
physically  present  or  just  present  in  viewers’  awareness,  and  in  discussions  going  on  around  them  and  around 
the  subject  the  works  are  about.  (Becker,  1998,  p.  88). 

The  Role  of  the  Researcher 

Prosser  and  Schwartz  (1998)  indicate,  “Before  qualitative  researchers  begin  to  mine  a site  for  the  data  it  holds,  we  need  to 
consider  how  we  present  ourselves  to  our  subjects”  (p.  119).  The  researcher’s  in  data  collection  is  a participant -researcher 
perspective,  where  workshop  activities  include  researcher  participation  and  data  collection  includes  the  researcher’s  perspective 
as  well  as  those  of  the  other  participants.  This  insider  approach  creates  an  open  and  sharing  environment  for  discussion.  The 
methods  of  data  collection  chosen  are  designed  to  empower  participants  and  foster  a sense  of  personal  ownership  over  data. 

Data  Collection 

Participants  in  this  study  were  educators  who  have  taught  in  Quebec.  Four  participants,  not  including  the  participant- 
researcher,  were  used  in  this  study  (N=4).  The  number  of  participants  falls  into  the  acceptable  range  for  field  testing  of 
instructional  workshops.  The  study  was  carried  out  at  Concordia  University  in  the  spring  of  2001.  Data  was  collected  directly 
from  the  participant  researcher  and  other  participants  through  the  following:  Teacher  Portfolio  Questionnaires  (TPQ),  group 
discussions  and  oral  activities  transcribed  from  videotape,  and  group  writing  activities. 


254 

263 


Validity  and  reliability  of  instruments 

In  qualitative  case  studies,  validity  and  reliability  issues  are  dependant  on  the  researchers  ability  to  convey  the 
trustworthiness  of  research  findings  and  the  researcher's  success  at  providing  a convincing  description  of  the  action  or  event 
taking  place  (Merriam,  1998,  p.  198).  Firestone  (1987)  points  out,  "The  qualitative  study  provides  the  reader  with  a depiction  in 
enough  detail  to  show  that  the  author's  conclusion  'makes  sense"  (p.l9).  Internal  validity  refers  how  much  the  research  findings 
match  reality  but  reality  is  holistic  multidimensional,  ever-changing,  researcher  influenced  under  a qualitative  research 
framework  (Merriam,  1998,  p.  202).  According  to  Lincoln  and  Cuba  (1985)  human  beings  are  the  primary  instrument  of  data 
collection  and  measuring  internal  validity  depends  how  closely  reality  can  be  approximated  by  close  observation  and  inquiry. 

Following  Merriam  (1998),  multiple  steps  were  taken  to  ensure  a high  degree  of  internal  validity  in  the  present  study: 
triangulation,  participatory  research  method,  inter-rater  reliability.  First,  multiple  sources  of  data  collection  were  used  to  provide 
an  adequate  diversity  of  finances  to  triangulate.  Second,  the  reseearcher  participated  in  the  study  and  all  activities.  Third,  inter- 
rater reliability  was  employed  by  having  a selections  of  activity  responses  scored  by  an  outside  scorer. 

Issues  of  reliability  were  also  important  considerations  in  this  study.  The  notion  of  reliability  is  viewed  as  the  consistency  of 
the  results  from  the  data  collected  (Lincoln  and  Cuba,  1985,  p.  288).  A high  degree  of  reliability  was  secured  through 
explanations  of  data  collection  and  interpretative  steps  (Lincoln  and  Cuba,  1981).  Finally,  efforts  were  made  to  provide  thick 
descriptions  and  participant  commonalties  (LeCompte  & Preissle,  1993)  in  order  to  allow  for  the  possibility  of  generalisations  to 
be  made. 

Data  Analysis 

Data  analysis  was  based  on  assumptions  stemming  from  a constructivist  framework  which  highlights:  the  importance  of  the 
individual  experience,  the  wide  range  of  perspectives  possible,  and  the  public  sharing  of  meaning.  Following  Maxwell  & Miller 
(1996)  two  types  of  qualitative  data  analysis  were  employed:  paridigmatic  and  syntagmatic.  Paridgmiatic  relations  are 
determined  based  on  their  similarity  or  difference  in  meaning,  whereas  syntagmatic  relations  are  concerned  with  the  relationship 
of  entities  within  the  context.  In  the  present  study,  questionnaire  and  activity  results  are  paridigmatically  coded  with  a themantic 
classificatons  derived  from  the  presentation  material.  Contextualizing  strategies  in  the  form  of  narrative  summaries  and  direct 
quatations  were  applied  to  discussion  and  oral  activity  data.  To  this  end,  I examined  perceived  abilities  and  attitudes  towards 
portfolios,  instructor  profile,  group  results  on  constructivist  design  activity,  group  results  on  photo  portfolio  activity,  and 
discussion  results.  In  addition,  I examined  my  own  beliefs  and  participation  in  the  study  conducted. 

Findings  and  Interpretations 

Teacher  Portfolio  Questionnaire  (TPQ) 

All  participants  received  the  Teacher  Portfolio  Questionnaire  (TPQ)  prior  to  the  workshop  field  test  to  complete  (Note:  all 
names  are  fictitious  to  preserve  the  identities  of  those  involved).  The  questionnaire  was  designed  to  assess  fevel  of  prior 
experience  with  and  attitudes  towards  portfolio  use.  Both  paridigmatical  and  non-paridigmatic  types  of  data  The  results  of  each 
participant  are  summarised  to  provide  participant  profiles: 

Len  is  a doctoral  student  in  an  educational  technology  program.  He  has  two  years  of  teacher  assistant 
experience  in  Quebec  and  in  currently  involved  in  project  collaborations  within  the  educational  reform.  He 
has  two  years  experience  using  art  and  writing  portfolios  and  enjoys  their  capacity  for  demonstrating  learning 
growth.  What  he  enjoys  least  is  how  poorly  they  are  been  used  for  evaluation  in  educational  settings. 

Bob  has  taught  for  35  years.  He  has  used  digital  and  photographic  portfolios.  He  perceives  his  own  abilities 
with  using  portfolios  in  teaching  as  ranging  from  excellent  to  fair.  What  he  likes  most  about  using  portfolios 
in  learning  activities  is  the  variety.  What  he  likes  least  is  that  they  are  difficult  to  evaluate  from  a criterion 
standpoint. 

Vicki  has  taught  for  2 years.  She  has  used  math,  art,  and  language  arts,  portfolios.  She  perceives  her  own 
abilities  with  using  portfolios  in  teaching  as  ranging  from  very  good  to  good.  What  she  likes  most  about 
using  portfolios  in  learning  activities  is  that  learners  can  see  their  own  work.  What  she  likes  least  is  that  they 
are  difficult  to  evaluate. 

Candice  has  taught  for  27  years.  She  has  used  math,  art,  and  language  arts,  portfolios.  She  perceives  her  own 
abilities  with  using  portfolios  in  teaching  as  ranging  from  very  good  to  good.  What  she  likes  most  about 
using  portfolios  in  learning  activities  is  that  learners  can  see  their  own  work.  What  she  likes  least  is  that  they 
are  difficult  to  evaluate. 

Betty  has  taught  for  15  years.  She  has  used  writing  and  art  portfolios.  She  perceives  her  own  abilities  with 
using  portfolios  in  teaching  as  ranging  excellent  to  very  good.  What  she  likes  most  about  using  portfolios  in 
learning  activities  is  that  they  are  flexible.  What  she  likes  least  is  that  they  are  time  consuming. 

All  participants  rated  their  ability  with  using  portfolios  as  excellent  to  fair.  Experience  with  portfolios  ranged  from 
participant  to  participant  from  the  types  used  and  the  number  of  years  using  them.  The  likes  and  dislikes  for  using  portfolios  in 
teaching  varied  from  participant  to  participant,  however,  three  of  the  four  participants  shared  a dislike  for  the  difficulty  of 
assessing  portfolios. 
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Instructor  profile  analysis 

Discussion  data  and  reflective  field  notes  from  the  instructor  were  analyzed  for  indicators  of  belief  and  attitude.  Categories 
extracted  include:  personal  belief,  optimist,  and  motivator. 

Personal  Belief  Len,  the  instructor  and  designer  or  the  workshop  field  test  had  his  own  views  on  life  and  learning.  He  believed 
that  no  amount  of  knowledge  would  compensate  for  the  need  to  understand  how  we  are  as  humans  whom  possess  a unique 
relation  to  the  world,  to  each  other,  and  to  themselves.  He  was  sensitive  to  issues  of  personal  meaning,  and  letting  people 
determine  their  own  educational  direction: 

Going  towards  some  method  of  learning  and  evaluation  that  everyone  can  get  something  out  of-something 
that  your  allowed  ownership  of  for  one  reason  or  another  that  you  can  develop  into  something.  Moving 
towards  portfolios  could  be  a really  good  way  to  accomplish  that  if  it  is  done  correctly  (discussion  extract 
from  Len). 

Optimist  During  the  workshop  instruction  there  was  some  apprehension  among  participants  as  to  what  professional  knowledge 
could  be  gained  from  engaging  in  personalised  learning  activities.  Participants  were  not  at  all  sure  how  teachers  could  both 
satisfy  personal  likes  while  making  sure  that  learners  acquired  necessary  formal  knowledge.  Len’s  approach  was  one  of 
optimism  and  openness: 

There  is  some  sort  of  formal  knowledge  that  is  important  to  acquire,  but  it  does  mean  that  you  have  to  do  it 
the  same  way.  It  would  be  interesting  if  everyone’s  activity  that  they  redesigned  suited  them  best  and  their 
style  and  the  things  that  they  like.  So  a lot  of  the  objectives  or  redefined  competencies  end  up  being 
accomplished,  but  in  a way  that  is  more  enjoyable  for  the  teacher  and  hopefully  more  enjoyable  for  the 
student  (discussion  extract  from  Len). 

Motivator  Len  found  himself  in  the  role  of  motivator  and  animator  in  the  workshop.  Given  the  open-endedness  of  activities, 
participants  were  unsure  of  the  appropriateness  of  their  responses.  This  was  revealed  through  comments  made  by  participants. 
Len  facilitated  the  discussion  by  having  participants  share  their  work.  He  attempted  to  provide  positive  feedback  by  summarising 
participants  work  and  highlighting  positive  points: 

That’s  kind  of  a wonderful  constructivist  activity.  They’re  engaged  in  a social  activity.  They  are  going 
beyond  the  information  given  discovering  things  maybe  the  instructor  didn’t  even  intend.  They  are  critically 
thinking  by  making  a evaluation  in  their  groups.  It  requires  them  to  work  together  individually  and  to 
communicate  effectively  in  a hospitable  and  democratic  fashion  (discussion  extract  from  Len) 

Together,  aspects  of  the  instructor’s  role  were  found  to  be  extremely  relevant  and  did  have  an  effect  on  the  participants  by 
being  optimistic  and  providing  motivation.  This  connects  with  attitudinal  and  ability  factors  identified  in  the  group  discussion 
(i.e.,  motivation,  perceived  integration  challenges,  and  transfer  of  learning).  One  possibility  is  that  the  instructor  played  a vital 
role  in  helping  participants  overcome  resistance,  rationalise  challenges,  and  re-create  their  learning  (learning  transfer). 
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Constructivist  Design  Activity 

All  four  participants  participated  in  an  activity  to  redesign  conventional  classroom  activities  as  constructivist  classroom. 
Findings  are  summarised  in  the  table  below: 


Table  I : Constructivist  Design  Activity 


Conventional  Activity 

Constructivist  Activity 

Conducted  a spelling 
bee  with  two  teams 

Redesign  of  Spelling  Bee. 

Instead  of  asking  them  to  form  competitive  groups,  I would  probably  have  them: 
-write  their  own  pieces  and  then  in  dyads  have  them  try  to  correct  their  spelling, 
-peer  proof  reading  using  dictionaries  and  asking  others 
-this  way  the  spelling  becomes  more  personally  meaningful.  (Candice) 

Lecture 

Redesign  of  Lecture: 

-Hand  out  typed  version  of  lecture  in  4 pieces. 

-request  each  student  make  a mind  map  of  the  words  (ideas)  and  turn  in  the  piece  they 
got. 

-duplicate  so  that  everyone  has  a complete  set.  (Bob) 

Teaching  math 
classification  on 
blackboard 

Redesign  Teaching  math  classification  on  blackboard. 

-Introduction  with  real  objects  that  are  familiar  to  kids  ( a shoebox  , marbles,  wood 
blocks  etc.).  Name  the  objects  and  divide  into  groups;  group  A ‘squares’  , group  B 
‘circles’  and  group  C ‘rectangles. 

-Groups  of  kids  go  on  a square,  rectangle,  or  circle  hunt  and  bring  objects  to  a specified 
spot  for  examination  and  decide  in  their  smaller  groups  whether  what  they  found  is 
accurately  classified. 

-Then  the  objects  are  examined  by  the  larger  group.  (Betty) 

Puzzle  Pieces 

5-yr  olds  put  together  a 
simple  puzzle. 

Redesign  of  Puzzle 

Provide  children  with  blocks  (3-D)  and  have  them  create  their  own  structures.  (Vicki) 

Teaching  Fractions 
I gave  examples  of 
math  from  text  book 
and  then  a 

demonstration  (divide  a 
plate  into  quarters) 

Redesigning  Teaching  Fractions 

-Bring  fruits  in  and  have  class  break  into  groups 

-Groups  cut  them  up  in  pieces  and  present  their  hypotheses  about  the  dividing  exercise 
and  come  up  with  a new  way  to  do  it  (the  fraction  exercise.).  (Len) 

Written  activities  were  rated  for  the  presence  of  constructivist  learning  competencies  delivered  h the  presentation.  The 
instructional  aims  at  the  beginning  of  the  workshop  were  to  demonstrate  three  constructivist  learning  competencies.  A scoring 
template  used  by  the  researcher  was  constructed  directly  from  the  workshop  material  to  score  the  presence  or  absence  of 
constructivist  competencies.  Findings  indicated  that  all  participants  incorporated  three  constructivist  learning  competencies  into 
their  written  responses. 


Table  2:  Percentage  of  Participant  Competencies  Achieved 


Participant 

Percentage  of  Competencies 

Bob 

100% 

Vicki 

100% 

Candice 

100% 

Betty 

100% 

Although  all  participants  were  able  to  fulfil  the  instructional  aims,  the  activities  themselves  were  not  perceived  as  easy.  None 
of  participants  found  the  workshop  overly  easy.  Comments  indicated  participants  found  the  activities  relatively  challenging:  “ 
This  is  hard  to  do  (Betty),”  ‘T  am  not  a conventional  teacher.  I am  finding  this  one  hard  (Candice),”  and  ‘This  is  tough 
(Vicki).”  This  can  be  interpreted  as  an  indication  participant  success  was  not  due  to  the  simplicity  of  the  activities  chosen. 
Another  possible  interpretation  is  that  participants  had  high  levels  of  personal  interest  in  the  material  and  were  highly  motivated 
to  learn. 
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Photo  Portfolio  Activity 

Completing  the  photo  activity  required  the  demonstration  of  the  following  portfolio  competencies:  selection,  reflection,  and 
peer  sharing.  Participant  responses  are  presented  as  direct  quotes  to  preserve  the  completeness  of  description  and  to  avoid 
possible  researcher  biases  that  arise  from  de-contextualizing  responses.  A scoring  template  used  by  the  researcher  was 
constructed  directly  from  the  workshop  material  to  score  the  presence  or  absence  of  portfolio  development  stages.  Findings 
indicated  that  all  four  participants  were  able  to  meet  the  workshop  competency  objectives  to  perform  three  competencies  of 
personal  portfolio  development  and  incorporate  three  stages  of  portfolio  development  into  personal  learning.  The  nature  of  the 
responses  was  unique  from  case  to  case. 

Candice* 

Candice  had  chosen  a photo  of  herself  sitting  on  a patio  on  a summer  afternoon.  The  patio  was  filled  with  guests  and  the  guests 
were  dressed  for  a formal  occasion.  In  the  photo  a man  was  turned  to  her  as  she  looked  in  his  direction: 

Candice:  ‘’One  of  the  things  I said  about  this  picture  in  my  own  comment  on  what  it  revealed  to  me  as  a learner  seemed  to  show 
my  intense  nature  There  is  a look  in  my  eyes  and  the  way  that  I am  sort  of  judging  and  gauging  things  and  reactions.  Now  can  I 
read  yours”  (Selection,  Reflection) 

Candice:  " One  of  the  things  Betty  said  was  it  reveals  awareness  of  ability  to  communicate  with  facial  expressions  so  she  saw 

something  in  my  face  as  well,  which  is  an  interesting  comment.  So  now  if  I wanted  to  write  more  about  that  picture  I would  use 
that  comment  which  I might  do.  . . ” (Sharing) 

Bob:  “ You  also  have  quite  a bit  of  body  language  in  there  ” (Sharing) 

Candice:  “ Yes  I do  and  I suppose  If  you  were  gonna  write  about  me  you  would  probably  mention  that.  ” “ In  the  upper  box 
Betty  said  that  this  person  looks  flabbergasted.  What  are  you  talking  about  is  that  lam  not  interested  and  it  is  true.  If you  look 
at  this  my  body  language  it  is  saying  I don’t  want  to  talk  to  you.  This  happens  to  be  at  a family  wedding  and  this  particular 
individual  I was  being  as  polite  as  I can  but  my  body  language  is  . . ” (Reflection) 

Candice’s  own  interpretation  is  expanded  when  Bob  picks  up  on  the  way  she  had  her  legs  crossed  in  the  photo.  This  was 

one  aspect  that  Candice  did  not  notice  and  something  that,  one  stated,  she  was  able  to  build  on  and  tie  into  another  participant’s 
written  comments  on  the  same  photo.  This  could  be  interpreted  as  a good  example  of  how  individual  reflection  can  be  expanded 
upon  by  others’  perspectives. 

Overall,  photo  portfolio  activity  findings  were  positive.  Interpretations  of  photo  portfolio  findings  revealed  the  following: 

1)  Individual  reflection  can  be  expanded  upon  by  others’  perspectives. 

2)  Individual  reflections  can  be  complemented  by  confirmatory  comments  from  others. 

3)  Photo  activities  are  not  exercises  that  are  detached  from  learners,  but  rather,  are  authentic  and  related  individuals  lives. 

4)  Personal  photo  activities  can  become  socially  motivating  and  relevant  to  others. 

5)  Photo  activities  can  acquire  a meaning  beyond  what  is  contained  in  the  photo  itself  and  connect  with  events  and  people 
independent  of  the  photo. 

Group  Discussion  Findings 

Discussion  results  were  transcribed  and  coded  thematically.  Categories  extracted  revealed  the  following  trends  in 
participants’  responses:  1)  motivation  2)  perceived  implementation  challenges,  and  3)  transfer  of  learning. 

Motivation  Findings  indicated  a lack  of  motivation  towards  the  constructivist  informed  educational  policies  currently  being 
implemented.  This  was  revealed  through  discussion: 

Then  you  can  have  constructivist  practices  or  constructivist  activities,  but  you’re  never  gonna  have  real 
constructivism  witch  is  why  in  my  opinion  it  makes  it  very  difficult  for  the  MEQ  to  say  we  want  everything 
to  be  constructivist  (discussion  extract  from  Candice). 

It  seems  to  me  that  probable  this  reform  is  not  legitimate,  that  for  a reform  to  be  legitimate  the  stakeholders 
have  to  discuss  it  and  come  to  some  understanding  about  it.  Then  you  can  bring  it  about.  If  you  just  have  top 
down  imposition  of  something.  If  people  have  low  power  they  will  comply  with  it  superficially.  If  they  have 
high  power  they  will  go  on  strike  or  something  like  that.  Unfortunately  the  teachers  are  mostly  in  a low 
power  situation.  However  the  parents  are  not.  The  parents  are  in  a high  power  situation  so  they  can  effect  it 
(discussion  extract  from  Bob). 

Implementation  challenge  Findings  also  indicated  a perceived  implementation  challenge  for  widespread  constructivist 
informed  educational  policies: 

I just  have  a question  about  the  constructivist  Ideas  for  education.  I think  the  biggest  challenge  for 
constructivist  philosophy  is  the  parents.  They’re  going  to  come  into  the  classroom,  and  they  want  to  know 
what  is  going  on  here.  You  have  kids  running  around  the  place  doing  whatever  they  want.  I am  taking  my  kid 
out  of  this  school  and  putting  them  in  another  school.  I can  see  the  parents  really  reacting  that  way  and  how 
do  you  handle  that.  I think  we  have  to  put  the  parents  in  there  somehow  (discussion  extract  from  Betty). 
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Transfer  of  Learning  Discussion  results  indicated  a transfer  of  learning  from  the  constructivist  activity  to  the  portfolio 
activity.  There  were  instances  where  the  participants  integrated  attributes  into  the  personal  photo  activity  that  drew  from  social 
and  critical  constructivist  characteristics  experienced  in  the  first  activity:  This  was  revealed  through  discussion: 

The  other  factor  is  the  photographer.  The  decision  a photographer  makes  about  how  to  frame  You,  where  to 
put  you  in  a picture.”  ” I am  just  trying  to  say  that  depending  on  the  photos  you  choose,  some  photos  do  not 
necessarily  have  that  capture  of  you.  So  if  you  went  to  a professional  photographer  to  get  a nice  head  shot  for 
a movie  you  are  not  going  to  look  like  you.  So  if  I give  a picture  on  how  someone  can  dress  up  a photo  of  you 
the  feed  back  is  really  about  the  photo  too.  You  choose  it  but  there  is  something  about  the  photo  that  you 
have  to  factor  in  (extract  from  Betty). 

Also,  the  constructivist  characteristic  of  communication  and  morality  became  a part  of  the  photo  activity  as  exemplified  by 
Bob’s  concern  ” This  is  an  intrusion,  I mean  when  I write  something  like  that  to  some  extent  that  is  an  intrusion  into  Betty’s 
world  so  we  have  to  have  a situation  where  there  is  a reasonable  trust  or  in  this  case,  the  okay  when  someone  can  intrude  in  on 
my  world.” 

The  categories  that  emerged  from  the  analysis  can  be  best  interpreted  by  recognising  their  attitudinal  and  ability  components. 
Accordingly,  participant  motivational  factors  and  perceived  implementation  challenges  can  be  interpreted  as  attitudinal 
indicators.  Scepticism  concerning  the  MEQ’s  capacity  to  succeed  and  questioning  the  legitimacy  of  the  educational  reform  can 
be  interpreted  as  attitudinal  indicators  of  resistance.  Similarly,  a perceived  lack  of  parental  involvement  and  teacher  ownership 
can  be  interpreted  as  attitudinal  indicators  of  rationalisation.  Participant  ability  factors  can  be  interpreted  as  ability  indicators. 
Being  able  to  integrate  aspects  of  the  constructivist  activity  into  the  personal  photo  activity  can  be  interpreted  as  an  ability 
indictors  of  recreation. 

Together,  the  attitude  and  ability  indicators  reveal  a different  set  of  threes  R’s  than  is  traditionally  associated  with  education: 
resistance,  rationalisation,  and  re-creation.  The  clearest  interpretation  from  a constructivist  standpoint  would  be  to  treat 
resistance,  rationalisation,  and  re-creation  as  developmental  steps  in  the  learning  process.  This  is  consistait  with  contemporary 
models  of  professional  development  where  new  learning  is  first  met  with  resistance,  followed  by  a number  of  additional  steps 
required  before  accommodation  can  take  place. 

Educational  Implications 

Contribution  to  educational  theory 

In  the  present  context,  individual  participants  demonstrated  resistance,  rationalisation,  and  re-creation  processes  with  a 
community  of  learners.  This  draws  attention  to  the  importance  of  providing  teachers,  as  advanced  learners  and  professionals,  the 
opportunity  to  discuss  personal  challenges  and  raise  issues  so  that  some  sort  of  rationalisation  can  be  achieved.  This  furthers 
constructivist  theory  in  education  by  identifying  the  impact  of  personal  and  social  factors  within  learning  contexts. 

Contribution  to  educational  research 

Having  the  opportunity  to  discuss  personal  challenges  and  raise  issues  with  others  could  have  considerably  impact  on 
individuals’  capacity  to  integrate  new  learning  (re-create).  This  provides  empirical  evidence  that  self-referential  knowledge  does 
impact  on  professional  development  interventions  carried  out  within  groups.  The  fact  that  personal  photo  activities  were  found  to 
be  motivating  and  relevant  offers  insights  into  constructivist  learning  research,  and  the  use  of  self-study  research  methods,  which 
have  attracted  much  recent  attention  in  the  educational  literature  (Bullough  & Pinnegar,  2001).  Future  research  will  consider  how 
patterns  of  resistance,  rationalisation  and  re-creation  occur  in  different  professional  development  contexts. 
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Introduction 

Digital  technology  has  turned  a new  page  for  TV  Broadcasting.  The  convergence  of  television  and  computer  brought  about 
powerful  effect  to  TV  viewing  experiences.  Not  only  in  the  TV  evolution,  but  also  in  the  distance  education,  digital  broadcasting 
combined  with  the  Internet  is  conceived  as  a new  driving  force  that  will  revolutionarily  change  the  mode  of  learning  in  the  very 
near  future. 

ITV  program  and  Web-based  instruction  have  appeared  to  be  the  most  common  ways  of  distance  course  delivering  in  the  past 
one  decade.  Each  of  them  has  different  attributes  that  meet  different  learners’  need.  However,  many  educators  believe  that  the 
interaction  and  two-way  communications  that  are  required  for  all  learning  are  not  as  effective  in  these  two  modes  as  in  the  face- 
to-face  instructional  format  due  to  the  limited  potential  of  the  existing  technologies.  Poor  or  time-buffered  images,  low-speed 
transmissions  and  high  costs  are  part  of  the  obstacles  that  were  experienced  by  educators  in  embedding  interactivity  in  the 
distance  education  program. 

Many  educators  perceive  that  Digital  Television  and  technology  would  possibly  improve  the  current  state  of  interactivity  in 
the  distance  education  due  to  its  advanced  features.  High  resolutions  and  digital  transmission  would  make  the  communication 
much  more  clear  and  easier.  One  possible  use  of  Digital  television  in  education,  as  Garvin  (1998)  described,  would  be  enhanced 
TV,  which  supplemented  by  multimedia  content  to  provide  interactivity  for  learning  needs.  Various  contents  such  as  web  pages, 
quick  time  movies  and  text  scripts  can  be  composed  into  the  framework  of  course  design.  However,  DTV  technology  is  still  in 
an  early  developing  stage.  There  is  insufficient  information  and  few  studies  had  been  done  for  educators  for  pedagogical  uses.  It 
would  be  necessary  to  investigate  those  potential  effects  DTV  could  bring  to  distance  education,  and  what  learning  aspects  DTV 
would  contribute.  This  review  will  explore  DTV’s  attributes,  discuss  some  possible  applications  for  interactivity  and  the 
underlying  theoretical  and  research  bases. 

Discussion 

The  basic  job  of  distance  educators  is  changing  adult  behavior  (Verduin  & Clark,  1991).  According  to  the  perceptual  theory 
of  psychology  (Bills,  1959;  Combs,  1959),  how  individuals  view  people  and  the  objects  and  events  in  their  environment  will 
greatly  influence  how  they  behave.  Verduin  and  Clark  (1991)  further  explained  the  theory  that  “Behavior  is  a function  of 
perceptions.  Perceptions  can  be  consciously  or  unconsciously  constructed  through  several  interactions  with  the  environment.”  “It 
is  through  this  construction  that  they  tend  to  display  behavior.”  Environment  is  critical  to  distance  learners  for  behavioral 
changes,  so  that  learning  may  occur.  In  distance  education,  environment  relies  heavily  on  the  technologies.  Through 
technologies,  distance  learners  are  able  to  interact  with  environment,  communicate  with  the  instructor  and  other  students.  In  his 
“guided  didactic  conversation”  (1989)  theory,  Holmberg  proposed  seven  background  assumptions  for  Interaction  and 
Communication.  He  believed  that  the  core  teaching  consists  of  interaction  between  the  teaching  and  learning  parties.  Emotional 
involvement  in  the  study  and  feelings  of  personal  relation  between  the  teaching  and  learning  parties  is  likely  to  contribute  to 
learning  pleasure.  Also,  a friendly,  personal  tone  and  easy  access  to  subject  matter  contribute  to  learning  pleasure.  Learning 
pleasure  supports  students  motivation,  and  strong  motivation  facilitates  learning. 

The  purpose  for  learning  is  to  be  proficient  in  a field  or  fields.  Knox  (1980)  in  his  theories  on  adult  education  suggests  that 
proficiency  is  a key  construct  and  is  actually  the  capability  to  perform  effectively  in  a given  situation.  This  capability  usually 
depends  on  some  combination  of  knowledge  (the  cognitive  domain),  physical  skills  (the  psychomotor  domain),  and  the  attitudes 
(the  affective  domain)  that  the  adult  processes.  Therefore,  to  enhance  an  adult’s  proficiencies  and  capabilities  to  perform  and  to 
facilitate  learning,  educator  must  place  the  three  major  domains  at  the  center  of  instructional  thinking  and  planning  for  adults 
(Cranton,  1989).  Cbnsequently,  to  build  adult  learners’  proficiency  in  the  distance  environment,  it  is  essential  that  the  three 
major  domains  achieved  by  the  students  through  an  interactive  manner. 

However,  the  limitation  of  technology  constrains  the  interaction  and  give-and-take  processes  of  learning.  When  interaction  is 
made  less  complete  due  to  loss  of  aural  and/or  visual  stimuli,  expressions  that  might  have  communicated  meaning  are  lost 
(Verduin  & Clark,  1991).  Besides,  difficult  access  to  subject  matter  discourages  leaming  pleasure  and  lessens  students’ 
motivation.  Many  efforts  have  been  put  on  the  improvement  of  technology -based  leaming  environment  to  produce  interactive 
instruction  and  leaming  activities.  Digital  broadcasting  comes  along  with  advanced  features  that  many  educators  aware  that 
might  bring  about  a leap  in  distance  education.  A discussion  of  these  features  from  the  perspective  of  Knox’s  (1980)  proficiency 
domains  and  interactive  leaming  is  followed. 
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Video/Audio  effects  in  learning 

Audio  accompanied  by  static  print  materials  has  demonstrated  to  be  as  effective  as  face-to-face  classroom  instruction  for 
step-by-step  procedural  tasks  (Wisher  and  Priest,  1998)  as  well  as  graduate  education  (Burge  and  Howard,  1990).  Similar  to 
audio  with  static  print  materials,  audio  graphics  transmits  a visual  image  accompanied  by  an  instructor’s  voice,  usually  through 
audio  conferencing  to  students  at  remote  sites.  The  students  do  not  see  the  instructor,  but  rather  attend  to  content.  Audio 
graphics  should  result  in  the  same  learning  outcomes  as  audio  with  print  materials  (Wisher  & Cumow,  1999).  Both  of  these  two 
forms  support  one  of  the  three  types  interaction  occurring  in  distance  education  as  described  by  Moore  (1989),  the  interaction 
between  student  and  content.  The  type  of  interaction  is  most  important.  In  a study  that  investigates  perceptions  and  effects  of 
image  transmission  during  Internet -based  training.  Wisher  and  Cumow  (1999)  indicated  that  there  are  categories  of  learning  for 
which  there  is  apparently  little  need  to  see  the  instmctor,  such  as  learning  declarative  knowledge  or  facts.  However,  there  are 
some  learning  tasks  that  instmctor’ s visual  clues  are  required  such  as  training  in  first-aid  procedure  or  communication  skills  such 
as  sign  language,  depending  on  visual  motion  cues  during  instmction  (Wisher  and  Cumow,  1999).  In  fact,  visual  images  of 
lesson  materials  and  instmctors  are  both  essential  depending  on  different  knowledge  learning.  Instmctional  or  instmctors’  image 
can  support  and  facilitate  students’  cognitive  learning  domain,  according  to  Bloom’s  systematic  way  of  viewing  cognitive  growth 
(Bloom  and  others,  1956),  from  knowledge  comprehension,  application,  analysis,  synthesis  to  evaluation. 

DTV  is  able  to  transmit  lesson  content  and  instmctor’ s image  as  well  as  motions.  Except  for  an  ordinary  image 
communication,  the  image  will  be  presented  in  an  aspect  ratio  of  16:9,  which  is  different  from  the  conventional  4:3.  The  screen 
is  formatted  more  rectangular  and  much  closer  to  the  way  people  see.  Image  fills  more  of  the  field  of  vision  and  has  a stronger 
visual  impact.  Generally,  publics  consider  that  a larger  and  more  rectangular  screen  is  preferred  by  most  of  the  audiences. 
Besides,  DTV  has  smaller  pixels  that  are  close  together,  so  there  are  more  pixels  in  a space  on  a screen  (4.25:1  compared  to 
NTSC  standard).  DTV  pixels  are  square,  just  like  most  computer  monitors,  are  able  to  remove  image  distortion  and  present  a 
high-resolution  image  (Cringely,  2001). 

But  in  what  degree  do  student’s  perceptions  and  preferences  toward  video  capability  relate  to  learning?  Research  findings 
from  instmctional  television  indicated  that  there  is  generally  no  significant  relationship.  In  a study  by  Greenhill,  Rich,  and 
Carpenter  (1962)  that  examined  preferences  for  screen  size,  students  are  divided  into  two  groups  and  assigned  to  small  or  large 
screen.  The  results  show  that  students  prefer  larger  screen,  but  their  preference  was  not  related  to  achievement  in  this  course.  In 
one  other  study  of  the  effect  of  visual  display,  Johnson  and  Stewart  II  (1999)  found  that  more  immersive  visual  display  had  no 
effect  on  gaining  special  knowledge.  However,  the  size  and  ratio  of  screen  might  have  impacts  on  students’  satisfaction  to 
learning  experience.  Students’  satisfaction  will  have  effect  on  their  affective  learning  domain.  Affective  domain  where 
receiving,  responding  and  valuing  (Krathwohl,  Bloom,  and  Masia,  1964)  foster  the  formation  of  attitude  toward  followed 
learning  behaviors. 

Audio  effect  had  been  found  influential  to  student’s  affective  learning  domain.  In  Kelsey’s  (2000)  study  of  the  interaction  in 
a course  delivered  by  interactive  compressed  video  technology,  the  most  significant  and  effective  barrier  to  interactions  was  the 
limitations  and  failure  of  ICV  technology.  Echoing  and  squealing  noises,  as  well  as  time  delays  and  disconnections,  inhibited 
interaction  between  guest  speakers  and  students.  Speech  that  was  mediated  through  ICV  technology  lacked  the  spontaneity  and 
lucid  flow  that  is  expected  during  face-to-face  conversion.  Students  reported  that  interacting  during  the  live  broadcast  was 
problematic  because  communication  delays  gave  speech  a choppy  and  unnatural  sensation. 

DTV  technology  adopts  Dolby  Digital/AG-3  audio  encoding  system  to  broadcast  sound.  The  sound  would  be  much  sharper 
and  crispier.  DTV  broadcasting  uses  digital  high-speed  transmission,  more  consistent  the  data  will  stay  over  distances.  Although 
both  analog  and  digital  signals  get  weaker  with  distance,  while  the  sound  and  picture  on  an  analog  TV  slowly  gets  worse,  the 
sound  and  picture  on  a digital  set  will  stay  perfect  until  the  signal  becomes  too  weak.  There  is  convincing  reason  to  speculate  that 
DTV  wil31  have  positive  effects  on  students’  affective  learning  domain  (Cringely,  2001). 

DTV  and  Interactivity 

Vrasidas  and  Marina  (1999)  in  their  study  of  factors  influencing  interaction  in  an  online  course  indicated  that  there  are  factors 
directly  influence  interaction,  including  learner  control,  transactional  distance,  feedback,  and  social  presence.  Among  the  three 
factors,  learner  control  is  central  to  the  notion  of  interaction.  Garrison  and  Baynton  (1987)  described  that  the  concept  of  control 
consists  of  three  components:  independence,  power,  and  support.  The  proper  learner  control  should  be  a balance  among  these 
three  components.  To  expand  more  precisely,  independence  is  the  degree  to  which  the  learner  is  free  to  make  choices.  Power  is 
the  abilities  and  competencies  of  the  learner  to  engage  in  a learning  experience.  Support  is  the  resources  available  that  will 
enable  the  learner  to  successfully  participate  in  the  distance  education  course.  Improvement  in  television  delivery  methods 
created  new  options  for  distance  learners  to  communicate  (Machtimes  and  Asher,  1996)  also  might  increase  the  capability  of 
learner  control. 

Currently,  HDTV  is  the  best-known  term  for  digital  broadcast  TV.  When  a show  is  digitally  broadcasting  on  the  television, 
the  audience  can  interact  with  the  content  of  the  show  online  and  get  immediate  feedback.  Yet,  the  potential  for  DTV  can  be 
greater. 

Datacasting 

According  to  the  Guidebook  to  DTV  developed  by  Harris  Corporation,  DTV  will  allow  broadcasters  to  deliver  ancillary 
digital  data  in  variety,  such  as  web  site  materials,  multimedia  content,  program,  and  non-program  related  information.  DTV  is 
also  capable  of  delivering  data  hundreds  times  faster  than  traditional  modem.  An  enormous  amount  of  content  could  be  sent 
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through  airwaves  without  requiring  the  user  to  subscribe  to  an  online  service  or  to  have  a wired  computer  connected  to  a high- 
speed Internet  connection.  These  are  things  that  traditional  broadcasting  can  never  accomplish. 

Digital  transmission  is  operated  in  a way  that  more  data  can  be  transmit,  and  more  types  of  data  could  be  carried  through  the 
same  amount  of  bandwidth  because  more  information  could  be  carried  in  a digital  signal  by  using  MPEG -2  encoder  to  compress 
data.  DTV  broadcasting  technology  has  one-to-many,  real-time,  high-speed  data-delivery  channels.  Any  program  created  by 
educators  for  delivery  over  digital  media  CD-ROM,  IP  web  stream  can  be  transmitted  to  any  individual  receiver  or  all  receivers 
in  the  coverage  area.  When  TV  stations  are  transmitting  less  demanding  signals,  greater  capacity  for  datacasting  becomes 
available  (Cringely,  2001). 

Multi-casting  ability 

Instead  of  broadcasting  in  a high  definition  mode.  Digital  TV  can  be  programmed  to  broadcast  in  standard  definition  mode 
while  it  multicasts  four  choices  at  the  same  time  and  the  same  channel.  Some  broadcasters,  including  many  PBS  stations,  have 
already  planned  to  multi-cast  four  choices  of  programs  during  the  day  and  then  switch  to  high -definition  for  prime-time.  This 
option  offers  more  choices,  make  viewing  experiences  more  interactive  (Cringely,  200 1 ). 

Interactivity?  Or  not? 

Corporation  for  Public  Broadcasting  (CPB)  conducted  a study  (1999),  “Will  TV  viewers  want  interactivity?”  to  evaluate 
prototypes  of  digital  public  television  programming.  According  to  CPB,  the  prototypes  were  designed  to  demonstrate  potential 
interactive  techniques  of  DTV,  and  to  get  reaction  from  viewers.  There  are  some  interesting  initial  findings.  They  are 
summarized  as  follows: 

Benefits  of  interactivity 

• Control:  Viewers  feel  that  interactivity  allows  them  to  have  more  control  over  what  and  how  much  they  wan  to  see  and 
learn  about  a particular  topic. 

• Additional  information:  At  the  most  basic  level,  people  want  interactivity  to  provide  more  detail  information. 

• Richer  viewing  experience:  Interactive  programs  are  perceived  to  be  more  engaging  and  providing  more  pleasurable 
experience. 

• Better  Education:  Almost  all  viewers  in  the  study  see  interactive  television  has  a vision  of  formal  and  informal 
educational  use  of  technology. 

Negative  impact 

• Distraction:  It  is  difficult  for  nx)st  people  to  concentrate  on  the  separate  information  streams  at  one  time.  It  is  easy  to 
lose  focus. 

• Weak  underlying  program:  Viewers  expressed  the  fear  that  producers  will  spend  too  much  time  on  the  interactivity 
material  and  reduce  the  quality  of  the  program. 

• Difficult  interface:  Viewers  get  frustrated  when  the  interaction  does  not  occur  the  way  they  would  like  them  to,  and  that 
different  programs  have  different  types  of  interface. 

• Make  television  too  much  work:  Viewers  have  a fear  that  complicated  interactivity  will  make  the  television  too  much 
like  computers  that  they  crash,  require  updating,  and  viewers  need  training  to  operate  the  television  rather  than  simply 
click  on  the  remote. 

Unanswered  questions 

After  the  study,  although  participants  see  great  potential  for  interactive  digital  television,  they  also  left  with  many  questions 
that  need  to  be  answered.  According  to  the  report  (CPB,  1999),  viewers  want  to  know  how  and  when  this  technology  will 
happen,  what  will  be  available,  what  equipment  they  will  need  to  get,  and  how  much  the  hardware  and  the  service  is  going  to 
cost.  They  said  that  they  need  to  get  clear  answers  to  these  basic  questions  before  they  can  make  intelligent  decisions  about  what 
hardware  to  buy,  what  service  to  choose,  and  when  they  should  start  the  process. 

Practical  obstacles 

As  the  results  of  the  prototype  study  show  that  there  are  potential  problematic  and  questionable  areas  in  the  technology  of 
digital  television.  Carvin  (1998)  also  summarized  several  issues  that  all  future  broadcaster  and  consumers  of  DTV  should 
consider  about  this  technology: 

• Cost:  At  the  beginning  of  any  new  type  of  technology,  the  cost  to  acquire  it  can  be  grant.  DTV  is  no  exception.  This 
may  put  consumers  to  a halt  when  buying  a piece  of  equipment  that  they  know  little  about.  Also,  consumers  may  feel 
reluctant  to  give  up  the  old  analog  television  that  they  already  have,  and  they  then  need  to  get  a converter  for  the  digital 
signals  to  decode  the  transmission  to  their  household.  Even  the  converter  can  be  expensive,  at  least  for  some  families 
that  need  it  most  to  afford.  The  price  for  all  DTV  equipment  may  go  down  someday,  but  not  in  the  most  immediate 
future. 

• Content  availability:  According  to  Carvin,  not  so  many  producers  are  actively  producing  materials  for  the  digital 
market,  but  a lot  of  them  are  taking  the  analog  TV  programming  and  retool  it  for  digital.  Therefore,  they  are  not 
producing  the  content  specifically  for  the  technology.  Also,  the  program  content  may  not  even  fit  with  the  new 
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medium.  If  the  consumers  pay  to  get  the  old  content  that  they  used  to  see  in  analog  TV,  and  to  find  that  there  is  not 
much  available  content  for  the  new  technology,  they  may  not  even  want  to  invest  their  money  on  it  in  the  first  place. 

• Interactivity:  Carvin  mentioned  that  though  digital  TV  signal  transmission  may  be  advanced,  one  thing  is  not  changed: 
DTV  is  still  a one-way  stream,  similar  to  the  traditional  analog  TV  because  DTV  spectrum  can  only  be  used  by 
licensed  broadcasters,  not  consumers.  It  means  that  in  order  to  have  interactivity,  DTV  needs  to  utilize  other  ends  to 
have  the  consumer  transmit  information  back  to  the  broadcasters.  So  far,  Internet  seems  to  be  the  primary  back  channel 
for  DTV  communication. 

• Commercialization:  Just  like  the  Internet,  when  a new  technology  starts  to  attract  and  become  accessible  to  a significant 
public  market,  commercialization  comes  into  view.  Also,  TV  networks  invest  huge  amount  of  money  into  the 
transition,  and  they  want  to  see  some  pay  back  from  the  market.  If  making  a profit  is  all  people  think  about  DTV,  they 
will  be  unlikely  to  put  educational  programming  as  a priority.  Even  if  Public  Television  takes  up  the  responsibility  to 
provide  educational  programming,  with  possible  high  cost  and  moderate  return,  how  long  can  they  last? 

Carvin  (1998)  made  no  mistake  in  saying  this,  “ There  is  a lot  of  potential  in  digital  television  but  there  are  also  a lot  of  excuses 
for  us  to  not  take  advantage  of  it.”  It  makes  one  think  more  about  how  this  new  technology  should  be  utilize  for  the  better  of  the 
society. 

Summary 

Digital  TV  is  the  most  recent  major  advance  in  television  broadcasting  in  nearly  48  years,  with  improved  audiovisual 
features,  and  supposedly  it  will  also  improve  students’  satisfaction  level  in  their  learning  experience.  However,  because  DTV  is 
relative  new  and  yet  to  be  accessible  to  majority  of  the  society,  little  research  has  been  done  to  see  its  effects  in  education.  The 
prototype  study  conducted  by  Corporation  for  Public  Broadcasting  did  yield  some  exciting  findings  and  implications  for  DTV 
program  developers  as  to  viewers’  feeling  of  being  in  control,  having  richer  viewing  experience,  encountering  weaker  content, 
and  trouble  dealing  with  complex  interface  functionality. 

David  Carvin  (1998)  also  proposed  several  issues  for  the  public  to  ponder  on  about  the  future  of  DTV.  There  is  first  the  cost 
of  hardware  or  service  subscription  that  may  scare  people  away,  then  the  content  availability  and  quality  for  DTV  programs, 
DTV  interactivity  back  channels,  and  the  possibility  of  major  commercialization  problem  for  DTV. 

In  May  of  1997,  Federal  Commission  of  Communication  mandated  that  the  United  States  begin  to  replace  standard  analog 
television  with  digital  television.  By  2002,  all  commercial  network  stations  need  to  be  finished  with  their  digital  process,  and  by 
2003,  public  broadcasters  need  to  get  the  job  done.  Viewers  will  have  the  options  to  continue  receiving  analog  broadcasting,  or  to 
purchase  a digital  television  or  a converter  between  2003  and  2006.  By  then,  stations  have  to  return  their  analog  licenses  to  the 
federal  government.  Between  now  and  then,  it  is  advisable  for  everyone  to  think  about  the  future  of  television. 
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Abstract 

Researchers  have  claimed  that  while  educational  computer  software  does  not  necessary  improve  students  ’ academic 
performance,  it  does  provide  for  students  a more  interesting  and  motivational  environment  for  learning.  It  naturally  fits  in  the 
context  of  students ' learning  since  it  can  deliver  nonstop  actions,  realistic  sounds  and  vivid  colors  to  get  students ' attentions.  It 
also  corresponds  to  the  ongoing  move  towards  more  student-centered  learning  (Ellington,  2000).  The  design  of  educational 
computer  software  adapts  games  and  simulations  to  help  students  learn  while  having  fun.  Although  nowadays  at  least  one 
computer  per  classroom  is  used  for  instruction  and  the  number  of  students  per  computer  is  decreasing,  teachers  are  facing 
challenges  to  utilize  educational  software  to  engage  students ' learning  due  to  the  lack  of  sufficient  technology  infrastructures,  the 
lack  of  proper  and  quality  educational  software,  and  the  lack  of  training  and  support  to  make  utilizations. 

This  paper  will  explore  if  educational  software  that  entertains  K-12  students  can  also  promote  learning  in  the  practical 
educational  context.  Also,  this  paper  is  to  investigate  the  emergent  phenomenon  of  whether  educational  software  works  as 
edutainment  to  facilitate  and  motivate  students'  learning,  and  whether  the  utilization  is  implemented  in  real  classroom 
instruction  without  obstacles.  The  ultimate  goal  of  this  paper  is  to  explore  the  emergent  phenomenon  of  utilizing  edutainment  to 
actively  engage  students  and  promote  students ' learning. 

Introduction 

The  increased  use  of  computers  in  education  comes  along  with  progressive  technology,  both  hardware  and  software.  It  is 
really  easy  to  discover  educational  software  targeting  at  K-12  students.  The  design  of  educational  computer  software  can  be  seen 
as  the  adaptation  of  the  attributes  of  games  and  simulations  to  help  students  learn.  The  manipulation  for  educational  purposes 
and  entertainment  is  remarked  as  one  kind  of  edutainment.  However,  we  do  not  know  specifically  how  students  are  likely  to 
interact  with  varieties  of  educational  software  in  terms  of  edutainment.  Is  it  for  educational  purp  oses  or  rather  for  leisure? 
Furthermore,  utilization  of  educational  software  in  classrooms  is  not  clinical.  The  exploration  of  teachers’  implementations  of 
educational  software  for  real  instruction  is  rare  reported  and  investigated.  It  is  uncertain  how  teachers  utilize  educational 
software  for  their  teaching.  The  focus  of  this  paper  is  a preliminary  analysis  of  at  what  degree  the  utilization  of  edutainment,  in 
terms  of  educational  software,  in  schools  is  taken  place.  We  are  particularly  interested  in  discovering  discrepancies  beyond  the 
emergent  phenomenon. 

Current  Status  of  Technology  in  K-12 

In  1996,  President  Clinton  articulated  a clear  vision  for  improving  21st  century  education  through  the  use  of  technology  in 
American  schools.  Defining  “Four  Pillars”  (U.S.  Department  of  Education,  1996)  as  part  of  his  Technology  Literacy  Challenge, 
the  President  called  for  broadening  educational  technology  objectives  to  include  not  only  hardware  and  connectivity,  but  also 
digital  content  and  professional  development.  The  “Pillar”  of  digital  content  asserts  that  effective  software  and  online  learning 
resources  can  increase  students’  learning  opportunities. 

Computers  in  K-12  public  schools 

In  fact,  with  student  populations  growing  (NCES,  1997)  and  many  buildings  and  facilities  aging  (U.S.  General  Accounting 
Office,  1995),  every  school  has  its  own  unique  priorities  that  compete  for  limited  resources.  All  schools  must  ensure  that  the 
drive  to  integrate  technology  does  not  supplant  the  fundamental  need  to  provide  all  students  with  basic  skills  such  as  reading  and 
math  although  technology  can  be  an  effective  tool  for  meeting  learning  objectives  (CEO  Forum,  1997).  In  1995,  reports  suggest 
that  nearly  60%  of  school  computer  purchases  were  used  to  replace  old  and  outdated  computers,  resulting  in  only  a marginal 
increase  in  the  number  of  machines  available  to  students  (QED,  1996).  In  the  1998-1999  school  year,  the  average  student  to 
computer  ratio  was  5:1  and  the  average  student  to  multimedia  capable  computer  ratio  was  10.1:1  (QED,  2000).  In  the  1999-2000 
school  year,  the  average  student  to  computer  ratio  was  5.4:1  and  the  average  student  to  multimedia  capable  computer  ratio  was 
9.6:1  (QED,  2001).  The  survey  results  for  school  year  1998-1999  and  school  year  1999-2000  were  very  close  showing  that  no 
significant  increase  of  computer  available  to  students.  The  differences  were  embedded  only  on  the  percentages  of  Internet 
connection  for  these  two  school  years.  In  year  1999,  95  percent  of  public  schools  and  64  percent  of  public  classrooms  are 
connected  to  Internet  and  have  Internet  access,  and  in  year  2000,  98  percent  of  public  schools  and  77  percent  of  public 
classrooms  are  connected  to  the  Internet  and  have  Internet  access  (CEO  Forum,  2001).  Teachers’  uses  of  technology  at  schools 
were  reported  as  well  but  mostly  stated  as  using  computers  daily  for  planning  and  teaching,  as  using  the  Internet  for  instruction. 
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and  as  using  ©mails.  Among  above  usages,  creating  for  instructional  materials  using  computers  accounted  for  the  major 
utilization  (CEO  Forum,  2001). 

Supports  and  training  for  teachers 

Scrogan  (1989)  reported  that  many  teachers  wanted  to  learn  more  about  technology  but  were  unable  to  find  satisfactory 
instruction.  To  overcome  this,  teachers  were  asking  school  systems  to  provide  more  relevant  and  consistent  training  and  support 
for  integrating  technology  into  curricular  (Becker,  1992).  A more  recent  survey  conducted  by  the  National  Center  for 
Educational  Statistics  (NCES,  1999)  indicated  that  less  than  20%  of  current  teachers  reported  feeling  very  well  prepared  to 
integrate  educational  technology  into  classroom  instruction  (Schrum,  1999).  In  general,  K-12  teachers  do  not  receive  enough 
time,  access,  support,  or  encouragement  to  become  comfortable  with  computers  (Siegel,  1995).  Recent  research  indicated  that 
although  teachers  are  eager  to  use  technology  for  professional  and  curricular  activities,  a lack  of  teacher-development  programs 
and  time  dedicated  to  experimentation  hinder  teachers’  drills  and  knowledge  (Schrum,  1995;  Schrum  & Fitzgerald,  1996). 
Teachers  have  indicated  that  their  greatest  barrier  for  use  of  technology  is  a lack  of  understanding  of  how  to  use  it  in  classrooms 
(Hancock  & Betts,  1994;  Becker,  1992).  Becker  (1992)  reported  that  teachers  who  had  more  experience  with  computers  were 
more  inclined  to  integrate  the  curriculum  across  subject  areas  and  use  computers  as  a tool  for  learning.  Koontz  (1992)  found  that 
teachers  with  some  experience  with  computers  had  more  favorable  attitudes  towards  technology  and  were  more  willing  to  use 
technology  in  their  classrooms.  Clearly,  access  to  technology  and  lack  of  ongoing  support  are  major  obstacles  for  educators 
interested  in  implementing  information  technologies  in  teaching  (Schrum,  1995).  Teachers  are  on  their  own  to  make  up  their 
decisions  whether  to  use  computers  to  teach  and  how  to  utilize  computers  into  lesson  activities.  Use  of  computers  in  instruction 
tends  to  be  by  individual  teachers. 

Adoption  of  technological  innovation 

Rogers  (1995)  offered  a significant  review  of  adoption  of  technological  innovation  and  found  that  the  adoption  depends  on 
the  potential  adopter’s  determination  of  five  criteria.  These  include  (a)  the  relative  advantage;  (b)  the  compatibility  with  personal 
values,  experiences,  and  needs;  (c)  the  complexity  of  use;  (d)  the  availability  for  experimentation;  and  (e)  the  observability  of 
results  to  others.  These  factors  all  require  attention  prior  to  the  initiation  of  the  innovation.  Moreover,  the  CEO  Forum  has 
developed  the  School  Technology  and  Readiness  Chart  (STaR  Chart)  to  provide  a clear  framework  for  assessing  how  prepared 
American  schools  are  to  meet  the  education  challenges  of  the  21  century.  The  STaR  Chart  describes  technology  presence,  use 
and  integration  in  a typical  school  in  four  school  profiles  ranging  from  the  “Low  Technology”  school  that  uses  technology 
primarily  for  administrative  functions,  to  the  “Target  Technology”  school  that  integrates  technology  throughout  the  curriculum 
(CEO  Forum,  1997).  Beginning  from  1997  and  for  each  of  the  following  three  years,  the  CEO  Forum  would  use  the  STaR  Chart 
as  the  Backdrop  for  an  assessment  of  how  ready  our  nation’s  schools  are  to  effectively  use  technology  to  enhance  teaching  and 
learning.  The  STaR  Qiart  is  available  on-line  and  is  handy  to  help  schools  and  teachers  accessing  their  readiness  and 
development  of  technology  for  the  current  and  future  status. 

Digital  content 

Digital  learning  is  an  educational  approach  that  integrates  technology,  connectivity,  content  and  human  resources.  When 
implemented  correctly,  it  builds  on  the  unique,  dynamic  characteristics  of  digital  content  to  create  productive  and  engaging 
learning  environments  (CEO  Forum,  2000).  Digital  content  can  be  randomly  accessed,  explored  on  many  levels,  interactive  and 
engaging,  manipulatable  and  creative.  There  is  more  about  digital  content  in  the  1997  STaR  Report  (CEO  Forum,  1997): 

The  digitization  of  information  has  led  to  more  dynamic  and  interactive  education  content.  Digitization  has  also  transformed 
the  way  educators,  parents  and  students  use  educational  content.  Not  only  can  information  now  be  packaged  by  traditional 
content  creators  in  new  and  exciting  ways  - software,  CD-ROMs  or  online  resources  - but  it  can  also  be  used  and  creatively 
re-packaged  by  teachers,  students  and  software  publishers.  In  addition,  new  tools  are  available  in  the  digital  age  allowing 
individuals  to  find,  organize  and  create  information  as  never  before  possible,  (p.  28) 

Assessing  the  degree  to  which  digital  information  has  been  incorporated  into  classrooms  can  only  be  accomplished  by  examining 
the  availability  and  use  of  digital  content  and  digital  learning  tools  (CEO  Forum,  1997).  The  report  of  CEO  Forum  (2000)  also 
indicates  that  people  have  placed  much  focus  on  hardware  and  connectivity  than  on  digital  content  and  digital  learning.  It  is  time 
for  people  to  refocus  on  utilization  and  development  of  digital  content  and  learning.  It  is  evidenced  to  tell  people’s  foci  from 
numbers.  Average  school  district  spent  11  dollars  out  of  121  dollars  per  student  on  instructional  software  in  the  1998-1999 
school  year  (QED,  2000).  Moreover,  statistic  data  often  omitted  identifying  kinds  of  instructional  software  in  practice. 

Instructional  computer  games  as  edutainment  to  students’  learning 

Educational  computer  software,  as  a form  of  highly  interactive  technology  and  a form  of  digital  content,  naturally  fits  in  the 
context  of  students’  learning  since  it  can  deliver  nonstop  actions,  realistic  sounds  and  vivid  colors  to  get  students’  attentions. 
Educational  computer  software  usually  incorporates  the  nature  of  games  and  simulations  into  its  design  and  development.  In 
other  words,  while  interacting  with  instructional  computer  games,  students  are  engaged  with  education  as  well  as  having 
entertainment.  The  integration  of  educational  purposes  and  entertainment  turns  the  educational  computer  software  into  the 
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Theoretical  framework  of  games  and  simulations  to  students’  learning 

Dempsey,  Rasmussen,  & Lucassen  (1994)  put  instructional  games  into  simulations,  puzzles,  adventures,  experimental  games, 
motivational  games,  modeling  and  others.  Instructional  games  sever  many  functions  such  as  tutoring,  amusing,  helping  to 
explore  new  skills,  promoting  self-esteem,  practicing  existing  skills,  drilling  existing  skills,  or  seeking  to  change  an  attitude. 
Research  has  shown  that  games  and  simulations  can  make  a significant  contribution  to  teaching  and  learning.  The  impact  of 
games  for  education  have  been  studied  since  1960’s  although  research  did  not  specify  computers  as  the  medium  of  games.  Past 
research  has  suggested  that  games  improve  student  motivation,  affective  and  cognitive  learning.  Randel,  Morris,  Wetzel,  and 
Whitehill  (1992)  examined  68  studies  regarding  the  effectiveness  of  games  and  simulations  in  terms  of  student  performance 
compared  with  traditional  classroom  instruction.  Of  these  68  studies,  38  delineated  no  difference  between  traditional  instruction 
and  games  and  simulations  in  terms  of  student  performance.  However,  22  of  the  68  studies  demonstrated  that  the  use  of  games 
and  simulations  enhanced  student  performance,  and  12  studies  indicated  that  students  reported  more  interest  in  games  and 
simulations  than  in  traditional  classroom  instniction,  Ricci,  Salas,  & Cannon-Bowers  (1996)  further  explained  that  although 
games  consistently  have  been  found  to  provide  a more  interesting  approach  to  learning  than  the  traditional  classroom 
environment,  games  did  not  necessarily  provide  a more  effective  training  approach.  In  Klein  and  Freitags’  study  (1991),  they 
indicated  that  several  researchers  proposed  instructional  games  can  motivate  learners  in  a practice  setting.  They  thought  that 
games  could  provide  extrinsic  motivation  for  iterative  practice  and  that  games  could  be  incorporated  into  instruction  to  enhance 
student  attention.  Games  were  motivational  because  they  generated  enthusiasm,  excitement,  and  enjoyment  and  because  they 
required  students  to  be  actively  involved  in  learning.  Stewart  (1997)  even  made  a statement  that  games  could  be  effective 
instructional  tools  that  entertain  while  motivating.  Dempsey,  Rasmussen,  & Lucassen  (1994)  further  suggested  that  technology - 
based  instructional  gaming  has  a wide  spectrum  of  utility  for  learning.  Ricci,  Salas,  & Cannon-Bowers  (1996)  defined  computer- 
based  gaming  as  a “rule-governed,  goal-focused,  microcomputer  driven  activity  incorporating  principles  of  gaming  and  computer 
assisted  instruction.” 

Educational  computer  games  and  students’  motivation  to  learn 

The  purpose  of  educational  computer  games  is  of  course  to  teach,  but  many  teachers  found  computer  games  a powerful 
motivator  for  initiating  the  learning  process  (Stewart,  1997).  Students  can  be  focused  on  the  instructional  content  using  an 
alternative  learning  mechanism,  playing  games,  to  the  instructor.  More  research  indicates  that  using  gaming  techniques  in 
multimedia  design  allows  students  to  become  actively  involved  while  making  the  session  enjoyable  (Metcalf,  Barlow,  Hudson, 
Jones,  Lyones,  Munfus,  & Piersall,  1998).  Children's  interest  in  computer  games  and  technology  may  be  harnessed  to  good 
educational  use  (Fisher,  1994).  The  goal  is  to  maintain  the  student’s  interest  while  increasing  her/his  skills  and  knowledge. 
Since  learning  occurs  with  repetition,  a trainer  wants  a student  to  be  motivated  to  come  back  to  the  game  often.  Educational 
software  disguised  as  games  will  capture  and  hold  a student's  interest.  Software  that  entertains  will  more  effectively 
communicate  its  educational  message.  Compared  to  learning  by  rote,  learning  with  colorful,  dynamic  computer  screens  will 
prevail  every  time  (Millman,  1992).  Educators  should  try  to  create  learning  environments  in  which  students  are  allowed  to  make 
choices,  initiate  activities,  and  view  learning  as  a celebration  (Murphy  & Thuente,  1995).  The  impact  and  implication  for  the 
educational  context  is  as  Dorman  (1997)  stated  that,  the  expectation  by  students,  all  learning  must  take  a gaming  approach  and  be 
fun.  The  idea  is  to  entertain  education  by  edutainment  media.  Teachers  and  educators  can  make  ultimate  use  of  novel  computer 
games  engage  classroom  learning  with  more  enjoyable  perspectives  and  with  better  student  motivations.  Moreover,  Evans 
(1996)  also  stated  that  students’  academic  performances  were  not  necessary  improved  because  of  using  computer  games.  Rather, 
computer  games  serve  a different  purpose.  Games  are  supposed  to  be  fun,  get  students’  attention,  keep  students  on  task,  and 
motivate  them  to  be  active  learners. 


Edutainment  and  the  learning  environment 

Educational  software  in  terms  of  edutainment,  not  limited  to  drill  and  practice  for  individual  students,  supports  individual 
learning  activities.  While  interacting  with  the  educational  software,  students  are  taking  their  own  paces  to  learn  and  construct 
their  own  understandings  of  the  instructional  messages  that  the  software  introduces.  It  corresponds  to  the  ongoing  move  towards 
more  student-centered  learning,  a move  that  has  been  steadily  accelerating  since  the  late  1960s  (Ellingon,  2000).  Moreover, 
adopting  computer  edutainment  to  engage  students  to  think  and  to  learn  is  also  an  approach  of  constructivist  learning.  According 
to  constructivism,  the  process  of  how  students  create  meaning  and  knowledge  of  the  reality  is  the  major  concern  of  the 
constructivist  learning  approaches.  Instead  of  focusing  solely  on  individually  cumulating  the  acquisition  of  facts  related  to 
specific  subject  areas,  students  have  more  chances  to  work  in  groups  and  do  tasks  collaboratively.  Students  are  engaged  in 
making  and  evaluating  their  queries  to  solving  complex,  authentic  problems  together.  They  construct  their  knowledge  based  on 
their  interpretations  of  instances  in  accordance  with  the  social  standards  and  regulations. 

The  emergent  phenomenon 

By  reviewing  statistics  numbers  and  literature  research  articles,  we  found  although  more  computers  are  available  to  students, 
the  access  and  training  are  not  consistent.  There  have  been  many  attempts  to  understand  patterns  of  technological  adoption  in 
education  (Dalton,  1989;  Dwyer,  Rignstaff,  & Sandholtz,  1991).  However,  the  answer  remains  uncertain  on  how  technology  is 
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used,  how  much  it  is  used,  and  whether  what  exists  is  broken,  worn  out,  or  still  in  unopened  boxes  (Mehinger,  1996).  In  addition, 
adoption  and  selection  of  computer  games  are  not  required  and  uniformed  to  every  school.  There  are  no  set  standards  and 
requirements  for  teachers  to  use  computer  games  within  or  between  schools.  The  choices  of  the  kinds  of  computer  games  and 
simulations  schools  choose  vary  greatly.  Although  people  are  aware  of  the  increasing  popularity  of  computer  games  and  their 
abilities  to  facilitate  teaching  and  motivate  learning,  there  are  still  obstacles  of  utilizing  educational  computer  games  in  schools  in 
terms  of  hardware  infrastructures,  software  design  and  availability,  and  teacher  training. 

Hardware 

Despite  the  number  of  students  per  computer  has  decreased  to  five  students  per  computer  in  2000  (QED,  2001),  most  regular 
classrooms  only  have  one  computer  available  for  instruction  and  mostly  for  the  instructor  to  use.  In  other  words,  it  is  impossible 
for  the  whole  class  to  work  with  an  educational  computer  game  in  a regular  classroom  where  hardware  is  insufficient  to  provide 
the  opportunity.  Therefore,  the  engagement  of  computer  games  for  instruction  must  be  taken  place  in  computer  labs,  whereas 
time  logs  of  computer  labs  are  full  and  in  use  most  of  the  time.  Consequently,  the  availability  of  hardware  is  not  ready  for 
teachers  to  apply  educational  computer  games  into  their  regular  teaching  strategies.  The  availability  of  hardware  at  this  point  of 
time  can  only  allow  educational  computer  games  as  the  supplemental  instructional  materials  or  extra  instructional  activities  to 
students.  To  individual  teachers  who  likely  are  eager  to  utilize  computer  games  to  facilitate  and  motivate  students’  learning,  it  is 
not  practical  to  happen  in  the  context  of  K-12  schools  so  far.  Another  problem  is  that  there  is  a huge  amount  of  outdated 
computers  that  are  still  serving  in  K-12  schools  but  cannot  run  educational  computer  game  software.  Glennan  & Melmed  (1996) 
indicated  in  the  RAND  report  that  despite  there  is  a rapid  growth  of  innovation  and  purchasing  of  computers,  the  average  school 
still  makes  limited  use  of  computers  and  substantial  numbers  of  schools  have  very  limited  access  to  technology  of  any  kind.  The 
platforms  in  schools  are  seriously  outdated  and  waiting  for  replaced. 

Software 

When  choosing  educational  software  to  use,  teachers  usually  find  it  is  difficult  to  have  good  quality  software  with  desired 
content  specified.  Moreover,  there  are  always  not  enough  funds  to  purchase  software.  Glennan  & Melmed  (1996)  pointed  out 
that  the  education  software  market  has  had  a rapid  expansion  in  home  education  software;  however,  the  market  for  school-based 
content  software  is  modest  and  comparatively  stagnant.  Quality  content  software  for  K-12  schools  is  not  broadly  available  since 
home  edutainment  materials  do  not  always  or  readily  translate  to  the  classroom.  They  further  explained  that  the  economics  of  the 
school  market  do  not  work  for  software  developers.  It  is  very  critical  to  gain  profits  on  developing  quality  content  software 
targeted  at  school  markets.  Schools  spend  very  little  on  software.  According  to  MDR’s  annual  survey  of  2001,  schools  spend 
20%  of  their  technology  budget  on  software  and  spending  toward  hardware  accounts  for  67%  of  total  technology  spending.  The 
shortage  in  content  software  persists.  The  development  of  content  software  for  use  in  schools  is  a difficult  challenge. 

Teacher  training 

When  technology  is  deeply  introduced  to  a school,  teachers  are  required  to  take  new  roles  and  learn  new  skills.  However, 
research  indicated  that  neither  the  initial  preparation  of  teachers  nor  the  current  strategies  for  continued  professional  development 
have  been  effective  in  developing  these  new  requirements  (Glennan  & Melmed,  1996).  Use  of  technology  to  significantly  affect 
classroom  practice  tends  to  be  limited  to  small  groups  of  teachers  who  are  excited  by  the  potential  that  they  feel  technology  has  to 
motivate  their  students  or  to  access  new  resources  (Glennan  & Melmed,  1996).  Most  teachers  have  little  formal  instruction  on 
how  to  use  technology  or  how  to  select  proper  types  of  software  to  help  them  teach.  The  lacks  of  continual  teacher  training  on 
technology  and  supports  on  utilizing  technology  have  made  some  teachers  with  novice  technology  competence  step  back  to  what 
they  are  more  familiar  with,  the  traditional  instruction  method.  According  to  MDR’s  annual  survey  of  2001,  average  schools 
devote  14%  on  professional  development.  There  must  be  many  schools  allocate  less  than  14%  of  the  whole  technology  budget 
on  professional  development.  A survey  revealed  that  teachers  are  less  likely  to  have  had  training  in  more  advanced  technologies, 
such  as  multimedia  computers  and  the  Internet  (Jerald,  1998). 

Conclusion 

Despite  more  and  more  educational  software  are  getting  available  with  good  qualities  in  the  market  and  despite  the  positive 
stimuli  that  educational  software  brings  to  students’  learning,  the  fundamental  focus  should  be  whether  or  not  the  schools, 
teachers  are  ready  to  utilize  it  to  facilitate  teaching  and  learning  in  terms  of  hardware  infrastructure,  software  adaptation,  and 
teacher  preparation.  As  Glennan  & Melmed  (1996)  stressed  there  are  two  particularly  important  incidences,  equipping  teachers 
to  effectively  exploit  technology  for  the  benefit  of  their  students  and  assuring  a plentiful  supply  of  high-quality  content  software. 
The  readiness  of  all  aspects  is  the  key  to  the  successful  utilization. 
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Abstract 

This  first  phase  of  a content  analysis  of  online,  asynchronous,  educational  discussions  is  designed  to  generate  a method  for 
automatically  categorizing  messages  into  cognitive  categories  using  neural  network  software.  This  phase  of  research  answers 
two  questions  regarding  the  method  of  automatically  analyzing  discussion  messages:  Can  a neural  network  reliably  categorize 
messages  under  optimum  circumstances,  and  how  can  the  method  be  improved  to  generate  greater  reliability?  To  determine 
whether  neural  network  software  can  reliably  categorize  messages,  two  trials  were  conducted.  The  first,  ‘‘best  fit " trial,  a proof 
of  concept  trial  comprised  only  of  messages  which  best  fit  the  categorization  model,  generated  strong  reliability  figures  (CR  ~ 
0,84;  k = 0,76),  and  the  second,  systematic  sample,  a sample  much  more  indicative  of  the  messages  generated  in  an  online 
educational  discussion,  produced  formative  reliability  figures  (CR  = 0,68;  k ~ 0.31)  from  which  the  method  of  analysis  may  be 
optimized.  This  analysis  also  provides  a distribution  based  on  cognitive  presence  categories  and  subcategories  of  one  semester 
of  graduate  online  educational  messages. 

Many  universities  and  KI2  educational  settings  have  adopted  online,  web-based  instruction  as  a tool  for  delivering 
instruction.  According  to  Green  (2000,  para  7),  ‘Today,  75  percent  of  two-  and  four-year  colleges  offer  some  form  of  online 
education.  By  next  year,  that  number  will  reach  90  percent.”  Hamm  (2000,  para  8)  makes  a slightly  more  conservative  claim  by 
quoting  a study  performed  by  the  Chronicle  of  Higher  Education:  “60%  of  American  colleges  and  universities  offer  online- 
learning  programs,  and  8%  more  plan  on  doing  so  in  the  next  year.”  He  also  notes  that  the  e-learning  market  is  expected  to  grow 
from  $1.2  billion  in  2000  to  S7  billion  in  2003.  Certainly,  online  delivery  of  instruction  is  growing  as  are  fora  whereby  students 
engage  each  other.  WebCT,  one  of  the  more  popular  suite  of  tools  to  support  web-based  collaborative  learning  boasts  1 600  new 
installations  in  the  past  1 8 months  and  nearly  1 1 million  student  accounts  (Goldberg,  para  3).  Although  there  is  no  clear  data  on 
the  number  of  students  participating  in  online  courses  in  which  every  transaction  is  electronic,  there  appears  to  be  a migration 
away  from  courses  delivered  solely  face-to-face  to  those  either  supplemented  with  or  completely  reliant  on  online  discussion. 
This  migration  toward  electronic  classrooms  means  that  the  discourse  from  these  learning  environments  is  very  easily  captured 
providing  an  opportunity  for  researchers  to  study  the  process  of  learning  in  a way  that  has  never  been  available  before.  Never 
before  have  we  had  access  to  electronic  texts  containing  virtually  every  exchange  made  by  every  student  for  an  entire  term. 
Concurrently,  our  ability  to  use  computers  to  process  text  and  reveal  underlying  themes  has  steadily  grown  (Rife,  Lacy,  & Fico, 
1998).  The  convergence  of  these  two  realities  brings  us  to  our  current  state  in  which  we  have  numerous  texts  available,  a 
growing  set  of  analysis  tools,  but  very  little  research  to  explain  the  phenomena  that  take  place  in  the  course  of  learning.  Kuehn 
(1994,  p.  172)  also  highlights  this  dilemma,  “few  researchers  have  adopted  current  communication  theory  to  investigate  computer 
impact  or  effects  in  instructional  settings....” 

Despite  the  availability  of  electronic  discussion  list  texts,  few  analyses  of  the  content  generated  by  students  have  been 
conducted.  A content  analysis  type  of  inquiry  allows  us  to  describe  how  students  engage  and  generate  material  within  an  online 
setting  thereby  providing  potential  answers  to  questions  such  as:  Does  a chatroom  conversation  produce  different  cognitive 
results  than  either  a teacher-led  asynchronous  discussion  or  a student-led  asynchronous  discussion?  Henri  (1992)  makes  apparent 
the  role  content  analysis  has  to  play  in  an  instructor’s  ability  to  guide  learning: 
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Content  analysis,  when  conducted  with  an  aim  to  understanding  the  learning  process,  provides  information 
on  the  participants  as  learners,  and  on  their  ways  of  dealing  with  a given  topic.  Thus  informed,  the  educator 
is  in  a position  to  fulfill  his  main  role,  which  is  to  offer  immediate  support  to  the  individual  and  the  collective 
learning  process,  (p.  1 1 8) 

Over  all,  this  study  outlines  the  initial  phase  of  the  construction  and  use  of  a neural  network  to  perform  a content  analysis  of  a 
large  body  of  student  messages  for  cognitive  presence,  one  portion  of  Garrison,  Anderson,  and  Archer’s  (2000)  model  to 
understand  online  learning  environments.  This  type  of  tool  may  ultimately  be  used  to  gauge,  guide,  direct,  and  manipulate  the 
learning  environment.  Despite  Howell-Richardson  & Mellar’s  (1996)  research  indicating  that  modifications  to  the  structure  of  an 
online  course  produce  significantly  different  communication  outcomes,  instructors  currently  have  little  ability  to  gain  a bird’s-eye 
view  of  the  overall  learning  taking  place,  much  less  an  ability  to  respond  to  that  learning,  assess  it,  or  intervene.  This  research 
seeks  to  answer  two  questions.  First,  can  neural  networks  be  used  to  analyze  and  describe  the  cognitive  landscape  of  online 
educational  discussions?  Second,  at  this  phase,  how  is  cognitive  presence  displayed  in  an  online  course? 

Theoretical  Background 

Cognitive  Presence 

Garrison,  Anderson,  and  Archer  (2000,  2001)  have  developed  a community  of  inquiry  model,  based  on  Dewey’s  (1933) 
practical  inquiry  model,  which  splits  community-based  learning  into  three  overlapping  areas:  social  presence,  cognitive  presence, 
and  teacher  presence.  They  operationalize  cognitive  presence  by  splitting  it  into  four  phases:  triggering  event,  exploration, 
integration,  and  resolution,  and  use  the  following  descriptors  respectively  for  each  phase:  evocative,  inquisitive,  tentative,  and 
committed.  Specifically,  cognitive  presence  is  defined  as  ‘‘the  extent  to  which  learners  are  able  to  construct  and  confirm  meaning 
through  sustained  reflection  and  discourse  in  a critical  community  of  inquiry”  (p.  11).  Garrison  et  al.  employ  their  cognitive 
presence  model  to  analyze  an  online  discussion  group.  Their  unit  of  analysis  is  the  entire  message  mainly  because  messages  are 
easiest  to  identify  and  occur  naturally  in  discussion  environments.  Because  a message  may  contain  indicators  for  multiple 
phases,  they  have  developed  two  heuristics  for  deciding  which  messages  fall  into  which  categories:  code  down  and  code  up. 
They  used  human  coders  to  classify  messages,  and  this  yields  a reliability  figure  (k=0.74)  which  Riffe,  Lacy,  and  Fico  (1998) 
accept  only  for  research  that  is  breaking  new  ground,  a category  under  which  this  research  clearly  fits.  Also,  they  found  that  the 
greatest  coding  discrepancies  occurred  between  coding  for  exploration  and  integration.  They  admit  low  occurrences  of  resolution 
and  believe  higher  instances  of  resolution  will  be  found  ‘‘where  applied  knowledge  is  valued — particularly  adult,  continuing,  and 
higher  education”  (p.  1 6) 

Can  Neural  Networks  Analyze  Messages? 

The  use  of  neural  networks  in  educational  settings  is  rare  and  there  are  no  accounts  outlining  the  use  of  a neural  network  to 
analyze  text  messages  of  an  online  discussion  group.  Garson  (1998)  provides  a number  of  reasons  why  social  scientists  have  not 
adopted  the  use  of  neural  networks  in  their  research.  First,  neural  network  software  has  been  available  to  the  social  scientist  only 
since  the  early  1990’s.  Second,  it  is  not  clear  how  neural  networks  arrive  at  their  conclusion;  unlike  an  expert  system,  neural 
networks  provide  no  audit  trail  outlining  their  reasoning.  Also,  neural  network  techniques  are  complex  and  leave  researchers 
unsure  whether  their  analysis  is  truly  optimal;  slight  modifications  to  a number  of  parameters  may  yield  a more  optimal  analysis. 
Nonetheless,  neural  networks  are  good  at  making  predictions  (e.g.  stock  market  forecasting)  and  at  classification.  Garson  cites 
34  research  studies  using  neural  networks  in  economics  and  business,  9 in  sociology,  7 in  political  science,  and  45  in  psychology 
(Garson,  1998,  pp.  8-22). 

Given  a high  enough  reliability  value,  neural  networks  have  the  ability  to  classify  large  quantities  of  data  which,  for  the 
present  study,  means  that  researchers  do  not  have  to  sample  a subset  of  all  online  messages  from  a course.  Instead,  the  neural 
network  classifies  each  message  thereby  eliminating  sampling  error.  In  comparison  with  statistical  methods  of  analysis,  Garson 
(1998)  mentions: 

[N]eural  models  may  outperform  traditional  statistical  procedures  where  problems  lack  discernible  structure, 
data  are  incomplete,  and  many  competing  inputs  and  constraints  related  in  complex,  nonlinear  ways  prevent 
formulation  of  structural  equations,  provided  the  researcher  can  accept  the  approximate  solutions  generated 
by  neural  models  (p.l) 

Clearly,  student  messages  are  filled  with  competing  inputs  related  in  a complex,  nonlinear  fashion.  Further,  traditional  textual 
analysis  of  this  type  would  require  the  use  of  multiple  human  coders  classifying  each  message  against  a set  of  classification 
criteria,  a resource-intensive  technique  which  also  generates  approximate  solutions. 

Method 

The  method  involves  four  steps  starting  with  a text  -based  transcript  of  an  online  discussion  and  ending  with  the  calculation  of 
reliability  statistics. 
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Database  Creation 

First,  one  semester’s  worth  of  asynchronous,  online  discussion  messages  were  converted  from  a single  text  file  containing  all 
messages  for  one  semester  into  a database  such  that  each  record  represented  one  message  and  contained  the  message  body, 
author,  date,  etc.  This  task  was  accomplished  using  SQL  Server  and  a series  of  SQL  statements  to  populate  the  database.  These 
generic  tools  were  used  to  streamline  the  process  of  making  them  publicly  available  over  the  World  Wide  Web. 

Word  Count  Tool 

Second,  a tool  was  constructed  to  page  through  each  message  body  and  perform  word  counts  in  both  self-defined  and  General 
Inquirer  categories  (see  Danielson  & Lasorsa,  1997;  http://www.wjh.harvard.edu/^inquirer).  The  categorical  word  count 
procedure  results  in  a database  table  with  categories  as  columns,  individual  messages  as  records,  and  cell  values  representing  the 
count  of  terms  from  each  cognitive  presence  category.  Self-defined  categories  allow  researchers  to  define  specific  indicators  for 
each  category.  For  example,  items  falling  into  the  cognitive  presence  phase  “integration”  often  refer  to  previous  messages  or 
draw  from  a course  participant’s  prior  knowledge;  therefore,  typical  “integration”  messages  incorporate  terms  and  phrases  such 
as  “thanks,”  “that  reminds  me  of,”  “compared  to,”  and  “I  agree.”  The  researcher  may  create  one  or  a number  of  categories  that 
serve  as  indicators  that  a message  should  be  categorized  as  an  integration  message.  This  tool  not  only  allows  for  the  creation  of 
new,  user-defined  input  categories  but  also  incorporates  existing  input  categories  from  the  dictionary  of  terms  found  in  the 
General  Inquirer.  This  dictionary  is  comprised  of  1 1,788  words  in  182  categories.  Each  message  was  analyzed  against  each  self- 
defined  and  General  Inquirer  category  of  terms  and  a simple  word  count  was  taken  to  determine  the  weight  of  each  category  of 
terms  in  each  message.  For  example,  the  General  Inquirer  category  “posit iv”  contains  the  words  “up,  abide,  and  yes”  meaning 
that  the  following  sentence  will  receive  a “positiv”  score  of  two:  “Yes,  I had  to  look  up  to  see  the  icon.”  Further,  the  “positiv” 
score  of  2 is  normalized  so  the  neural  network  can  accurately  compare  scores  across  messages.  Normalization  is  performed  by 
dividing  the  number  of  times  the  terms  in  a single  category  appear  in  a message  by  the  total  number  of  words  in  the  message 
(2/10  = 0.2). 

Neural  Network  Training 

Third,  a feedforward,  backpropagation,  neural  network  was  trained  to  classify  each  message  as  falling  into  one  of  five 
categories  (triggering  event,  exploration,  integration,  resolution,  or  noncognitive).  This  was  done  by  human-classifying  a group 
of  messages  to  be  used  as  the  training  set,  training  the  neural  network  on  that  set  of  messages,  and  then  classifying  a second  set  of 
messages  for  reliability  purposes. 

Reliability  Measures 

Fourth,  reliability  measures  were  taken  comparing  human-coded  messages  with  those  classified  by  the  neural  network.  Huck 
(2000)  recommends  the  use  of  multiple  reliability  measures  for  a single  study  (p.  98).  For  this  reason  and  because  this  study 
replicates  a similar  study  by  Garrison,  Archer,  and  Anderson  (2001),  two  reliability  measures  were  employed:  Holsti’s  (1969) 
coefficient  of  reliability  (CR)  which  measures  the  agreement  between  two  coders  divided  by  the  total  number  of  messages 
analyzed  and  Cohen’s  kappa  which  corrects  for  chance  agreement  among  coders.  The  difference  between  the  Garrison  et  al. 
study  and  this  one  is  that  Garrison  et  al.  performed  a human  - human  comparison  whereas  this  study  performed  a neural  network 
- human  comparison. 

Results 

To  determine  whether  the  neural  network  analysis  produces  results  comparable  to  human-coded  content  analysis,  benchmarks 
from  a human- coded  content  analysis  by  Garrison,  Anderson,  and  Archer  (2001)  were  compared  to  results  from  this  neural 
network  analysis.  Garrison  et  al.  went  through  three  phases  of  training  human  coders  to  reliably  categorize  messages  and  used 
both  Holsti’s  (1969)  coefficient  of  reliability  (CR)  and  Cohen’s  (1969)  kappa  (k)  to  measure  inter-rater  reliability.  Garrison  et  al. 
generated  the  following  reliability  figures: 


Reliability  Measure 

Trial  1 

Trial  2 

Trial  3 

CR 

0.45 

0.65 

0.84 

Kappa 

0.35 

0.49 

0.74 

Table  I.  Reliability  measures  for  Garrison,  Anderson,  & Archer  *s  (2001)  content  analysis  of  an  online  discussion. 

Best  Fit  Sample 

The  analysis  using  the  neural  network  to  classify  messages  is  a multi-phase  process  of  which  this  papCT  presents  the  first 
phase.  This  phase  seeks  to  answer  whether  it  is  possible  at  all  for  a neural  network  to  classify  messages.  In  this  phase,  the 
messages  best  representing  each  category  were  coded  and  used  to  train  and  test  the  neural  network  model.  This  “best  fit”  trial 
yielded  the  following  reliability  figures:  CR  = 0.84  and  k = 0.76.  This  test  set  (n=26)  of  optimal  messages  generates  the 
following  matrix  after  being  run  through  the  trained  “best  fit”  model. 
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Chart  1.  The  desired  results  are  compared  to  the  neural  network’s  estimated  results,  and  the  numbers  appearing  diagonally 
indicate  that  the  neural  network  matched  the  coded  test  set  of  messages. 


In  this  set,  1 indicates  a triggering  event,  2 is  an  exploratory  message,  3 is  an  integration  message,  and  4 is  a resolution  message. 
This  trial  indicates  that  a neural  network  can  reliably  discern  the  first  three  categories. 

Systematic  Sample 

The  purpose  of  the  “best  fit”  trial  is  to  determine  whether  a neural  network  can  be  used  to  categorize  text  messages  at  all;  the 
second  trial  uses  a systematic  sample  of  messages  in  which  both  the  training  set  (n=100)  and  the  test  set  (n=100)  are  a systematic 
sample  of  every  20  messages.  There  are  1,997  messages  in  all;  therefore,  this  sample  represents  a cross-section  of  messages 
occurring  throughout  the  term.  Further,  this  sampling  technique  introduces  noise  into  the  analysis;  to  accommodate  for  this,  a 
fifth  (noncognitive)  category  was  used.  This  category  represents  non-cognitive  messages  (e.g.  greeting  and  short  agreement 
messages),  course  management  messages  (e.g.  “When  will  the  textbook  be  available,”  or  “when  is  the  next  chat?”),  and  technical 
support  messages  (e.g.  “ I can’t  get  into  the  chat  room,”  and  “Why  are  my  messages  not  showing  up  on  the  discussion  list?”).  A 
neural  network  trained  against  100  messages  using  all  five  categories  yielded  a CR  value  of  0.68  and  a kappa  value  of  0.31 
generating  the  following  message  category  results: 
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Chart  2.  The  introduction  of  a fifth  (miscellaneous)  category  adds 


real-world  noise  to  the  neural  network  analysis. 


Finally,  the  systematic  sampling  of  20%  of  the  messages  from  the  term  provides  insight  into  the  cognitive  effort  displayed  by  the 
course  participants.  The  first  chart  displays  the  percentage  of  messages  by  broad  cognitive  category  type,  and  the  second 
displays  messages  by  the  subcategories  which  make  up  each  category  type. 


Cognitive  Presence  by  Category 

Percentage 

Not  Cognitive 

48% 

Triggering  Event 

3% 

Exploration 

39% 

Integration 

9% 

Resolution 

1% 

Table  2.  Messages  by  cognitive  presence  category. 
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Cognitive  Presence  By  Subcategory 

Percentage 

Not  Cognitive 

Unrelated 

20% 

Course  Management 

12% 

Technical  Support 

12% 

External  Reference 

4% 

Total  Not  Cognitive 

48% 

Triggering  Event 

Recognizes  Problem 

1.5% 

Puzzlement 

1.5% 

Total  Triggering  Event 

3% 

Exploration 

Personal  Narrative 

6% 

Information  Exchange 

12% 

Brainstorming 

5% 

Divergence  Among 

4% 

Leap  to  Conclusion 

5% 

Suggestion 

6% 

Divergence  Within 

1% 

Total  Exploration 

39% 

Integration 

Creating  Solutions 

0.5% 

Synthesis 

1.5% 

Convergence  Within 

2% 

Convergence  Among 

5% 

Total  Integration 

9% 

Resolution 

1% 

Table  3.  Messages  by  cognitive  presence  subcategory. 


Discussion 

Findings 

The  first  trial  indicates  that  in  the  absence  of  noise,  a neural  network  can  be  used  to  categorize  messages  into  the  cognitive 
categories  outlined  by  Garrison,  Anderson,  and  Archer  (2001)  based  on  linguistic  cues.  In  the  trial  which  introduced  the  noise 
which  naturally  occurs  in  discussion  lists,  we  see  that  the  model  overgeneralizes  on  categories  two  (exploration)  and  five 
(miscellaneous),  that  it  undergeneralizes  on  integration  messages,  and  that  it  does  not  discern  triggering  events  and  resolution 
messages  from  the  others.  These  findings  provide  critical,  formative  information  which  can  be  used  to  optimize  and  therefore 
improve  the  model.  Methods  for  improving  the  model’s  ability  to  correctly  categorize  are  outlined  below. 

Optimization 

Just  as  Garrison  et  al.’s  coders  optimized  their  coding  algorithm  between  times  they  coded,  the  neural  network  method  of 
analysis  may  also  be  optimized.  The  above  reliability  reflects  an  initial  brute-force  analysis  of  each  message  and  takes  as  input 
weights  generated  by  analyzing  each  message  against  a category  of  terms  in  the  General  Inquirer  dictionary.  The  following  steps 
may  be  taken  to  improve  the  model: 

Word  sense  disambiguation:  This  simply  means  that  individual  words  are  classified  according  to  their  parts  of  speech.  For 
example,  the  word  “test”  may  be  used  as  either  a verb  or  a noun,  and  a word  sense  disambiguation  routine  will  clearly  separate 
those  instances  of  “test”  that  are  nouns  and  those  that  are  verbs.  This  should  dramatically  reduce  the  amount  of  noise  in  the 
database. 

Increased  training  set:  The  next  phase  of  this  research  is  the  analysis  of  six  eCore  courses,  online  post-secondary,  core 
curriculum  courses  offered  by  the  University  System  of  Georgia.  In  this  phase  of  research,  six  instructors  will  analyze  200 
messages  each  thereby  generating  a training  set  of  1 100  messages.  In  comparison,  the  current  research  used  100  messages  as  its 
training  set  and  100  messages  as  a test  set.  Building  a model  from  1 100  coded  messages  should  improve  the  generalizability  of 
each  category  and  therefore  the  model’s  ability  to  correctly  classify  messages. 

Message  hierarchy  metainformation:  In  the  current  model,  the  only  hierarchy  information  fed  into  the  neural  network  is  whether 
each  message  is  a reply  to  another  message  or  not.  Garrison  et  al.’s  model  indicates  that  messages  are  partially  classified  not 
only  based  on  their  textual  content  but  on  their  place  within  a given  thread  hierarchy.  If  a message  is  the  first  in  it’s  thread 
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hierarchy,  it  is  most  likely  a triggering  event.  If  it  is  near  the  beginning  of  a thread  and  is  a response  to  another  message,  it  is 
most  likely  either  an  exploratory  or  integrative  message. 

Improved  categories:  Create  subsets  of  each  category  that  are  very  specific,  and  ensure  that  each  message  fits  cleanly  into  a 
category. 

Cognitive  Presence  Distribution 

The  distribution  of  messages  into  cognitive  presence  categories  is  similar  to  that  found  by  Garrison,  Anderson,  and  Archer 
(2001)  in  that  a majority  of  messages  fell  into  the  exploration  category  with  fewer  integration  messages,  only  a few  triggering 
events,  and  practically  no  resolution  messages.  The  discussion  topics  and  goals  of  the  course  define  the  distribution  we  found. 
The  goal  of  the  course  is  to  give  each  student  experiential  knowledge  of  web-based  learning.  It  is  up  to  the  instructor  to  define 
whether  resolution  can  practically  be  achieved  in  the  course;  resolution  is  usually  reserved  for  more  practical  tasks  in  which 
students  state  that  they  have  resolved  an  issue  which  means  they  have  applied  knowledge  in  a real-world  setting  and  have  found 
that  the  real-world  outcome  affirms  knowledge  gained  from  the  course.  Although  students  were  creating  their  own  web-based 
learning  modules,  these  modules  were  not  intended  to  be  the  product  of  a learned  body  of  knowledge;  rather  these  modules  were 
intended  to  be  tasks  from  which  questions  emerge.  Given  this  course  structure,  it  makes  sense  that  resolution  is  rare  and 
exploration  dominates.  Interestingly,  the  number  of  triggering  events  is  fairly  low  which  may  also  be  attributed  to  the  course 
structure;  students  were  not  given  a formal  triggering  event  or  question  by  the  instructor  each  week;  instead,  the  instructors 
allowed  the  students’  exploration  to  define  the  direction  of  the  course.  In  this  case,  triggering  events  were  more  likely  to  be 
found  embedded  within  exploratory  messages.  Tracing  triggering  events  may  be  assisted  by  the  creation  an  overall  diagram  of 
the  course  structure  allowing  us  to  see  not  triggering  events  as  defined  by  the  linguistic  cues  within  the  message  but  rather  as 
defined  by  the  messages  emanating  from  these  triggering  events.  That  is,  if  we  find  that  one  message  spawns  a critical  debate, 
then  we  may  in  retrospect  define  that  message  as  a triggering  event.  This  information  can  be  displayed  graphically  for  use  by 
those  coding  the  training  set  of  messages  and  numerically  for  use  by  the  neural  network. 

The  Next  Phases  of  this  Research 

It  is  expected  that  a well-trained  neural  network  will  perform  just  as  reliably  as  a set  of  human  coders  at  classifying  messages 
into  cognitive  presence  categories.  This  method  of  analysis  will  then  provide  a broad  overview  of  the  cognitive  effort  displayed 
by  students  throughout  the  semester  and  allows  for  instructors  to  make  adjustments  to  their  approach  in  order  to  bring  about 
desired  displays  of  cognitive  effort.  Further,  this  rapid  analysis  method  provides  a tool  instructors  may  use  to  conduct  their  own 
research  on  finely  grained  aspects  of  the  cognitive  dynamics  of  a course.  This  method  may  allow  us  to  answer  questions  such  as: 
Which  displays  of  instructor  involvement  generate  exploration  and  which  generate  integration?  Are  socially  engaged  students 
also  cognitively  engaged?  How  many  course  participants  is  optimal  for  higher  order  thinking?  Which  class  participants 
encourage  the  integration  of  ideas? 
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Abstract 

This  paper  provides  a review  of  literature  that  relates  research  on  Carl  Rogers  ’ person-centered  learning  theory  to  web- 
based  learning.  Based  on  the  review  of  literature,  a set  of  criteria  is  described  that  can  be  used  to  determine  how  closely  a web- 
based  course  matches  the  different  components  of  Rogers'  person-centered  learning  theory.  Using  the  criteria  based  on  Rogers ' 
learning  theory  and  the  person-centered  model  of  instruction  developed  by  Miller  and  Mazur  (2001)  a case  example  of  a web- 
centric course  is  evaluated. 


Relevance  of  Rogers’  Approach  to  Learning  for  Web-based  Instruction 

Education  through  the  years  has  integrated  new  technologies  to  enhance  the  learning  experiences  of  students.  During  the 
twentieth  century  we  have  seen  the  advent  of  technologies  such  as  projection  systems  and  television  expanding  educational 
capabilities  beyond  the  face-to-face  classroom  setting  to  teaching  at  a distance.  The  1990s  ushered  in  the  World  Wide  Web 
(WWW),  a technology  expanding  the  capabilities  of  teaching  at  a distance  possibly  as  much  as  television  and  projection  systems 
expanded  education  in  the  twentieth  century.  While  the  WWW  has  expanded  the  possibilities  of  education  it  is  also  moving 
educators  to  consider  new  ways  of  teaching  and  learning  because  the  WWW  has  literally  created  a new  type  of  learning 
environment  with  possibly  as  much  potential  for  student  learning  as  a traditional  classroom. 

Many  educators  and  researchers  have  begun  discussions  about  the  potentials  for  learning  as  well  as  some  of  the  outcomes 
experienced  through  web-based  instruction.  Some  of  the  outcomes  discussed  are  the  aspects  of  active  participation  shown  by 
students  in  web-based  learning  environments  (Brown,  1997;  Greening,  1998;  Oliver  & Omari,  1999),  the  construction  of 
knowledge  (Stefanov,  Stoyanov,  & Nikolov,  1998),  problem  solving  (Corren-Agostinho,  Hedberg,  & Lefoe,  1998;  Oliver  & 
Omari,  1999),  students  leading  the  learning  (Berge,  1997;  Bonk  & Cummings,  1998;  Stefanov  et  al.,  1998),  and  teachers  acting 
as  facilitators  (Brown,  1997;  Stefanov  etal.,  1998).  The  outcomes  mentioned  in  many  articles  on  web-based  learning  studies  such 
as  Chalmers’  (Chalmers,  1997)  and  Lazio  and  Castro’s  (Laszlo  & Castro,  1995)  articles  can  be  identified  as  components  of 
learning  as  described  by  Carl  Rogers’  person-centered  learning  theory. 


Individual  Learning  in  Web-based  Environments 

Because  the  WWW  is  based  cai  hypertext,  which  is  words  and  images  linked  by  multiple  paths  (Landow,  1997;  McKnight, 
Dillon,  & Richardson,  1996),  the  learner  selects  the  paths  to  explore  and  thus  discovers  new  links  of  information.  It  is  through  the 
linking  of  these  multiple  paths  of  the  hypertext  environment  in  which  students  become  active  participants  in  the  construction  of 
knowledge  and  meaning  (Brown,  1997).  Another  aspect  of  the  Web  affecting  learning  is  the  ability  to  enter  or  leave  at  any  point 
within  the  Web.  While  there  are  often  centralized  information  centers,  a user  can  often  find  information  from  any  access  point 
based  on  their  own  investigation  and  experience  (Landow,  1997).  Student  explorations  of  web-based  information  will  not  only 
help  them  become  active  learners  but  it  will  also  generate  attitudes  for  personal  development.  These  explorations  allow  learners 
to  pick  and  choose  where  they  want  to  go  and  what  they  want  to  learn  as  well  as  determine  the  importance  of  the  information 
found.  The  learning  possibilities  could  increase  the  potential  impact  on  areas  such  as  self-esteem,  self-control,  self-efficacy  and 
motivation  (Laszlo  & Castro,  1995). 


Collaborative  Learning  in  Web-based  Environments 

While  the  WWW  can  provide  opportunities  for  individual  active  learning  it  can  also  provide  opportunities  for  collaborative 
learning  and  the  development  of  learning  communities.  Using  the  WWW  and  the  various  forms  of  communication  available  on  it, 
students  can  learn  through  group -based  assignments,  research  projects,  and  presentations.  Collaborative  learning  on  the  web  can 
enhance  the  learning  experience  for  the  students  more  so  than  individualistic  learning  on  the  web  because  it  could  create  a more 
communal  or  team  atmosphere.  Chickering  and  Ehrmann  (1997)  found  that  working  with  others  increases  involvement  in 
learning  because  good  learning  is  collaborative  and  social  not  competitive  and  isolated.  While  increasing  student  involvement, 
the  web  also  allows  the  students  to  assist  each  other  in  the  exploration  of  answers  to  areas  of  common  interest,  share  constructed 
knowledge  and  interpretations  of  knowledge,  and  build  upon  that  knowledge  through  feedback  from  peers  (Graham  & 
Scarborough,  1999;  Harasim,  1990). 

While  collaborative  learning  can  occur  just  as  easily  in  a face-to-face  situation  there  are  some  advantages  to  web-based 
collaborative  learning.  Collaborative  learning  on-line  can  be  done  either  in  a same  time/any  place  situation  (synchronous 
communications)  or  in  an  any  time/any  place  situation  (asynchronous  communications).  Using  asynchronous  communications  the 
learners  are  allowed  to  respond  at  a time  most  appropriate  for  them,  which  can  allow  for  opportunities  of  reflection.  The  potential 
for  anonymity  of  the  contributors,  can  also  be  a great  equalizer  when  learners  will  not  need  to  struggle  for  their  chance  to  speak. 
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and  everyone  can  make  a contribution  that  will  be  "heard"  by  all  class  members  (Brown,  1997).  Anonymity  can  also  allow 
learners  to  feel  that  they  can  be  open  to  expressing  their  opinions  in  an  open  environment  allowing  better  comprehension  and 
affirmation,  which  are  key  behaviors  at  the  core  of  collaborative  communication  (Zimmer  &,  Alexander,  1996).  To  ensure 
development  of  a learning  community  the  individuals  will  need  to  develop  and  maintain  a group  environment  with  all  members 
assuming  at  least  some  of  the  responsibility  for  the  environment  starting  initially  with  the  facilitator  (Armstrong  & Yarbrough, 
1996). 

The  Need  for  a Change  in  Web-based  Learning  Environments 

While  web-based  learning  environments  seem  attractive  in  their  potential  to  widen  the  scope  of  users’  experiences,  these  new 
technologies  also  have  the  potential  to  create  artificial  and  possibly  depersonalizing  social  circumstances  through  the  lack  of 
communication.  In  the  past  one  of  the  largest  complaints  from  students  in  distance  education  situations  is  a sense  of  alienation 
that  leads  to  dissatisfaction  of  the  learning  experience  (Biner,  Dean,  &,  Mellinger,  1994).  In  a web-based  learning  experience  it  is 
necessary  for  the  instructor  to  establish  and  facilitate  continual  lines  of  communication  so  that  students  do  not  feel  that  they  are 
all  alone.  Another  change  that  is  needed  for  web-based  learning  is  the  locus  of  control.  As  in  the  past,  when  new  interactive 
technologies  are  introduced  into  learning  contexts,  the  role  of  instruction  and  the  instructor  needs  to  be  examined.  In  web-based 
learning  environment  the  locus  of  control  shifts  from  the  instructor  to  the  student  due  to  the  environment.  This  change  will  reflect 
less  control  on  the  part  of  the  instructor  as  he/she  facilitates  the  learning  learners’  opportunities  for  controlling  more  of  their  own 
exploration  and  interaction.  Instruction  enhanced  and  shaped  by  facilitation. may  be  a key  to  web-based  learning  environments 
because  students  will  need  specialized  guidance  exploring  their  on-line  learning  contexts.  These  needs  are  all  focused  upon  in  the 
person-centered  learning  theory  developed  by  Carl  Rogers. 

Carl  Rogers’  Person-centered  Learning  Theory 

When  Rogers  wrote  Freedom  to  Learn  (1969;  Rogers  & Freiberg,  1994)  he  focused  on  traditional  schools  but  saw  the  person- 
centered  educational  approach  developing  its  strongest  roots  in  alternative  schools  and  universities  without  walls.  While  many 
instructional  theories  focus  on  the  learners'  achievement  of  specific  learning  objectives,  Rogers'  learning  theory  focused  on  a goal 
of  helping  the  learner  learn  how  to  learn.  Rogers  felt  the  learner  would  become  a freely  functioning,  self-enhancing,  self- 
actualizing,  creative,  and  dependable  person  with  this  focus. 

Carl  Rogers  claimed  he  developed  his  person-centered  theory  because  we  live  in  a constantly  changing  world  and  as  people 
in  this  changing  world,  we  need  to  be  willing  to  change  our  thinking  to  adapt  to  these  changes.  More  importantly,  students  need 
to  learn  how  to  learn  in  order  to  adapt  to  the  different  types  of  learning  required  in  a variety  of  settings  and  for  a myriad  of 
purposes.  Rogers  boldly  suggested  the  facilitator  should  encourage  the  learners  to  charge  off  in  new  directions  dictated  by  their 
own  interests  and  to  unleash  their  sense  of  inquiry  and  exploration  (Rogers  & Freiberg,  1 994). 

Rogers’  model  theorizes  a person  emerging  from  therapy  or  from  the  best  of  education  has  experienced  optimal 
psychological  growth  meaning  the  learner  is  able  to  freely  function  in  all  of  his  or  her  potentials,  self-enhancing,  continuing  to 
develop,  and  always  seeking  newness  in  each  moment,  resulting  in  a more  self- actualized  pCTSon  (Rogers  & Freiberg,  1994). 
Maslow  (1970),  describes  this  self-actualized  person  as  someone  who  has  developed  or  is  developing  into  the  full  stature  of 
which  he  or  she  is  capable.  The  development  of  a self-actualized  person  through  the  person-centered  theory  is  important  in 
education  because  one  goal  of  education  is  to  develop  learners  into  whole  people.  The  learners  will  become  people  freely 
functioning  in  all  aspects  of  love,  feeling,  and  creativity.  Of  importance,  is  that  these  people  may  continue  to  learn  through  life 
rather  than  becoming  automatons  able  to  recite  the  information  provided  to  them  (Patterson,  1973). 

Another  aspect  of  the  person-centered  learning  theory  connected  to  self-actualization  is  that  person-centered  learning 
experiences  help  the  learners  become  a more  creative  (Patterson,  1973).  As  a learner  becomes  more  self-actualized  he  or  she  will 
be  able  to  perceive  reality  more  accurately,  accept  himself  or  herself  and  others,  become  more  spontaneous,  independent,  and 
more  creative  (Davis,  1992;  Maslow,  1970).  As  the  learner  understands  himself  or  herself  more  he  or  she  will  be  able  to  make 
meaning  of  the  world  as  well  as  become  more  understanding  of  varying  views  and  perspectives. 

While  the  development  of  self-actualized  people,  who  are  life-long  learners,  is  a commendable  goal  how  can  this  goal  be 
achieved?  First,  a teacher  should  realize  that  the  only  person  who  can  reach  the  goal  of  becoming  a self- actualized  person  is  the 
individual  learner.  Second,  unlike  in  some  instructional  methods  of  providing  the  knowledge  to  the  learners,  the  teacher  in  a 
learner- centered  environment  becomes  a facilitator  of  the  learning  the  students  themselves  conduct.  It  becomes  an  experience  of 
significant  learning  because  the  individual  initiates  it,  allowing  the  individual  to  provide  personal  control  and  the  element  of 
learning  is  built  into  the  whole  experience  (Sahakian,  1970). 

It  is  important  for  the  teacher  to  facilitate  the  learning  and  allow  the  students  to  be  challenged  to  think  for  themselves  rather 
than  being  given  information  (Rogers  & Freiberg,  1994).  There  are  several  tasks  a teacher  should  do  when  positioning 
themselves  in  a facilitator  role  instead  of  a traditional  teacher  role.  The  first  is  to  set  the  mood  for  the  environment.  There  should 
be  a sense  of  cooperation  and  trust  within  the  group.  A competitive  attitude  among  members  of  the  learning  group  will  disrupt  the 
sense  of  trust  and  cooperation.  Next,  the  teacher  as  facilitator  should  make  themselves  available  as  one  of  many  resources  of 
information  rather  than  as  the  main  source  of  information  for  the  students.  Most  importantly  though,  the  teacher  as  facilitator 
should  be  aware  of  the  attitudes  he  or  she  holds.  The  teacher  needs  to  feel  acceptance  of  his  or  her  own  feelings  thus  becoming  a 
real  person  in  relationship  with  the  students  (Rogers,  1961).  It  is  important  for  the  teacher  to  show  realness  as  a person  or  in  other 
worlds  showing  his  or  her  self-actualization  because  if  the  goal  is  to  help  the  students  learn  how  to  learn  and  become  more  self- 
actualized,  the  teacher  needs  to  be  self-actualized  to  foster  it  in  others  (Patterson,  1973).  The  characteristics  Patterson  describes 
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was  also  found  when  Rogers  and  Freiberg  (1994)  talked  to  students  and  found  that  students  wanted  many  of  the  same  tasks 
required  of  a facilitator.  They  discovered  that  students  want  to  be  trusted  and  respected,  wanted  freedom,  a place  where  people 
care,  chances  to  make  decisions,  teachers  as  helpers,  and  teachers  who  help  them  succeed  (Rogers,  1961). 

How  Does  Rogers’  Person-centered  Learning  Theory  Fit  with  Web-based  Learning? 

How  does  Rogers'  person-centered  learning  theory  apply  to  web-based  learning  as  currently  many  web-based  courses  could 
fit  well  under  the  problem-based  constructivist  instructional  model?  Rogers'  theory  would  work  with  web-based  courses  on 
several  levels.  The  first  is  Rogers  felt  his  theory  of  learning  was  most  applicable  in  non-traditional  classes  (Rogers  & Freiberg, 
1994).  Most  people  would  agree  web-based  courses  fit  into  being  non-traditional  classes.  Many  web-courses  also  meet  the  needs 
of  a person-centered  learning  theory  because  the  teacher  assumes  the  position  of  a facilitator  rather  than  the  traditional  role  of 
knowledge  giver.  As  the  teacher  takes  on  the  role  of  facilitator  the  students  take  on  the  roles  of  knowledge  seekers  as  they  focus 
on  what  they  want  to  learn  within  the  class  topics.  The  roles  of  teacher  and  students  is  the  key  focus  of  Rogers’  person-centered 
instruction  because  Rogers  believed  that  the  student  should  be  the  center  of  instruction  and  should  conduct  the  learning  thus 
providing  personal  control  into  the  experience  of  learning  (Rogers  & Freiberg,  1994;  Sahakian,  1970). 

The  major  focus  of  Rogers’  learning  theory  is  that  the  learning  should  be  student  directed  and  web-based  courses  provide  a 
fertile  ground  for  student  directed  learning.  The  nature  of  the  World  Wide  Web  puts  learners  in  a situation  where  they  will  search 
for  information.  Through  their  searches  they  will  experience  success  and  failure,  but  it  is  the  experience  of  directing  their  own 
individual  learning  that  promotes  a growth  of  self  This  growth  will  allow  students  the  opportunities  to  learn  the  process  of 
learning  firsthand  and  allow  them  to  further  develop  themselves  into  their  fullest  potential  as  human  beings. 

Criteria  for  Evaluating  Web-based  Courses  Consistent  with  Rogers’  Approach  to  Learning 

Using  Rogers’  person-centered  learning  theory  I have  developed  nine  criteria  to  determine  if  a web-based  course  fits  with 
Rogers’  approach  to  learning  (See  Table  1).  These  criteria  are  based  on  a person-centered  instructional  design  model  developed 
by  Miller  and  Mazur  (2001).  Using  these  criteria,  an  evaluation  of  a course  can  be  made  to  determine  if  it  meets  the  needs  stated 
by  Rogers  of  a person-centered  learning  experience.  The  criteria  can  be  used  to  not  only  determine  if  a class  is  consistent  with  a 
person-centered  focus  but  also  can  help  instructors  and  instructional  designers  enhance  classes  to  fit  closer  to  Rogers’  approach  to 
learning. 

Table  I.  Person-centered  Instructional  Approach  Criteria  for  Web-based  Courses 


Criteria  Criteria  Description 

1 The  course  provides  an  emphasis  on  the  learners’  interests,  personal  ability,  and  prior  knowledge  of  the  instructional 
topic. 

2 The  facilitative  instructor  should  connect  students’  knowledge  and  interests  with  content  principles  of  the  course. 

3 The  facilitative  instructor  should  select  an  environment  that  supports  collaborative  learning  and  learner  control. 

4 The  facilitative  instructor  allows  students  to  develop  individually  achievable  objectives  based  on  their  interests  and 
abilities  within  the  context  of  the  course.  This  could  be  done  in  the  form  of  a written  contract  with  the  facilitative 
instructor. 

5 The  facilitative  instructor  allows  students  to  develop  individually  achievable  objectives  based  on  their  interests  and 
abilities  within  the  context  of  the  course.  This  could  be  done  in  the  form  of  a written  contract  with  the  facilitative 
instructor. 

6 Learners  work  with  the  facilitative  instructor  to  organize  the  areas  of  interest  to  cover  so  he  or  she  can  meet  the  needs 
of  the  students'  learning  objectives  and  maximize  the  learning  potential. 

7 Learners  work  with  the  facilitative  instructor  to  organize  the  areas  of  interest  to  cover  so  he  or  she  can  meet  the  needs 
of  the  students'  learning  objectives  and  maximize  the  learning  potential. 

8 The  learners  conduct  self-evaluation  based  on  their  individual  learning  objectives  outlined  in  their  learning  contracts. 
The  self-evaluation  should  show  the  significance  of  the  learning  experience  and  could  also  provide  focus  on  levels  of 
personal  involvement,  self-initiated  involvement,  and  learners'  pervasiveness. 

9 Outcomes  of  the  course  should  show  significant  learning.  This  could  include  an  accumulation  of  knowledge  on  the 
topic,  satisfaction  in  the  learning,  desire  to  master  the  experience,  and  a greater  understanding  of  any  problems  and 

potential  resolutions  within  the  content.  The  learner  should  also  experience  increased  levels  of  self-actualization. 


Case  Study  Example  of  the  Application  of  the  Person-centered  Learning  Criteria 

Description  of  the  Video  for  Distance  Learning  and  Multimedia  Course 

The  course  I selected  as  an  example  of  the  application  of  applying  Rogers’  person-centered  instructional  approach  to  a web- 
based  course  was  on  video  design  for  distance  learning  and  multimedia  that  was  taught  during  the  Spring  2000  semester.  The 
case  example  course  was  a graduate  level  course  offered  as  an  elective  in  an  instructional  design  program  from  a southeastern 
university.  The  purpose  of  the  course  was  to  discuss  a variety  of  video  applications  for  multimedia  development  and  distance 
education.  The  course  was  framed  around  concepts  and  principles  of  film  theory  and  cinematic  narrative  research  and  used 
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classroom  exercises  and  projects  focusing  on  basic  video  and  video  production  skills  to  enhance  multimedia  and  distance 
education  course  development. 

The  course  is  described  as  “web-centric”  due  to  a majority  of  the  course  materials  being  available  from  the  course  web  site 
however  the  course  does  include  four  “face-to-face”  meetings  during  the  semester.  The  web  site  provided  web-based  mini- 
lectures, relevant  web-site  links,  and  a threaded  discussion  feature  for  discussions  of  course  readings  and  collaborative 
interaction.  While  the  class  did  meet  in  a face-to-face  setting  four  times  during  the  semester,  students  could  attend  class  at  the 
specified  site  or  they  could  attend  via  2-way  compressed  video  hook-up.  This  attendance  option  allowed  students  outside  the 
central  Kentucky  region  to  take  part  in  the  class  without  coming  to  the  class  meeting  site.  Based  on  the  web-centric  format  of  this 
course,  I felt  this  would  be  an  appropriate  candidate  for  evaluation  based  on  a Rogerian  approach  to  learning.  The  evaluation  was 
conducted  by  reviewing  the  course  syllabus,  course  documents,  and  a brief  interview  with  the  course  instructor.  Each  of  the  nine 
criterion  are  presented  in  italics  followed  by  a statement  of  meeting  or  failing  to  meet  the  criterion  and  a brief  description  of  how 
the  course  meets  or  fails  to  meet  the  criteria. 


Case  Study  Evaluation  Results 

Criterion  1:  The  course  provides  an  emphasis  on  the  learners’  interests,  personal  ability,  and  prior 
knowledge  of  the  instructional  topic. 

The  course  met  criterion  I . The  course  needs  to  accommodate  a wide  range  of  skill-levels  and  abilities  students  bring  to  the 
course.  The  instructor  developed  three  areas  or  constellations,  as  called  in  the  course,  based  on  conceptual,  technical,  and 
application  skills  to  meet  these  needs.  Each  constellation  listed  topics  students  could  select  and  focus  their  personalized 
instruction  upon  during  the  course.  An  example  of  this  selected  focus  was  the  technical  constellation,  which  required  students  to 
choose  5 out  of  1 2 technical  areas  they  wanted  to  master  within  the  course. 

Criterion  2:  The  facilitative  instructor  should  connect  students’  knowledge  and  interests  with  content 
principles  of  the  course. 

The  course  met  criterion  2.  The  application  constellation  of  the  course  required  students  to  select  2 out  of  6 topic  areas  with 
one  of  the  topics  required  of  all  students  connecting  the  technical  areas  students  focused  on  mastering  within  the  course. 

Criterion  3:  The  facilitative  instructor  should  select  an  environment  that  supports  collaborative  learning  and 
learner  control. 

The  course  met  criterion  3.  Because  the  class  only  met  face-to-face  four  times  during  the  semester  the  teacher  used  a threaded 
discussion  list  on  the  Nicenet  web  site  (http://www.nicenet.org).  This  site  allowed  the  instructor  to  create  discussion  topics,  post 
questions,  and  moderate  online  discussions  with  the  students.  Using  the  Nicenet  threaded  discussion  list,  the  students  were  able  to 
post  messages  to  other  students  and  the  teacher  either  publicly  or  privately. 

Criterion  4:  The  facilitative  instructor  allows  students  to  develop  individually  achievable  objectives  based  on 
their  interests  and  abilities  within  the  context  of  the  course.  This  could  be  done  in  the  form  of  a written 
contract  with  the  facilitative  instructor. 

The  course  met  criterion  4.  The  course  met  this  criterion  by  allowing  students  to  set  their  objectives  by  selecting  topic  areas 
that  would  help  them  at  their  present  skill  level  or  help  them  to  advance  to  a higher  skill  level  using  the  technologies  available  to 
them. 

Criterion  5:  The  facilitative  instructor  allows  students  to  develop  forms  of  self-evaluation  to  demonstrate 

significant  learning  based  on  the  individual  students’  learning  objectives. 

The  course  did  not  meet  criterion  5.  While  the  course  had  been  designed  to  allow  students  to  develop  individually  achievable 
objectives  there  were  no  opportunities  within  the  course  that  allowed  students  to  develop  personalized  forms  of  self-evaluation  to 
demonstrate  significant  learning.  The  instructor  set  the  evaluation  methods  used  within  this  course. 

Criterion  6:  Learners  work  with  the  facilitative  instructor  to  organize  the  areas  of  interest  to  cover  so  she  can 
meet  the  needs  of  the  students’  learning  objectives  and  maximize  the  learning  potential. 

Based  on  the  evaluation  using  only  publicly  available  course  information  and  a brief  interview  with  the  instructor  the  course 
cannot  be  evaluated  on  criteria  6.  To  answer  this  question  it  would  be  necessary  to  conduct  observations  of  the  interactions  that 
occur  in  the  learning  environment. 

Criterion  7:  The  facilitative  instructor  identifies,  selects,  and  presents  to  the  learners  resources  to  enhance 
their  learning  experience. 

The  course  met  criterion  7.  The  instructor  provided  a links  page  off  of  the  course  homepage  listing  several  \\eb  sites 
providing  additional  learning  resources  for  the  students.  The  instructor  also  provided  a course  video  with  instructional  clips  as 
well  as  several  mini  web  lectures  students  could  access  to  help  with  their  understanding  of  the  required  course  text. 
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Criterion  8:  The  learners  conduct  self-evaluation  based  on  their  individual  learning  objectives  outlined  in 
their  learning  contracts. 

The  course  did  not  meet  criterion  8.  There  was  no  information  provided  about  learners  conducting  self-evaluations.  Based  on 
the  available  information  the  instructor  evaluated  students  based  on  class  participation,  technical  and  application  constellation 
exercises,  and  the  final  project. 

Criterion  9:  Outcomes  of  the  course  should  show  significant  learning. 

The  course  met  criterion  9.  The  course  had  been  offered  for  three  semesters  with  the  constellation  first  being  introduced  in 
the  second  semester  offered.  Students  rated  the  course  during  the  initial  semester  at  3.6  of  a possible  score  of  5.0.  The  second 
semester  using  the  constellation  approach  has  reached  a 4.2  out  of  a total  of  5.0.  The  instructor  also  found  a high  quality  of  work 
in  the  video  products  presented  by  the  students.  The  students  of  the  second  semester  also  suggested  additional  constellation 
activities  that  have  been  incorporated  into  the  course.  The  suggested  activities  showed  that  students  were  interested  in  further 
accumulation  of  knowledge  on  the  topics  and  wanted  to  suggest  the  learning  opportunities  to  help  future  students. 

Summary  of  the  Course  Evaluation 

Overall  this  course  promotes  a Rogerian  approach  based  on  the  criteria  developed  for  evaluation.  Based  on  information 
available  for  the  evaluation,  the  main  area  of  concern  for  this  course  using  a person-centered  instructional  mode  was  the  lack  of 
student -developed  forms  of  self-evaluation  and  a major  focus  on  teacher  evaluation  of  student  progress.  To  allow  students  to  feel 
they  are  developing  their  own  learning  objectives  it  would  be  recommended  to  also  allow  students  to  develop  individualized 
evaluations  of  their  performance.  Although  the  course  does  not  meet  all  of  the  criteria,  there  has  been  an  obvious  change  in  the 
satisfaction  of  the  course  based  on  the  increase  of  course  ratings  since  the  constellation  approach  was  used.  The  increased  course 
ratings  met  one  of  the  components  of  significant  learning,  which  can  promote  increased  interests  in  the  topics  and  increased 
levels  of  self-actualization  within  the  students.  Based  on  the  results  of  this  case  evaluation,  specifically  the  outcomes  that  were 
reported  from  the  instructor  it  can  be  inferred  that  providing  the  course  with  a Rogerian  person-centered  focus  increased  the 
potential  for  positive  outcomes. 

Conclusions 

Several  conclusions  have  been  made  from  the  application  of  the  Rogerian  person-centered  criteria  to  a web-based  course.  The 
nine  criteria  posited  in  this  paper  can  be  used  to  identify  the  extent  that  a web-based  course  fits  with  Rogers’  person-centered 
learning  theory.  Using  the  person-centered  learning  criteria  can  be  used  to  help  adapt  the  instruction  of  the  course  to  fit  a full 
model  of  person-centered  instruction  such  as  Miller  and  Mazur’s  person-centered  model  of  instruction  (2001).  It  is  necessary  to 
conduct  additional  research  into  the  impact  of  person-centered  instruction  such  as  Miller  and  Mazur’s  model  (2001)  on  web- 
based  courses.  It  will  also  be  necessary  to  begin  studying  what  types  of  instructional  models  work  best  in  web-based  instructional 
learning  environments. 
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Abstract 

What  is  not  fully  understood  about  educational  technology  is  how  to  transform  computer  technology  into  a powerful 
pedagogical  tool.  One  way  to  address  instructional  issues  associated  with  integrating  computer  technology  in  classrooms  is  to 
analyze  educational  best  practices  associated  with  technology  integration  in  classrooms.  Technology  standards  for  teachers  were 
established  and  a tool  to  examine  the  quality  of  computer  technology  integration  was  developed  and  validated  for  a school 
district  undertaking  a district-wide  technology  professional  development  initiative.  Pattern  analysis  procedures  demonstrated 
that  the  measure  was  appropriate  for  determining  the  extent  of  technology  integration  and  for  identifying  appropriate  technology 
training  themes. 

Several  national  organizations  are  currently  involved  in  the  establishment  and  proliferation  of  national  technology  standards 
for  teachers  including  the  International  Society  for  Technology  in  Education  (ISTE)  and  the  National  Council  for  Accreditation 
of  Teacher  Education  (NCATE).  Although  many  reasonable  and  appropriate  technology  standards  for  teachers  exist,  these  goals 
are  often  stated  in  abstract  or  general  terms.  Additionally,  since  there  is  a great  deal  of  variability  in  educational  beliefs, 
technological  availability,  and  state  and  community  expectations,  technology  integration  should  be  locally  defined,  using 
available  research  models  and  national  standards  as  a foundation  (Pierson,  2001). 

We  launched  a technology  professional  development  initiative  in  a school  district  by  establishing  a set  of  technology 
standards  and  indicators  that  clearly  described  educational  best  practices  for  teaching  and  learning  with  technology  that  teachers 
could  implement  in  their  classrooms.  The  technology  standards  on  which  the  professional  development  model  was  established 
was  formulated  by  synthesizing  national,  state,  and  local  technology  standards  and  then  identifying  educational  best  practices  that 
supported  these  standards  within  the  local  context.  We  then  reinforced  a pedagogy  that  enhanced  and  improved  teaching  and 
learning  using  technology  tools  and  resources  through  performance  assessment  and  financial  incentives.  This  study  evaluated  and 
validated  a comprehensive,  standards-based  technology  professional  development  model  for  teachers  that  can  be  customized  for 
each  local  context. 

Theoretical  Framework 

The  integration  of  technology  in  classrooms  and  schools  is  a complex  change  process  that  emails  supporting  curriculum  goals 
through  the  instructional  use  of  computer  technology  to  enhance  student  learning  (Dockstader,  1999).  Educational  change  models 
often  attempt  to  assess  and  explain  the  change  process  in  terms  of  dimensions  or  degrees  of  change.  Consequently,  one  way  to 
better  understand  the  difficult  instructional  issues  associated  with  the  integration  of  computer  technology  in  classrooms  is  to 
examine  how  teachers  implement  computer  technology. 

Several  models  or  strategies  have  been  employed  by  educational  researchers  and  practitioners  to  provide  a systematic 
approach  for  determining  the  quality  of  innovation  implementation.  The  Concems-Based  Adoption  Model  (CBAM)  (Hall, 
Wallace  & Dossett,  1973)  emphasizes  change  as  a developmental  process  experienced  by  individuals  implementing  innovations 
within  an  organizational  context.  CBAM  has  evolved  into  a comprehensive  systemic  change  model  that  allows  change 
investigators  and  facilitators  to  understand  organizational  change  from  the  point  of  view  of  the  persons  affected  by  the  change 
(Surry,  1997). 

CBAM  is  based  on  the  assumption  that  change  is  best  understood  when  it  is  expressed  in  functional  terms — what  persons 
actually  do  who  are  involved  in  the  change.  Since  change  involves  developmental  growth,  the  focus  of  facilitation  is  with 
individuals,  innovations,  and  the  context  (Hord,  Huling-Austin,  & Hall.,  1987).  CBAM  provides  for  the  development  of 
diagnostic  tools  based  on  the  design  of  the  innovation  being  evaluated  and  the  operational  patterns  of  those  using  the  innovation. 

One  such  tool  that  a CBAM  investigator  may  develop  and  use  is  the  Innovation  Configuration  Matrix  or  Map  (ICM).  The 
ICM  delineates  an  innovation  in  the  form  of  a two-dimensional  matrix  along  a scale  that  renders  closer  approximations  of 
conceptualized  implementation  or  use  along  one  dimension  of  the  matrix  and  the  various  configuration  components  along  the 
other  dimension  of  the  matrix.  The  ICM  has  relevance  for  instructional  designers  and  educational  practitioners.  Rather  than  being 
a static  measure,  the  ICM  provides  a procedural  definition  of  the  specific  educational  components  and  features  of  the  innovation 
within  the  context  in  which  it  is  being  implemented. 

Research  questions  about  educational  technology  are  better  formulated  when  they  are  concerned  with  issues  of  instructional 
quality  and  productivity.  This  study  developed  and  validated  an  ICM  for  a school  district’s  technology  professional  development 
model  based  on  technology  standards  and  educational  best  practices  associated  with  these  standards.  The  ICM  was  used  as  a tool 
to  analyze  the  integration  of  computer  technology  in  classrooms  in  the  school  district. 
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Methods 


Instrumentation 

An  instrument  for  analysis  of  technology  integration  and  implementation  in  classrooms  was  developed.  This  instrument,  the 
Technology  Standards  Integration  Configuration  Matrix  (TSICM)  was  based  on  a consensus-building  process  that  followed  a 
procedure  developed  by  Heck,  Steigelbauer,  Hall,  and  Loucks.  (1981)  and  used  previously  by  the  researcher  (Mills  & Ragan, 
2000).  Relevant  national,  state,  and  local  technology  standards  were  reviewed  and  evaluated  by  the  researcher  in  conjunction 
with  the  district  technology  committee  and  technology  coordinator.  The  committee  agreed  upon  18  technology  integration 
standards  that  were  appropriate  for  the  school  district.  Technology  integration  standards  were  organized  into  three  skill  sets  or 
phases:  Using  and  Operating  Technology  in  the  Classroom  (Standards  1-6),  Facilitating  and  Managing  Classroom  Technology 
(Standards  7-12),  and  Technology  Integration  (Standards  13-18).  Each  successive  phase  was  intended  to  identify  a $t  of 
instructional  strategies  that  exemplified  a more  appropriate  application  of  technology  or  a higher  quality  of  technolog>' 
integration  into  classroom  instruction  and  learning. 

Each  technology  standard  was  established  as  a component  of  the  TSICM  and  then  variations  for  each  component  were 
identified.  Variations  for  each  component  consisted  of  discrete  categorizations  of  technology  implementation  for  the 
corresponding  component.  Component  variations  were  designed  to  represent  teacher  classroom  practices  along  a continuum  from 
unacceptable  use  to  ideal  use.  The  component  variations  were  refined  by  the  technology  committee  to  reflect  the  actual  practices 
of  teachers  using  computer  technology  in  classrooms.  The  components  and  component  variations  were  organized  into  matrix 
comprised  of  four  variations  for  each  of  the  18  components  with  each  successive  variation  indicating  a level  of  use  representing  a 
closer  approximation  of  ideal  or  appropriate  educational  use.  The  TSICM  was  deployed  as  a paper-  and  web-based  checklist. 

Data  Collection 

The  school  district  used  in  this  study  was  located  in  a small  town  in  a Midwestern  state.  The  school  district  had  a total 
enrollment  of  almost  2,200  students  in  grades  K-12  with  147  certified  teachers.  Computer  technology  was  used  in  all  the  schools 
in  the  district.  All  schools  except  the  high  school  had  computer  labs  and  all  teachers  had  classroom  computers. 

The  school  district  had  made  a substantial  investment  in  computer  technology  and  was  beginning  a district-wide  technology 
professional  development  initiative.  To  collect  data  regarding  computer  technology  implementation  occurring  among  teachers,  all 
teachers  at  all  grade  levels  were  provided  with  a paper  version  of  the  TlS'/CAf  checklist  and  the  option  to  complete  a web-based 
version  of  the  TSICM  checklist  on  the  school  district  web  site.  The  checklist  was  designed  in  a multiple-choice  format  in  which 
respondents  could  select  more  than  one  response  for  each  TSICM  component. 

Data  collection  occurred  at  both  the  start  and  end  of  a school  year.  A usable  TSICM  was  completed  by  70  teachers  at  the  start 
of  the  school  year  and  84  teachers  at  the  end  of  the  school  year.  57  teachers  completed  both  the  start  and  end  of  year 
administration  of  the  TSICM. 

Data  Analysis 

The  rubric  for  recording  teacher  responses  on  the  checklist  was  to  rate  to  the  highest  level  of  use  for  each  component  on  the 
checklist.  The  responses  to  the  TSICM  checklist  were  analyzed  by  cluster  analysis  to  identify  relatively  homogenous  groups  of 
cases  based  on  the  TSICM  components.  Discriminant  analysis  (DA)  was  then  used  to  assess  the  adequacy  of  the  groupings  from 
the  cluster  analysis  by  using  the  TSICM  implementation  components  as  predictor  variables.  A step-wise  methodology  was  used  to 
enter  variables  into  the  discriminant  functions.  One-way  analysis  of  variance  was  used  to  determine  if  the  component  attributes  of 
each  group  were  statistically  significant.  Comparisons  were  made  between  the  start  and  end  of  year  data  collections  using  a 
paired-samples  /-test.  Descriptive  statistics  for  the  data  collections  are  provided  in  Table  1. 


Table  1.  Descriptive  Statistics  for  Start  and  End  of  Year  Administration  of  TSICM. 


TECHNOLOGY  STANDARD 

Start  of  Year 

Administration 

(N=70) 

End  of  Year 
Administration  (N=84) 

SUM 

MEAN 

STD. 

DEV. 

SUM 

MEAN 

STD. 

DEV. 

1.  Operate  common  technology  input  devices. 

217 

3.10 

.82 

313 

3.73 

.73 

2.  Perform  basic  file  management  tasks. 

206 

2.94 

1.11 

316 

3.76 

.63 

3.  Apply  trouble-shooting  strategies  and  install  software. 

226 

3.23 

.87 

318 

3.79 

.70 

4.  Use  software  productivity  tools. 

182 

2.60 

1.34 

297 

3.54 

1.01 

5.  Use  technology  to  communicate  and  collaborate. 

228 

3.26 

.72 

316 

3.76 

.59 

6.  Use  technology  to  collect  data  and  perform  research. 

188 

2.69 

1.10 

286 

3.40 

.95 

7.  Model  responsible  use  of  technology. 

174 

2.49 

1.42 

274 

3.26 

1.13 

8.  Facilitate  regular  student  use  of  computer  technology. 

208 

2.97 

1.45 

257 

3.06 

1.43 

9.  Conduct  learning  activities  using  computer  technology. 

187 

2.67 

1.43 

234 

2.79 

1.46 

10.  Select  appropriate  technology  resources  for  classroom  use. 

83 

1.19 

1.33 

194 

2.31 

1.69 

1 1 . Evaluate  the  validity  of  data  collected  using  technology. 

22 

.31 

.91 

93 

1.11 

1.69 
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12.  Use  technology  to  present  classroom  instruction. 

154 

2.20 

1.16 

235 

2.80 

1.23 

13.  Integrate  technology -based  learning  experiences  into  classroom 
instruction. 

138 

1.97 

1.43 

207 

2.46 

1.48 

14.  Use  computer  technology  for  problem-solving  and  critical  thinking. 

118 

1.69 

1.48 

199 

2.37 

1.48 

15.  Use  technology  to  facilitate  individualized/cooperative  learning 
experiences , 

94 

1.34 

1.39 

157 

1.87 

1.40 

16,  Assess  student  use  of  technology  using  multiple  methods  of 
evaluation. 

66 

.94 

1.57 

91 

1.08 

1.53 

17,  Develop  and  maintain  electronic  student  portfolios. 

23 

.33 

.88 

48 

.57 

1.01 

18,  Use  computer  technology  to  maintain  and  analyze  student 
performance. 

136 

1.94 

1.23 

224 

2.67 

1.08 

RESULTS 

Since  the  initial  cluster  centers  and  the  number  of  dominant  patterns  were  unknown,  cluster  analysis  was  performed  on  the 
first  administration  of  the  TSICM  using  all  1 8 components  of  the  TSICM  and  incrementing  the  number  of  clusters  until  a 
reasonable  model  was  obtained,  The  cluster  analysis  was  run  for  2,  3,  4,  and  5 clusters  before  a reasonable  model  was  selected.  A 
reasonable  model  occurred  with  the  number  of  clusters  set  at  3.  When  the  number  of  clusters  was  set  at  3,  the  number  of  cases  in 
Group  1 was  21 , Group  2 was  33,  and  Group  3 was  16.  In  order  to  make  comparisons  between  the  start  and  end  of  year  data,  this 
same  grouping  model  (3  clusters/groups)  was  used  for  analysis  of  the  end  of  year  data  collection. 

In  order  to  assess  the  adequacy  of  the  classification  of  implementation  pattern  groups  by  the  cluster  analysis  a Discriminant 
Analysis  (DA)  was  performed,  The  1 8 TSICM  components  were  used  to  separate  the  groups  into  the  discriminant  functions.  As  a 
result  of  this  procedure  97%  of  the  cases  or  68  of  70  cases  were  correctly  classified.  The  DA  reclassified  1 case  in  Group  2 for 
Group  3 and  1 case  in  Group  3 for  Group  2. 

The  TSICM  components  were  entered  into  the  DA  using  a stepwise  model  in  order  to  discard  variables  that  were  weakly 
related  to  group  distinctions.  Table  2 identifies  the  unstandardized  discriminant  coefficients  for  each  TSICM  component  that  best 
predicted  the  discriminant  functions  for  the  start  of  the  year  administration.  Based  on  the  discriminant  coefficients.  Component 
13 — Integrate  Technology -based  Learning  Experiences  into  Classroom  Instruction  made  the  most  important  contribution  to 
Function  1 and  Component  8 — Facilitate  Regular  Student  Use  of  Computer  Technology  made  the  most  important  contribution  to 
Function  2,  Teachers  identified  with  Group  1 (Technology  Operators)  were  characterized  by  low  or  inverse  relationships  to 
Functions  1 and  2,  Group  2 (Technology  Facilitators)  by  high  Function  2,  and  Group  3 (Technology  Integrators)  by  high 
Function  1 , 


Table  2.  Canonical  Discriminant  Function  Coefflcients. 


TSICM  Component 

Function 

1 

2 

8,  Facilitate  regular  student  use  of  computer  technology. 

.196 

.756 

9,  Conduct  learning  activities  using  computer  technology. 

.471 

.072 

10.  Select  appropriate  technology  resources  for  classroom  use. 

.415 

-.496 

12.  Use  technology  to  present  classroom  instruction. 

.177 

-.539 

13.  Integrate  technology -based  learning  experiences  into  classroom  instruction. 

.590 

.402 

1 6,  Assess  student  use  of  technology  using  multiple  methods  of  evaluation. 

.408 

-.716 

1 8,  Use  computer  technology  to  maintain  and  analyze  student  performance. 

.175 

.568 

A cluster  analysis  was  performed  on  the  end  of  year  data  collection  with  the  number  of  clusters  set  at  3 to  compare  with  the 
clusters  from  the  first  of  year  data  collection.  With  the  number  of  clusters  set  at  3,  the  number  of  cases  in  Group  1 was  35,  Group 
2 was  1 8,  and  Group  3 was  3 1 . The  DA  was  repeated  for  the  end  of  year  data  collection  of  the  TSCIM  to  make  comparisons  with 
the  first  of  year  data  collection.  As  a result  of  this  procedure  92%  of  the  cases  or  77  of  84  cases  were  correctly  classified.  The  DA 
reclassified  1 case  in  Group  1 for  Group  2 and  6 cases  in  Group  3 for  Group  1 . Table  3 identifies  the  unstandardized  discriminant 
coefficients  for  each  TSICM  component  that  best  predict  the  discriminant  functions  for  the  start  of  the  year  administration  of  the 
TSICM,  Based  on  the  discriminant  coefficients.  Component  9 — Conduct  Learning  Activities  using  Computer  Technology  made 
the  most  important  contribution  to  Function  1 while  Component  1 — Operate  Common  Technology  Input  Devices  made  the  most 
important  contribution  to  Function  2.  Teachers  identified  with  Group  1 (Expert  Technology  Users/Operators)  were  characterized 
by  high  Function  2,  Group  2 (Beginning  Technology  Users/Operators)  by  low  or  inverse  relationships  to  Functions  1 and  2,  and 
Group  3 (Technology  Facilitators)  by  high  Function  1 . 


Table  3.  Canonical  Discriminant  Function  Coejjficients. 


TSICM  Component 

Function 

1 

2 

1.  Operate  common  technology  input  devices. 

.317 

.764 

9.  Conduct  learning  activities  using  computer  technology. 

.614 

.540 

10.  Select  appropriate  technology  resources  for  classroom  use. 

.409 

-.146 

1 1 . Evaluate  the  validity  of  data  collected  using  technology. 

.531 

-.774 

14.  Use  computer  technology  for  problem-solving  and  critical  thinking. 

.602 

.287 

Paired  samples  correlations  for  each  of  components  of  the  TSICM  (technology  standards)  were  computed  for  matched  cases 
on  the  start  and  end  of  year  administrations  of  the  TSICM.  (see  Table  4).  Almost  all  components  of  the  TSICM  indicated 
significant  differences  on  the  r-test  (p<.05)  between  the  start  and  end  of  year  administrations.  Additionally,  paired  samples 
correlations  were  computed  when  TSICM  components  were  grouped  by  skill  set  or  phase  (see  Table  5)  and  significant  differences 
on  the  Hesi  (p<.05)  were  indicated  for  all  three  phases. 

Conclusions 

An  interesting  conclusion  we  deduced  from  the  start  of  year  data  collection  for  this  population  of  teachers  was  that 
proficiency  in  the  use  and  operations  of  computer  technology  (Phase  1)  was  not  necessarily  a distinguishing  attribute  of  high 
quality  technology  integration.  Differences  among  the  groups  in  this  study  at  the  beginning  of  the  school  year  were  delineated 
more  by  attributes  of  technology  integration  than  by  technology  use  and  operations.  This  finding  had  relevance  for  the  provision 
of  technology  professional  development  activities.  These  results  clearly  demonstrated  that  technology  training  activities  for  this 
school  district  needed  to  focus  more  on  instructional  strategies  and  methods  to  integrate  technology  in  the  classroom  than  on 
training  activities  to  increase  skills  in  the  operation  of  computer  hardware  and  use  of  software  applications. 

Table  4^  Paired  Samples  Correlations  by  Technology  Standard  for  Start  and  End  of  Year  Administrations  of  TSICM  (N-57). 


TSICM  Component  (Technology  Standard) 

Correlation 

t 

1.  Operate  common  technology  input  devices. 

.273 

-4.375* 

2.  Perform  basic  file  management  tasks. 

.190 

-4.169* 

3.  Apply  trouble-shooting  strategies  and  install  software. 

.330 

-4.119* 

4.  Use  software  productivity  tools. 

.260 

-5.314* 

5.  Use  technology  to  communicate  and  collaborate. 

.094 

-2.950* 

6.  Use  technology  to  collect  data  and  perform  research. 

.002 

-2.713* 

7.  Model  responsible  use  of  technology. 

.419 

-4.158* 

8.  Facilitate  regular  student  use  of  computer  technology. 

.659 

-.882 

9.  Conduct  learning  activities  using  computer  technology. 

.696 

-1.135 

10.  Select  appropriate  technology  resources  for  classroom  use. 

.374 

-4.820* 

11.  Evaluate  the  validity  of  data  collected  using  technology. 

.373 

-3.853* 

12.  Use  technology  to  present  classroom  instruction. 

.379 

-3.340* 

13.  Integrate  technology -based  learning  experiences  in  classroom  instruction. 

.586 

-3.040* 

14.  Use  computer  technology  for  problem-solving  and  critical  thinking. 

.596 

-4.428* 

15.  Use  technology  to  facilitate  individualized/cooperative  learning  experiences. 

.485 

-2.573* 

16.  Assess  student  use  of  technology  using  multiple  methods  of  evaluation. 

.474 

-.338 

17.  Develop  and  maintain  electronic  student  portfolios. 

.581 

-2.379* 

18.  Use  computer  technology  to  maintain  and  analyze  student  performance. 

.320 

-2.982* 

Total  Score  All  Standards 

.708 

-7.447* 

*Significant  at  .05  level  for  two-tailed  test 

Table  5.  Paired  Samples  Correlations  by  Type  or  Technology  Standard  for  Start  and  End  of  Year  Administrations  of  TSICM 
(N-57). 


Skill  Set/Phase  of  Technology  Integration 

Correlation 

t 

Using  and  Operating  Technology  in  the  Classroom 

.441 

-6.777* 

Facilitating  and  Managing  Classroom  Technology 

.746 

-6.141* 

Integrating  Classroom  Technology 

.638 

-4.058* 

*Significant  at  .05  level  for  two-tailed  test 


By  the  end  of  the  school  year,  the  characteristics  that  delineated  differences  among  the  teachers  in  technology  integration  was 
more  sharply  defined  by  teachers  who  were  beginning  or  expert  operators  of  computer  technology  and  those  who  were 
facilitators  and  managers  of  classroom  technology.  Thus,  there  was  a clear  progression  among  the  teachers  from  technology 
operations  to  technology  facilitation.  While  the  technology  professional  development  program  at  the  school  did  not  make 
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technology  integrators  out  of  all  participants,  it  clearly  accommodated  reasonable  growth  and  advancement  in  the  technology 
integration  skills  of  the  participants.  When  we  considered  only  those  teachers  for  whom  we  had  both  start  and  end  of  year  data,  a 
significant  pattern  of  growth  across  standards  and  at  all  skill  levels  was  indicated.  This  observation  suggests  that  when 
educational  best  practices  for  teaching  and  learning  with  technology  are  clearly  defined  and  established,  the  professional  skills  of 
teachers  will  begin  to  exemplify  the  stated  expectations. 

We  have  learned  from  this  study  that  classroom  technology  integration  was  not  so  much  about  the  quantity  of  teacher 
interactions  with  technology,  but  rather  it  was  about  the  quality  of  teacher  interactions  with  technology.  When  teacher 
interactions  with  technology  were  accompanied  by  expert  teaching  practices  and  related  to  curriculum  objectives,  the  quality  of 
technology  integration  was  increased.  Over  time  we  have  refined  our  technology  integration  professional  development  model  to 
include  more  powerful  technology  integration  strategies  in  classrooms  beyond  that  of  computer  technology  use  and  operations. 
We  have  learned  that  through  the  establishment  of  a well-defined  set  of  pedagogical  standards  and  indicators,  higher  levels  of 
technology  integration  in  classrooms  can  be  identified  and  achieved.  Consequently,  when  teachers  know  how  to  use  and  then 
actually  use  all  the  tools  at  their  disposal,  classroom  pedagogy  is  expanded  and  improved. 

Although  many  school  districts  have  established  benchmarks  or  standards  for  the  integration  of  technology  in  classrooms,  no 
model  or  methodology  exists  for  substantiating  technology  standards  with  actual  classroom  practices.  The  TSICM  represents  a 
flexible  and  adaptable  approach  to  the  evaluation  of  technology  integration  in  classrooms  because  the  TSICM  components  reflect 
a set  of  widely -used  standards  that  can  be  contextualized. 

A methodology  to  provide  comprehensive  and  continuous  analysis  of  technology  implementation  is  needed  to  sustain  high 
levels  of  use  and  integration  of  computer  technology  in  classrooms.  This  study  demonstrated  that  the  TSICM  is  an  effective  tool 
to  determine  technology  integration  in  classrooms,  to  reveal  the  technology  integration  characteristics  of  teachers  integrating 
technology  in  classrooms,  and  to  distinguish  appropriate  technology  training  themes  that  focus  on  specific  technology  standards. 
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Abstract 

We  explore  cases  of  three  classes  that  implemented  online  teaching  / learning  technologies  as  part  of  a university  wide 
faculty  development  grant  program.  We  examined  students'  satisfaction  with  key  components  of  the  learning  experience. 
Instruments  included  students  ’ pre-assessments  and  post  assessments,  an  instructors  ’ post  survey,  course  syllabi,  grant 
proposals,  and  instructors  ’ end  of  grant  reports.  We  present  implications  for  campus  wide  implementation  of  online  teaching/ 
learning  technologies.  We  include  discussion  of  a model  for  implementing  innovative  technologies  campus  wide  (online  teaching 
support  model). 

Background 

In  1997,  faculty  interest  in  online  teaching  at  California  State  University  San  Bernardino  (CSUSB)  started  increasing.  By 
1999,  the  number  of  faculty  who  wanted  to  do  online  teaching  had  grown  significantly.  Faculty  faced  many  questions  and  issues 
as  they  embarked  to  do  online  teaching.  The  Teaching  Resource  Center  (TRC),  the  faculty  development  unit  that  support 
teaching  and  innovative  instruction,  also  faced  the  task  of  helping  in  faculty’s  new  role  as  online  teachers. 

To  address  these  needs,  TRC  drew  upon  instructional  design  and  evaluation  models  (Van  Slyke,  Kittner  & Belanger,  1998; 
Belanger  & Jordan,  2000;  Dick  & Carey,  1996;  Salisbury,  1996;  Seels  & Glasgow,  1998)  and  identified  the  major  steps  involved 
in  course  development  as  applied  to  online  teaching.  TRC  developed  a systematic  and  holistic  plan  that  serves  both  as  the  map 
and  the  glue  that  holds  the  various  phases  and  players  together  (Santiago,  2001).  This  systematic  plan  has  7 major  steps: 


Figure  2.  Online  Teaching  Support  Model:  7 Major  Steps 

Planning  and  Preparation.  The  instructor  studies  the  big  picture  and  the  instructional  and  non-instructional  issues  that  will  have 
an  impact  on  successful  online  teaching.  Addressing  issues  up  front  gives  a good  estimate  of  how  much  time  and  effort  need  to 
be  invested,  and  helps  determine  one’s  readiness  for  online  teaching. 

Funding.  Faculty  are  encouraged  to  seek  funding  through  grant  programs  that  lead  to  a course  buyout,  money  for  resources 
and/or  hire  a student  assistant 

Instructional  Design  and  Training:  Designing  lessons  for  online  teaching  is  not  simply  converting  existing  materials  into  digitized 
format.  Skills  and  training  for  goal  analysis,  task  analysis,  assessment,  instructional  materials  development,  and  the  use  of 
hardware  and  software  are  addressed  in  this  step. 

Instructional  Materials  Development  and  Testing.  This  step  involves  developing  instructional  materials,  testing  them,  and 
making  sure  that  they  can  be  delivered  successfully  online,  and  that  users  will  be  able  to  access  them  with  minimal,  if  not  totally 
free  of  instructional  and  technological  glitches. 

Implementation.  The  course  is  actually  taught  online.  The  instructional  design  is  put  into  action,  the  instructional  materials  are 
used,  and  technology  is  relied  upon  to  deliver  instruction,  all  towards  the  achievement  of  learning  goals. 
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Evaluation.  Evaluation  identifies  the  strengths  and  weaknesses  of  the  instructional  design,  of  instructional  materials,  and  delivery 
medium.  Most  importantly,  it  measures  the  extent  to  which  goals  were  met. 

Dissemination.  Innovation  and  change  that  is  well  planned  and  appropriately  evaluated  also  results  in  lessons  learned  and  quality 
products.  When  reported  to  a professional  community,  this  leads  to  further  validation,  peer  evaluation,  and  replication,  which 
then  leads  to  scholarship  of  teaching  (Hutchings  and  Schulman,  1999). 


Description  of  Grant  Program 

To  support  online  teaching,  TRC  sponsors  the  Web-based  Course  Development  Grant  program  that  awards  summer  stipend 
($5,000)  to  faculty.  The  course  redesign  or  development  could  range  from  using  a significant  combination  of  online  features  as 
teaching  enhancement,  or  to  deliver  the  whole  course  online.  When  first  offered  in  1999,  12  faculty  proposals  were  funded.  The 
Course  Development  Grant  Program  was  funded  by  the  Office  of  the  Provost.  Administrative  support  for  online  teaching  also 
resulted  in  student  support  through  the  Student  Technology  Support  Center,  the  establishment  of  an  Office  for  Distributed 
Learning  and  the  hiring  of  a librarian  to  support  distance  courses. 


Analysis  of  the  Efficacy  of  the  Grant  Program 

To  inform  our  analysis  of  the  efficacy  of  the  online  components  of  the  courses,  we  utilized  a variety  of  assessment  tools 
including  pre  and  post  surveys  of  students,  a post  survey  of  faculty,  and  an  analysis  of  course  syllabi,  grant  proposals  and  final 
reports. 

Comparison  data  from  the  three  classes  in  which  we  had  sufficient  data  from  both  the  student  pretest  and  posttest  are  listed 
below.  We  report  results  from  a comparison  of  the  uses  of  online  technologies  in  the  three  courses.  We  examined  student  data 
(pretest  and  posttest),  faculty  data  (free  response  and  multiple  choice  questions),  course  syllabi,  and  the  course  development 
proposal.  Our  intention  was  to  utilize  triangulation  in  an  attempt  to  ascertain  factors  that  led  to  successful  and  efficacious  uses  of 
online  technologies  from  both  the  faculty  and  the  student  perspectives. 

Three  classes  that  were  offered  in  the  1999-2000  academic  year  were  examined.  The  description  of  each  course  is  listed  as  it 
appeared  in  the  course  syllabi  and/or  the  catalog  of  programs: 

1.  English  as  a Second  Language  (EESL)  - Research  in  English  as  a Second  Language:  Covers  various  perspectives  used  to 
guide  research  in  TESOL.  Includes  analysis,  discussion,  reflection  and  writing  about  key  issues  and  concepts  in  research. 
Students  will  design  and  carry  out  literature  review  working  towards  a theoretical  framework  to  gain  foundation  as  expert 
practitioner  or  future  researcher. 

2.  Criminal  Justice  (CJUS)  - Research  Methods  in  Criminal  Justice:  Introduction  to  scientific  methodology  and  research 
designs  used  to  conduct  basic  and  applied  research  in  the  criminal  justice  field.  Emphasis  on  scientific  operationalization, 
survey  methodology,  and  concepts  of  evaluation  design. 

3.  Information  Sciences  (INFO)  - Decision  Support  Systems:  Formal  hformation  systems  that  support  organizational 

decision-making.  Topics  include  the  strategy,  framework,  design,  implementation  and  evaluation  of  decision  support 
systems.  Students  will  create  and  apply  decision  support  systems  to  planning,  coordinating,  organizing,  controlling  and/or 
directing  tasks. 


Pre-Survey 

In  the  student  pre  survey,  we  obtained  data  on  three  categories:  student  demographics,  student  background,  and  student 
experience  on  the  use  of  technology.  A summary  of  demographic  data  for  61  students  who  took  the  pretest  in  the  3 classes  is 
listed  below. 

Table  L Student  Demographics 


Demographics 


CJUS 

EESL 

INFO 

Level 

Graduate 

3% 

91% 

5% 

Senior 

40% 

90% 

Junior 

47% 

5% 

Degree  sought 

Masters 

10% 

100% 

15% 

Bachelors 

90% 

75% 

Age 

40’s  & 50’s 

27% 

20% 

30’s 

7% 

45% 

15% 

20’s 

87% 

27% 

65% 

Gender 

Male 

43% 

18% 

45% 

Female 

53% 

82% 

55% 

Miles  from  campus 

> 50  mi. 

10% 

18% 

5% 

40-49  mi. 

3% 

9% 

10% 

30-39  mi. 

17% 

0% 

20% 

20-29  mi. 

17% 

9% 

5% 

10-19  mi. 

20% 

0% 

15% 

<9  mi. 

27% 

55% 

40% 

Table  2.  Student  Background 
Background 

Reason  for  taking  course 


Hours  to  spend  on  course 


Same  instructor  before 


CJSU 

EESL 

INFO 

Recommended 

27% 

9% 

25% 

Required 

83% 

100% 

70% 

Interesting 

17% 

18% 

30% 

Fits  schedule 

37% 

18% 

60% 

>=  17hrs 

3% 

36% 

0% 

13-16  hrs 

3% 

9% 

0% 

9-12  hrs 

23% 

36% 

10% 

5-8  hrs 

57% 

18% 

75% 

0-4  hrs 

13% 

0% 

15% 

Yes 

37% 

73% 

10% 
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As  shown  above,  the  majority  of  the  students  were  in  upper  division  working  for  a bachelor’s  degree  except  for  the  Education 
masters  students  (EESL).  Most  of  the  students  were  in  their  20’ s and  30 ’s.  Female  students  were  in  the  majority,  especially  in 
the  Education  course.  Almost  half  of  the  students  live  close  to  campus  (<  9 miles)  except  for  the  CJUS  students  who  are 
distributed  in  various  distances  from  campus. 

For  background  data,  we  surveyed  students’  reasons  for  taking  the  course,  the  number  of  hours  that  they  intended  to  spend  on 
the  course  per  week,  and  whether  they  have  had  the  instructor  before.  Data  from  this  component  of  the  survey  is  listed  below. 

In  general,  students  were  taking  the  course  because  it  was  required.  A secondary  reason  was  that  it  fits  their  schedule.  The 
majority  of  the  undergraduate  students  planned  to  spend  5-12  hours  per  week  on  the  course,  with  graduate  students  planning  to 
spend  more  than  17  hours  per  week  on  their  course.  Most  of  these  graduate  students  have  had  a course  with  the  instructor  before. 

We  also  surveyed  students’  use  of  and  comfort  level  with  online  technologies.  A summary  of  students’  use  of  technology  is 
given  below: 

Table  3.  Student  Use  of  Technology 


Use  of  technology 
Access  at  home 


Used  in  previous  course 


Comfort  level 
(5=highest) 


Computer 

Internet 

Online  syllabus 
Online  Lectures 
Online  Tests 
Online  Research 
Online  Discussion 
Email 

Chat  Rooms 

Using  computers 
Using  internet 
Using  Chat  Rooms 
Using  Email 
Taking  Online  Course 
Joining  Online  Discussion 


CJUS 

EESL 

INFO 

93% 

91% 

100% 

87% 

91% 

90% 

47% 

73% 

40% 

33% 

73% 

30% 

7% 

82%- 

20% 

70% 

91% 

80% 

10% 

36% 

20% 

77% 

91% 

80% 

23% 

9% 

25% 

4.3 

4.0 

4.8 

4.3 

3.9 

4.7 

3.2 

3.4 

4.1 

4.1 

4.6 

4.6 

3.2 

3.8 

3.6 

2.8 

3.5 

3.7 

Based  on  the  data,  student  access  to  computers  and  Internet  at  home  is  very  high  (87-100%).  Of  the  three  groups,  the 
graduate  students  had  the  most  experience  in  the  use  of  online  technology  in  a course.  Online  experience  is  high  for  research  and 
email  use  in  all  three  courses.  Students  are  most  comfortable  in  the  use  of  computers,  Internet  and  email. 

Post-Survey 

In  the  student  post  survey  we  coupled  the  internally  developed  and  tested  Student  Evaluation  of  Learning  Effectiveness  (or 
SELE,  a self  report  measure  of  learning  that  utilized  a 5 point  Likert  scale  and  which  had  been  approved  for  use  in  evaluation  of 
teaching  by  the  Faculty  Senate),  with  a self-report  instrument  concerning  technology  use  in  the  course. 

One  particularly  striking  result  involves  the  post  survey  item  concerning  how  well  the  online  learning  experiences  fostered 
interaction  and  teamwork  among  class  members.  In  INFO,  this  was  unmistakably  the  most  highly  rated  item  (mean  = 4.5  of  5). 
In  the  other  two  courses,  EESL  and  CJUS,  this  was  clearly  the  lowest  rated  item  (mean  = 3.1  of  5,  mean  = 2.9  of  5,  respectively). 
In  comparing  scores  of  both  EESL  and  CJUS  with  INFO,  two  tailed  t-tests  indicated  significance  with  p<.  01. 
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Table  4,  Online  Components  of  Course  Fostered  Interaction  and  Teamwork 


Course  fostered  interaction  and  teamwork 

CJUS 

EESL 

INFO 

5=Excellent 

29% 

18% 

59% 

4=Very  Good 

0% 

9% 

35% 

3=Satisfactory 

21% 

45% 

6% 

2=Poor 

29% 

18% 

0% 

1=N/A 

21% 

9% 

0% 

Mean 

2.9 

3.1 

4.5 

We  found  the  highest  mean  posttest  scores  in  the  answers  to  three  questions  that  related 
involvement  of  the  teacher  in  the  learning  process. 

Table  5.  Satisfaction  With  Course  Items  Controlled  by  Instructor 

CJUS 

to  the  structure  of  the  courses 
EESL  INFO 

Matched  objectives  with  assessment 

5 = Excellent 

50% 

27% 

41% 

4 = Very  Good 

36% 

36% 

47% 

3 = Satisfactory 

7% 

36% 

12% 

2 = Poor 

0% 

0% 

0% 

1 =N/A 

7% 

0% 

0% 

Mean 

4.2 

3.9 

4.3 

Utilized  clear  grading  criteria 

5 = Excellent 

43% 

55% 

47% 

4 = Very  Good 

36% 

0% 

41% 

3 = Satisfactory 

14% 

27% 

12% 

2 = Poor 

0% 

18% 

0% 

1 =N/A 

7% 

0% 

0% 

Mean 

4.1 

3.9 

4.4 

Instructor  contributed  to  learning 

5 = Excellent 

43% 

36% 

35% 

4 = Very  Good 

50% 

36% 

53% 

3 = Satisfactory 

0%. 

27% 

12% 

2 = Poor 

0% 

0% 

0% 

1 =N/A 

7% 

0% 

0% 

Mean 

4.2 

4.1 

4.2 

ERIC 


2302  BESTCOPYAVAIWBLE 


Table  6.  Did  Online  Components  Foster  Discussion  on  Multicultural  and  Diversity  Issues? 
Fostered  discussion  on  multicultural  and  diversity  issues 


CJUS 

EESL 

INFO 

5 = Excellent 

29% 

27% 

18% 

4 = Very  Good 

14% 

18% 

35% 

3 = Satisfactory 

21% 

36% 

29% 

2 = Poor 

14% 

18% 

18% 

1 =N/A 

21% 

0% 

0% 

To  the  question:  “The  course  matched  objectives  with  assessment”,  the  mean  for  CJUS  was  4.2,  for  EESL,  it  was  3.9,  and  for 
INFO,  it  was  4.3.  “Utilizing  clear  grading  criteria  was  also  rated  high  by  all  three  courses,  with  INFO  mean  at  4.4,  CJUS  at  4.1 
and  EESL  at  3.9.  All  three  courses  had  almost  the  same  means  for  the  question  “Instructor  contributed  to  learning”  (CJUS  = 4.2, 
EESL  = 4.1,  INFO  = 4.2).  In  tandem,  this  data  would  be  an  indicator  of  clear  grading  criteria  that  connected  with  stated 
objectives  of  the  courses. 

The  lowest  means  reflected  responses  to  whether  the  online  components  of  the  course  “Fostered  discussion  on  multicultural 
and  diversity  issues”. 

When  it  came  to  identifying  who  contributed  to  the  learning  experience,  the  instructor  was  ranked  highest,  based  on  the  mean 
scores.  EESL  ranked  “self’  equally  with  the  instructor,  while  CJUS  ranked  “self’  as  second  and  other  students  as  lowest. 
Interestingly,  concerning  who  contributed  to  their  learning,  INFO  ranked  “Other  student/s”  equally  with  the  instructor.  For  both 
CJUS  and  EESL  this  was  the  lowest  rated  item  of  the  three. 

Table  7.  Rating  For  How  Well  the  Instructor,  the  Student,  and  Other  Students  Contributed  to  the  Learning  Experience 


Contributed  to  learning  experience 

CJUS 

EESL 

INFO 

Instructor 

4.2 

4.1 

4.2 

Self 

3.6 

4.0 

4.0 

Other  Student/s 

3.4 

3.5 

4.2 

We  also  surveyed  how  satisfied  students  were  with  online  course  materials.  Satisfaction  was  rated  high  by  all  three  courses. 
However,  when  asked  if  on  line  materials  helped  facilitate  learning,  means  for  all  three  courses  were  low. 

Table  8,  Online  materials  survey  results 


CJUS 

EESL 

INFO 

Satisfied  with  online  materials 

4.7 

4.1 

4.2 

Online  materials  facilitated  learning 

2.3 

3.4 

3.5 

For  all  3 courses  examined,  online  materials  were  reported  to  have  saved  students  time.  However,  in  the  EESL  class, 
students  reported  spending  more  time  on  the  online  tests  but  saving  time  through  use  of  discussion  board. 


294 


303 


Table  9:  Time  Spent/Saved  Using  Online  Components 


Time  Spent 


Online  Materials 

CJUS 

EESL 

INFO 

Saved  time 

36% 

64% 

69% 

Spent  more  time 

0% 

9% 

6% 

No  answer 

64% 

27% 

13% 

Online  Tests 

Saved  time 

0% 

27% 

0% 

Spent  more  time 

0% 

55% 

0% 

No  answer 

100% 

27% 

100% 

Discussion  Board 

Saved  time 

7% 

64% 

0% 

Spent  more  time 

0% 

9% 

0% 

No  answer 

93% 

27% 

100% 

Faculty  Syllabi,  Proposals,  Grant  Reports,  and 

Post  Surveys 

The  faculty  post  survey  was  composed  of  items  concerning  their  needs  and  uses  of  online  technologies,  coupled  with  an 
evaluation  of  students'  learning  that  paralleled  the  student  post  survey.  We  also  examined  how  the  use  of  online  technologies 
impacted  faculty  workload. 

In  our  ongoing  analysis  we  are  considering  the  factors  that  led  to  successful  and  efficacious  use  of  online  technologies  for 
both  faculty  and  students.  We  are  currently  conducting  our  analysis  via  triangulation  where  we  compare  student  data,  faculty 
data  (composed  of  free  response  and  multiple  choice  questions),  information  provided  in  the  course  syllabi,  the  grant  proposals, 
and  the  end  of  grant  reports. 

We  were  not  able  to  directly  correlate  the  faculty  post  data  with  the  student  post  data  for  individual  courses  due  to  the 
anonymity  of  the  faculty  post  surveys.  However,  we  were  able  to  glean  some  data  in  the  aggregate  for  professors’  general 
satisfaction  with  the  use  of  online  technologies  in  their  classes.  Based  on  that  data  it  was  clear  that  faculty  appreciated  the 
support  that  was  given  to  them. 

Two  of  the  three  instructors  provided  access  to  syllabus  information.  Based  on  this  information,  in  the  INFO  class,  students 
were  required  to  work  in  teams  to  complete  projects.  The  professor  used  online  materials  but  did  not  list  use  of  the  discussion 
board  in  his  syllabus.  By  contrast,  in  the  EESL  class,  students  were  not  required  to  work  in  teams.  The  instructor  required  the 
use  of  the  discussion  board,  online  materials  and  online  tests. 

Table  10:  Online  components  of  Course  Design 


COURSE 

Based  on  Proposal: 

Based  on  syllabus: 

Based  on  final  report: 

DESIGN 

CJUS 

EESL615 

INFO 

CJUS 

EESL  INFO 

CJUS 

EESL 

INFO 

EESL 

Online 

All  course 

Informatio  Course 

Assignme 

Course 

Course 

Materials 

notes. 

n 

content. 

nt 

materials 

content 

exercises 

modules/ 

glossary, 

descriptio 

tutorials 

site  links 

ns 

using 

PowerPoi 

for  articles 

nt  and 
audio 

Online 

Practice 

Part  of 

Online 

100-point 

Quizzes 

Tests 

quizzes. 

tutorial 

quizzes 

quiz  set. 

assessment 

and 

review 

surveys 

quizzes 

Online 

Accessing 

Research 

TESOL 

research 

via 

technolog 

y 
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Online 

Discussion 

Email 

Chat 

Rooms 


Bulletin 

board 


Mentioned 

Mentioned 

in  the 

it  affords 

justificatio 

students  to 

n but  not 

interact 

part  of 

but  was 

project 

not 

descriptio 

reported 

n 

to  have 

been  done 

Weekly 

Used 

announce- 

extensively 

ments 

In  the  grant  proposals,  teaching  was  to  be  enhanced  with  the  posting  of  online  materials  and  links  to  relevant  websites  (see 
figure  1 0).  EESL  also  incorporated  the  use  of  online  tests  and  discussion  board.  INFO  proposed  to  implement  more  technologies 
than  were  actually  incorporated  into  the  course. 


Discussion 

Based  on  the  data  presented  above,  it  appears  clear  that  in  all  three  courses,  students  were  satisfied  with  the  connections  made 
between  the  objectives  of  the  course  and  the  assessments  used.  This  would  imply  that  the  courses  were  relatively  well  designed. 

However,  though  the  courses  were  relatively  well  received,  it  appears  that  improvements  could  have  been  made  in  the 
effective  use  of  online  technologies.  For  instance,  the  INFO  course  did  not,  according  to  the  syllabus,  the  final  report  and  the 
students’  post  surveys,  use  a discussion  board  for  collaboration  and  file  sharing.  This  was  the  case  even  though  extensive 
collaboration  was  necessary  for  successful  completion  of  course  projects  and  even  though  at  least  40%  of  the  students  lived  20  or 
more  miles  from  campus. 

Myer  (2001)  indicated  that  a fundamental  problem  with  much  research  on  online  instruction  is  that  it  has  failed  to 
disaggregate  the  effects  of  the  instructional  design  from  the  effects  attributable  to  the  technology.  This  appears  to  be  the  case  in 
the  courses  that  we  examined  above,  as  each  could  be  shown  to  have  been  efficacious  (based  on  student  post  survey  data). 

It  is  striking  that  two  of  the  three  courses  did  not  reflect  good  scores  on  online  learning  activities  fostering  interaction  and 
teamwork.  This  is  striking  as  online  technologies  such  as  threaded  discussion  boards  and  chat  rooms  can  be  very  powerful  tools 
for  collaboration  (see  Jonassen,  et  al.,  1999).  This  would  suggest  that  the  full  power  of  the  technologies  available  through  the 
WebCT  tools  to  the  instructors  had  not  been  utilized.  In  the  third  course,  the  INFO  course,  it  is  not  clear  whether  chat  rooms 
were  utilized  by  students  (see  table  10).  However,  the  structure  of  the  course  required  extensive  collaboration  on  projects 
(according  to  the  course  syllabus). 

Similarly,  for  all  three  classes,  the  mean  scores  on  facilitating  discussions  concerning  diversity  were  low.  Online 
technologies  have  the  potential  to  enable  ready  access  to  global  perspectives  (see  Papert,  1 998),  yet  it  would  appear  that  this 
characteristic  of  the  technology  was  not  utilized  effectively. 

The  fact  that  students  in  all  three  classes  generally  believed  that  using  online  technologies  saved  time  would  suggest 
improved  instructional  efficiency  for  all  classes.  With  a population  composed  of  commuters,  who  often  hold  full  time  jobs,  this 
result  is  weighty. 

Though  we  asked  students  to  rate  how  well  the  online  learning  experiences  (using  5=excellent;  4=Very  good;  3=satisfactory; 
2=poor;  l=not  applicable),  contributed  to  the  effectiveness  of  the  class  (as  measured  by  multiple  items  detailed  in  the  above 
tables  of  results),  respondents  may  not  have  read  the  items  with  respect  to  online  technologies  but  rather  with  respect  to  the  entire 
class.  For  instance,  in  the  INFO  class,  students  identified  online  components  of  the  class  to  have  contributed  to  fostering 
interaction  and  teamwork,  yet  a discussion  board  was  not  utilized  in  the  course,  according  to  the  proposal,  the  syllabus,  the  final 
report,  and  student  post  surveys.  We  are  also  concerned  that  in  some  cases,  students  may  not  have  understood  the  terms  used  and 
may  have  therefore  answered  questions  erroneously  (e.g.  some  students  rated  discussion  boards  and  chat  rooms  that  did  not 
appear  to  have  been  implemented  in  a particular  course). 

Given  our  data  analysis,  several  implications  are  suggested  for  implementation  of  the  Online  Teaching  Support  Model  at  our 
university.  In  examining  the  7 components  of  the  model,  there  is  evidence  that  several  of  the  components  may  require  substantial 
additional  support.  Specifically,  the  instructional  design  and  training  area  may  need  additional  attention  to  enable  faculty  to  best 
integrate  online  technologies  into  their  course  design.  We  are  concerned  that  our  evidence  suggests  that  instructors  may  not  use 
the  most  powerful  or  appropriate  technologies  to  assist  with  learning  outcomes  for  their  class  or  that  they  may  use  technologies  in 
ways  that  are  not  effective.  Additionally,  in  the  evaluation  area,  several  flaws  were  present.  Faculty  did  not  administer  the 
instruments  consistently  in  their  courses  (even  though  evaluation  was  a critical  component  of  the  grant).  Thus,  of  the  twelve 
funded  projects,  we  were  able  to  realize  comparison  data  from  only  three  classes  in  which  we  had  sufficient  data  from  both  the 
student  pretest  and  posttest. 
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We  surmise  that,  in  general,  there  is  a tendency  for  faculty  to  focus  on  innovation  and  on  technology  and  not  on  evaluation  of 
the  online  course  components.  In  order  to  better  collect  data  from  faculty,  and  to  encourage  them  to  have  students  participate 
fully  in  pretest  and  posttest  administration,  additional  faculty  incentives  and  reminders  are  likely  to  be  necessary.  Engaging 
faculty  in  a post-course  interview  would  also  likely  assist  analysis  of  the  efficacy  of  the  online  course  components. 

We  will  apply  the  lessons  learned  from  this  analysis  to  future  faculty  development  grant  projects.  This  will  assist  us  in 
serving  the  educational  needs  of  our  students  through  the  use  of  technologies  that  can  bridge  barriers  to  access.  We  believe  that 
the  Online  Teaching  Support  Model  can  be  effectively  applied  at  other  institutions  that  would  internally  fund  faculty 
development  efforts.  Based  on  our  experience,  special  attention  should  be  applied  to  the  instructional  design  and  evaluative 
components  of  such  efforts. 
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Abstract 

Online  collaborative  documents  can  be  used  effectively  for  conducting  collaborative  research  and  for  learning 
collaboratively  via  the  Internet.  Collaborative  documents  are  dedicated  online  workspaces  that  allow  individuals  or  groups  to 
use  the  Internet  to  share  their  work  with  others,  edit  it,  and  finalize  it.  This  paper  identifies  Basic  Support  for  Cooperative  Work 
and  FirstClass  as  Internet  tools  that  have  enabled  collaboration  across  distance  for  both  research  and  coursework,  and  it 
provides  real-life  applications  of  those  tools  in  higher  education. 

Introduction 

We  use  online  collaborative  documents  for  conducting  collaborative  research  and  for  learning  collaboratively  via  the  Internet. 
The  increased  use  of  the  Internet  in  higher  education  has  allowed  researchers  to  conduct  collaborative  research  and  faculty  to 
provide  opportunities  for  students  to  work  collaboratively  in  their  courses.  Collaborative  documents  are  dedicated  online 
workspaces  that  allow  individuals  or  groups  to  use  the  Internet  to  share  their  work  with  others,  edit  it,  and  finalize  it.  Real-time 
chats,  computer  conferences,  and  email  may  be  used  to  support  collaborative  documents  but  are  not  collaborative  documents 
according  to  our  definition  because  they  are  not  dedicated  workspaces. 

Collaborative  learning  is  an  instructional  method  in  which  small  groups  of  learners  work  together  to  accomplish  shared  goals 
(Slavin,  1994).  Similarly,  computer-supported  collaborative  work  allows  working  groups  distributed  in  time  and  space  to  use 
groupware.  Various  Internet  tools  have  enabled  collaboration  across  distance  for  both  research  and  coursework. 

Collaborative  Documents  for  Research 

Since  the  advent  of  email,  and  fax,  researchers  across  distances  have  used  telecommunications  to  write  documents  together — 
getting  feedback  from  co-researchers  and  incorporating  those  ideas  into  the  manuscript.  Eventually,  through  multiple-  iterations, 
the  document  becomes  a final  product.  The  early  Internet  researchers  had  to  think  creatively  so  that  their  co-collaborators  could 
discern  the  changes  made  to  the  document.  For  example,  in  1991  two  researchers  co-authored  a book  chapter  using  boldface  type, 
square  brackets,  and  initials  and  dates  via  email  between  their  two  universities  (Murphy  & Rogers,  1993). 

Electronic  bulletin  board  systems  and  computer  conferencing  software  such  as  Wildcat,  VAXNotes,  LotusNotes,  and  Web 
boards  allow  users  to  hold  asynchronous,  threaded  discussions  online.  Through  such  server  software,  co-researchers  can  share 
their  ideas  in  an  organized  manner  without  using  their  own  disk  space.  Most  of  the  early  software  programs  did  not  allow 
attachments.  More  recently,  we  have  used  several  Web  tools  with  shared  workspaces  for  conducting  our  research:  Basic  Support 
for  Cooperative  Work  (BSCW)  because  of  its  capabilities  for  file  attachments  and  version  control;  and  FirstClass  computer 
conference  software  because  of  its  asynchronous  and  synchronous  communication,  file  transfer,  and  collaborative  document 
functions.  See  Table  1 for  shared  characteristics  and  Table  2 for  differences  between  BSCW  and  FirstClass. 

Table  1.  Shared  Characteristics  of  BSCW  and  FirstClass 
Server-based 

Individual  and  collaborative  workspaces 
Text  -based 

Asynchronous  threaded  discussions 
File  attachments 
Links  to  Internet 
Cross-platform 
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Table  2.  Differences  between  BSCW  and  FirstClass 


BSCW 

FirstClass 

Free 

Requires  a license 

Web  accessible 

Web  and  client  accessible 

Icon  and  menu  driven 

Graphic  user  interface 

No  synchronous  chat  capability 

Synchronous  chat  capability 

Automatic  version  control 

Manual  version  control 

Steep  learning  curve 

Short  learning  curve 

Complex  organization 

Simple  organization 

Meeting  notification 

Special  forms 

BSCW  description  and  research  examples 

The  process  of  file  attachments  and  version  control  in  BSCW  is  relatively  simple.  Researcher  1 first  attaches  a file  in  a 
workspace  and  then  establishes  version  control  on  the  file.  Researcher  2 then  downloads  the  file,  uses  typical  editing  functions 
(e,g.,  colored  font,  boldface  type,  comments  in  brackets,  dates),  saves  it,  and  uploads  it  to  the  dedicated  workspace.  All  versions 
of  a document  are  maintained  in  the  workspace  without  being  over-written.  Although  the  researchers  do  not  rename  the  file  as 
they  upload  it,  the  BSCW  software  automatically  assigns  the  new  file  with  a new  version  number  and  names  the  author  of  that 
version.  When  multiple  collaborators  create  a document,  this  process  of  version  control  keeps  track  of  multiple  iterations.  The 
primary  disadvantages  of  using  BSCW  are  its  steep  learning  curve  and  a slow  rate  of  speed  in  updating  Web  pages  due  to  the 
extensive  graphics  on  the  BSCW  Web  site. 

Researchers  have  used  BSCW  for  shared  research  in  co-authoring  papers  and  manuscripts  between  two  countries  or  cities. 
For  example,  prior  to  working  on  manuscripts  when  one  would  be  in  Taiwan  and  the  other  in  the  U.S,,  the  second  and  third 
authors  met  face-to-face  to  establish  the  method  of  exchange  and  encountered  the  following  problems  only  after  one  researcher 
traveled  to  Taiwan:  (a)  figures  created  on  one  platform  did  not  always  show  up  on  another;  (b)  communication  was  interrupted 
when  an  undersea  cable  linking  the  United  States  and  Taiwan  was  damaged;  and  (c)  communication  was  delayed  when  Internet 
traffic  was  jammed  in  Taiwan,  This  Internet  traffic  worsened  when  students  were  out  of  school  during  vacations.  Fortunately,  the 
telecommunications  company  arranged  for  the  net  users  to  transmit  data  through  a backup  cable  as  well  as  a satellite-based 
network  (Cifuentes  & Shih,  2001;  Hsu,  1999;  Staff  Reporter,  The  China  Post.  2001).  Similarly,  the  first  author  met  face-to-face 
twice  with  two  co-authors  to  brainstorm  the  method  and  later  to  analyze  data,  and  they  accomplished  the  remainder  of  the 
research  using  BSCW’s  version  control  to  communicate  between  two  cities  in  Texas  (Murphy,  Mahoney,  & Harvell,  2000), 

A second  example  of  how  researchers  use  BSCW  for  shared  research  is  in  guiding  dissertation  research.  The  first  author 
designates  a shared  workspace  for  each  of  her  doctoral  students  in  her  own  workspace.  The  students  are  each  responsible  for 
posting  drafts  and  final  versions  of  their  work  in  their  workspaces  for  the  dissertation  chair  to  read,  edit,  comment,  and  guide  the 
process.  A typical  doctoral  student’s  BSCW  workspace  includes  the  following  folders:  Bibliographies,  Discussions,  Meeting 
Notes,  Prelims  and  Orals,  Proposal  Components,  Dissertation  Components,  and  Resources,  The  researcher  finds  BSCW  a 
convenient  and  powerful  tool  to  track  the  progress  of  her  doctoral  advisees  as  well  as  share  resources  among  the  students, 

FirstClass  description  and  research  examples 

Researchers  suggest  that  the  very  nature  of  computer  conferencing — its  capacity  to  support  interaction  among  students — 
fosters  a collaborative  approach  to  learning  (Harasim,  Hiltz,  Teles,  & Turoff,  1995),  FirstClass  computer  conference  software 
provides  multiple  functions  that  foster  collaborative  research  and  learning:  threaded  discussions  in  icon-based  conferences,  file 
attachments,  private  email,  real-time  text -based  chats  (Persico  & Manca,  2000),  and  collaborative  document  writing  spaces.  The 
text -based  collaborative  document  writing  spaces  allow  only  one  person  to  edit  a document  at  a time,  using  word  processing 
capabilities  like  font  types,  colors,  and  sizes.  However,  multiple  readers  can  access  these  continuous  unbroken  documents 
simultaneously.  Currently,  the  client  version  of  FirstClass  must  be  used  in  order  to  open  and  write  in  collaborative  documents,  as 
the  Web  version  does  not  provide  that  capability. 

The  first  author  and  several  local  doctoral  students  used  FirstClass  collaborative  documents  to  prepare  two  conference  papers 
(Murphy,  Harvell,  Epps  et  al,,  1999;  Murphy,  Harvell,  Sanders,  & Epps,  1999),  The  shared  workspaces  provided  the  authors  with 
“boundaries  around  a protected  space,  with  the  members  of  the  group  sharing  a common  experience”  (Palloff  & Pratt,  1 999,  p, 
61),  In  both  cases,  flie  researcher  was  careful  to  save  the  collaborative  documents  to  her  hard  drive  daily,  because  such 
documents  cannot  be  protected  and  still  allow  multiple  users  to  access  the  document. 

Collaborative  documents  for  coursework 

With  the  advent  of  the  Web  for  course  support  or  delivery,  designing  authentic  and  relevant  activities  that  foster  collaboration 
among  learners  has  become  a challenge  for  instructors  and  course  designers  (Carr-Chellman  & Duchastel,  2000;  Cifuentes, 
Murphy,  Segur,  & Kodali,  1997).  Collaborative  workspaces  and  documents  facilitate  a variety  of  learning  activities  in  Web-based 
courses  including  tutorials  (Cifuentes  & Shih,  2001;  Cifuentes  & Shih,  in  press;  Davis  & Chang,  1994/95;  Kamhi-Stein,  1997; 
Leh,  1997),  small-group  discussions,  project-based  work,  collaborative  problem-solving  activities  (Romiszowski,  1997), 
brainstorming  (Kay,  1995;  Neuhaus,  1997;  Siau,  1995,  1996),  and  case-based  learning  (Ertmer  & Quinn,  1998),  In  collaborative 
workspaces  and  documents  in  BSCW  and  FirstClass,  learners  can  identify  their  contributions  by  dating  and  signing  them  and  by 
using  different  colors.  Because  a FirstClass  collaborative  document  is  a continuous  unbroken  document  that  allows  users  to  “get 
a holistic  view  of  the  activity  without  having  to  open  and  close  numerous  messages”  (Murphy  & Gazi,  in  press),  learners  find  the 
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tool  easy  to  use  for  brainstorming  and  planning  activities  as  well  as  for  editing  each  other’s  work.  In  the  following  sections,  we 
illustrate  applications  from  our  coursework  of  each  of  the  preceding  learning  activities. 

Tutorial 

American  and  Taiwanese  university  students  corresponded  via  e-mail  and  a Web-board  to  prepare  U.S.  preservice  teachers 
for  online  teaching  and  reaching  diverse  learners,  and  to  provide  English  instruction  to  the  Taiwanese  students.  U.S.  preservice 
teachers  explored  theory  and  practice  of  online  instruction,  corresponded  as  tutors  to  teach  English  language  and  American 
culture,  and  reflected  upon  their  experiences.  Taiwanese  students  practiced  English  and  ecchanged  cultural  information. 
Throughout  the  correspondence,  the  two  parties  made  use  of  different  strategies  to  achieve  their  predefined  instructional  and 
learning  goals.  The  preservice  teachers  employed  a total  of  seven  online  teaching  strategies:  (a)  facilitative  information,  (b) 
questions  and  answers  from  tutor  to  student,  (c)  questions  and  answers  from  student  to  tutor,  (d)  topic  discussion,  (e)  problem 
solving,  (f)  critique  of  writing,  and  (g)  recommendations  for  metacognition.  Similarly,  the  Taiwanese  learners  used  ten  online 
learning  strategies:  (a)  responsive  dialogue,  (b)  translation,  (c)  responding  to  tutors’  questions,  (d)  asking  questions  of  tutors,  (e) 
explanation,  (0  elaboration,  (g)  decision-making,  (h)  self-reflection,  (i)  metacognitive  strategies,  and  (j)  transfer  (Cifuentes  & 
Shih,  2001;  Cifuentes  & Shih,  in  press). 

SmalUgroup  discussions 

All  online  classes  in  our  educational  technology  program,  and  most  classes  that  are  not  online,  include  online  discussion 
spaces.  Students  discuss  course  readings  and  learn  how  to  moderate  discussions  on  specific  topics. 

Project-based  work 

In  an  introductory  instructional  design  course,  students  used  FirstClass  to  conduct  project-based  work  in  small  groups. 
Students  work  with  a client  and  often  with  an  online  team  to  carry  out  the  phases  of  instructional  design.  For  instance,  two 
students  living  in  cities  separated  by  hundreds  of  miles  designed  and  developed  teacher  training  on  thematic-unit  design  for  a 
school  district.  They  shared  design  and  development  processes  using  a dedicated  space  in  FirstClass.  They  established  goals; 
designed  instruments  for  analyses  of  the  learners,  context  and  goals;  wrote  objectives  and  developed  assessment  instruments, 
shared  evaluation  data,  and  assigned  development  tasks  to  each  other  in  FirstClass. 

Students  in  introductory  distance  learning  courses  at  two  universities  used  the  FirstClass  server  at  one  of  the  universities  to 
design,  develop,  and  evaluate  telecommunications-based  case  studies  in  small  teams.  Most  teams  scheduled  regular  online  chats 
with  their  team  members  as  well  as  working  asynchronously  in  threaded  discussions  to  collaborate  on  the  project  Each  group 
developed  its  own  evaluation  criteria,  which  the  other  groups  used  to  evaluate  its  products. 

In  other  courses  students  collaboratively  produced  instructional  thematic  units,  Web  pages,  and  computer  graphics  using 
FirstClass.  In  a class  that  focused  on  design  and  development  of  student -centered,  technology  rich  thematic  units  (Roberts  & 
Kellough,  2000),  students  teamed  to  develop  those  units.  They  brainstormed  to  agree  on  a theme  for  their  team’s  unit  and 
submitted  contributions  to  the  unit  in  the  shared  workspace.  Contributions  included  readings  of  interest,  PowerPoint 
presentations,  WebQuests,  ideas  for  telecommunications  partnerships,  and  lesson  plans. 

Students  also  collaboratively  created  computer  graphics  by  submitting  those  graphics  to  a team  in  a dedicated  space  in 
FirstClass.  Team  members  could  alter  the  PowerPoint  presentations,  PhotoShop  files,  and  Director  files  and  resubmit  them  with 
their  embedded  feedback  and  changes.  In  this  way  students  were  able  to  receive  concrete  feedback  from  both  the  instructor  and 
fellow  students  in  order  to  learn  principles  of  effective  design. 

Collaborative  problem-solving  activities 

For  a survey  course  in  educational  technology,  the  instructor  posed  a simulated  problem  for  students  to  address  in  FirstClass 
collaborative  documents  and  threaded  discussions.  The  instructor  invented  Mythica,  a mythical  oil-producing  country  comprised 
of  18  islands  and  inhabited  by  people  accustomed  to  learning  by  rote  memorization.  The  students'  challenges  were  to  answer 
questions  related  to  the  simulation  and  to  submit  a bid  on  a project  for  teaching  English  to  the  entire  Mythica  population.  Three 
co-facilitators  presented  the  simulation  to  two  teams  of  their  classmates,  who  responded  to  the  questions  in  their  collaborative 
workspaces.  The  teams  competed  for  the  winning  bid  by  helping  their  teammates  with  refinements  of  their  rep  lies  (Murphy, 
Moran,  & Weems,  2000). 

In  a course  about  distance  learning,  students  used  FirstClass  to  complete  training  modules  on  Internet  tools  in  dyads:  a trainer 
and  a trainee.  The  trainer  first  developed  a training  program  for  a specific  tool,  such  as  CU-SeeMe,  and  then  delivered  the 
training  to  the  trainee  via  FirstClass  as  a pilot  test.  The  trainee  was  responsible  for  giving  feedback  to  the  trainer  regarding 
accuracy  and  clarity.  After  the  trainer  made  the  necessary  revisions  and  the  trainee  approved  them,  the  student  Webmaster 
published  the  training  programs  on  the  Web  for  other  students  to  access  and  use.  In  the  final  step  of  this  process,  the  trainer  and 
trainee  individually  recorded  their  reflections  about  the  experience  (Murphy,  Flarvell,  Sanders,  & Epps,  1999).  In  most  cases,  the 
more  experienced  Internet  user  was  the  trainer;  however,  some  dyads  reversed  their  roles  so  that  the  inexperienced  user  would 
gain  more  expertise. 
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Brainstorming 

Students  in  a Computer  Graphics  for  Learning  course  used  a dedicated  space  in  FirstClass  to  brainstorm  ideas  prior  to 
creating  instructional  animations.  The  assignment  was  for  each  student  to  develop  an  animation  that  would  help  learners 
understand  a complex  concept.  In  previous  semesters  several  students  had  submitted  animations  that  were  primarily  cosmetic 
rather  than  instructive.  In  an  attempt  to  hinder  such  submissions,  the  instructor  set  up  a brainstorming  conference  in  FirstClass.  In 
that  conference,  students  were  requested  to  enter  their  ideas  for  instmctive  animations  and  they  were  allowed  to  give  each  other 
feedback  regarding  the  value  and  attributes  of  such  an  animation.  One  student’s  idea  led  to  many  students’  ideas,  and  ultimately 
all  students  formulated  good  ideas  for  meaningful  animations. 

Case-based  learning 

In  an  introductory  instructional  design  course,  students  analyzed  cases  following  recommendations  for  case  analysis  (Ertmer 
& Quinn,  1998).  They  negotiated  meaning  regarding  the  nature  of  the  problems  posed  and  potential  solutions  using  FirstClass. 
Students  discussed  24  ID  cases  according  to  key  issues  of  the  cases,  key  players’  perspectives,  potential  solutions  related  to 
problems,  and  recommendations  for  action. 

Conclusions 

Garrison’s  (2000)  review  of  distance  education  theories  describes  the  current  need  for  “sustained  real  two-way 
communication”  (p.  13)  to  be  at  the  core  of  the  educational  experience.  Until  recently,  the  field  was  dominated  by  organizational 
(structural)  assumptions.  However,  with  the  advent  of  new  methods  and  technologies,  the  field  will  “demand  theories  that  reflect 
a collaborative  approach  to  distance  education... and  have  at  their  core  an  adaptive  teaching  and  learning  transaction”  (p.  13).  One 
way  to  apply  such  theories  to  research  and  coursework  is  to  use  collaborative  documents  to  promote  transaction  culminating  in 
deliberation  among  the  key  players.  Such  deliberation  can  create  “a  particular  kind  of  democratic  public  culture  among  the 
deliberators:  listening  as  well  as  talking,  sharing  resources,  forging  decisions  together  rather  than  only  advocating  positions  taken 
earlier,  and  coming  to  disagreement”  (Parker,  Ninomiya,  & Cogan,  1999,  p.  129). 
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Abstract 

The  benefits  of  collaboration  and  team  skills  have  long  been  recognized  in  both  business  and  academic  settings.  In  a study 
involving  a team  of  business  professionals  performed  in  1967,  Maier  concluded  that  teams  have  a distinct  advantage  over 
individuals  in  that  they  have  a greater  pool  of  knowledge,  receive  more  input  and  solutions  to  problems,  and  encourage  a better 
collective  acceptance  and  understanding  of  group  decisions.  A more  recent  study  of  student  teams  concluded  that  team  projects 
develop  individual  collaborative  skills,  strengthen  both  individual  and  group  commitment  to  teamwork  and  prepare  students  for 
the  2 century  workplace  (Rooney,  2000). 

In  online  collaborative  learning  situations,  however,  research  suggests  that  team  assignments  can  often  frustrate  and  annoy 
students  (Bowen,  1998).  Teams  must  deal  with  new  expectations,  attitudes,  boundaries,  and  responsibilities  which  can  all  cause 
clashes  (Joinson,  1999).  Further,  since  the  tendency  of  online  learners  leans  toward  self-direction,  students  have  a natural 
resistance  to  team  projects  where  outcomes  rely  upon  the  input  of  others  (Ko  & Rossen,  2001). 

Two  investigators  developed  a web-based  teambuilding  instructional  and  activities  module  to  improve  the  collaboration  skills  of 
team  members  involved  in  online  collaboration.  This  module  was  formatively  evaluated  using  24  graduate  students  who  were 
enrolled  in  an  online  course  that  assigned  two  online  team  projects  during  the  Fall  2000  semester.  The  objectives  of  the 
investigation  were:  1)  to  explore  whether  or  not  online  learners  valued  the  VTB  instruction  they  received  as  preparation  for 
online  team  projects,  2)  to  explore  how  online  learners  who  received  specific  instruction  on  online  teambuilding  perceived  their 
actual  collaborative  experiences  as  members  of  a virtual  team,  3)  to  determine  the  online  collaboration  skills  online  learners 
gained  after  they  had  completed  the  module,  and  4)  to  determine  whether  learners  believed  that  they  would  use  the  online 
teamwork  skills  and  knowledge  they  gained  from  the  instructional  module  in  future  online  collaborative  teams. 

Both  quantitative  and  qualitative  data  indicated  that  students  were  satisfied  with  the  teambuilding  instruction  and  their 
virtual  team  experiences,  gained  online  collaboration  skills  and  indicated  they  would  apply  these  skills  to  future  online 
collaborative  projects.  These  findings  imply  that  educators  who  teach  online  graduate  courses  and  assign  online  team  projects 
should  consider  integrating  online  teambuilding  instruction  into  their  coursework  and  study  the  value  of  this  instructional 
approach  for  their  students. 

statement  of  the  Problem 

With  the  advancement  of  technology,  online  instruction  that  often  involves  team  projects  s becoming  more  prevalent 
(DeNigris  & Witchel,  2000).  As  this  trend  is  still  fairly  recent,  there  are  major  discrepancies  between  the  available  resources  to 
increase  the  effectiveness  of  instructors  who  teach  online  and  the  resources  available  to  instructors  who  teach  face-to-face.  Kara 
and  Kling  (2000)  found  that  the  quality,  quantity,  and  accessibility  of  materials  available  to  online  teachers  are  inadequate. 

Although  there  has  been  an  increase  in  the  frequency  of  educational  teams,  it  has  not  been  matched  with  adequate  teacher 
training  in  collaborative  group  processes  to  promote  team  potential,  team  productivity,  and  team  maintenance  (Trimble  & Irvin, 
1996).  Moreover,  the  members  of  teams  could  also  use  training  in  areas  like  team  problem-solving,  conflict  management,  and 
meeting  management  (Joinson,  1999;  Mazany  & Francis,  1995). 

The  title  of  this  paper.  Building  Team  Collaboration  in  the  Virtual  Classroom^  uses  two  terms,  team  and  collaboration y which 
may  appear  at  first  glance  to  be  redundant.  People  do  not  become  collaborators,  however,  merely  because  they  are  grouped  into 
teams.  Just  like  any  other  ability,  effective  collaboration  involves  a set  of  skills  that  needs  to  be  learned  and  cultivated.  This 
becomes  especially  important  in  a virtual  realm  where  team  members  may  not  be  able  to  meet  face-to-face. 

The  benefits  of  collaboration  and  team  skills  have  long  been  recognized  in  both  business  and  academic  settings.  In  a study 
involving  a team  of  business  professionals  performed  in  1967,  Maier  concluded  that  teams  have  a distinct  advantage  over 
individuals  in  that  they  have  a greater  pool  of  knowledge,  receive  more  input  and  solutions  to  problems,  and  encourage  a better 
collective  acceptance  and  understanding  of  group  decisions.  A more  recent  study  of  Sudent  teams  concluded  that  team  projects 
develop  individual  collaborative  skills,  strengthen  both  individual  and  group  commitment  to  teamwork  and  prepare  students  for 
the  2 P*  century  workplace  (Rooney,  2000), 

In  online  collaborative  learning  situations,  however,  research  suggests  that  team  assignments  can  often  frustrate  and  annoy 
students  (Bowen,  1998).  Teams  must  deal  with  new  expectations,  attitudes,  boundaries,  and  responsibilities  which  can  all  cause 
clashes  (Joinson,  1999),  Further,  since  the  tendency  of  online  learners  leans  toward  self-direction,  students  have  a natural 
resistance  to  team  projects  where  outcomes  rely  upon  the  input  of  others  (Ko  & Rossen,  2001).  How,  then,  can  online 
collaborative  teamwork  be  optimized  so  that  online  team  experiences  will  be  satisfying  and  successful? 
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Teambuilding  Training 

Barker  and  Franzak  (1997)  believe  that  it  is  “prudent”  for  educators  to  provide  needed  experience  and  information  to  prepare 
students  to  be  better  team  members.  For  online  teams,  this  training  should  also  include  technical  support  and  technology  training 
on  the  hardware  and  software  used  to  support  online  teamwork  (Duarte  & Snyder,  1999). 

Team  training  and  development  should  also  be  ongoing  to  include  teambuilding  interventions  that  are  matched  to  the  specific 
needs  of  the  team.  For  example,  an  intervention  to  improve  problem-solving  skills  may  have  little  impact  on  a team  experiencing 
difficulties  developing  workflow  processes.  Similarly,  McClure  and  Werther  (1993)  determined  that  specific  team  interventions 
that  target  personality  characteristics  assisted  teams  in  identifying  and  resolving  personality-based  barriers  by  improving 
interpersonal  communication  and  teambuilding. 

Team  lifecycles  should  also  be  carefully  considered  before  implementing  training.  Teams  exist  on  a continuum:  from  short- 
term project  teams  to  relatively  permanent  teams  (Tannenbaum,  Beard,  & Salas,  1991).  Druskat  & Kayes  (2000)  propose  that 
many  popular  teambuilding  models  are  inappropriate  for  short-term  project  teams.  For  example,  the  conflict  depicted  as 
inevitable  during  the  “storming”  stage  of  Tucbnan’s  (1965)  team-development  model  (i.e.,  forming,  storming,  norming, 
performing)  might  confuse  or  obstruct  short-term  project  teams  whose  lifecycle  is  too  brief  to  benefit  from  this  information 
(Porter  & Lilly,  1996). 

Finally,  in  educational  settings,  teambuilding  interventions  that  encourage  establishing  clear  responsibilities,  procedures,  and 
due  dates  for  team  members  may  have  some  positive  impact  on  team  performance  but  may  also  reduce  exploration  or  risk-taking 
among  team  members  that  they  might  otherwise  use  to  seek  answers  and  make  decisions,  which  could  compromise  team  learning 
(Druskat  & Kayes,  2000), 

Elements  Found  in  Successful  Teams 

Successful  teams  share  certain  characteristics  or  traits.  For  the  purposes  of  this  investigation,  these  characteristics  or  traits 
were  labeled  “elements  found  in  successful  teams.”  These  elements  included  receiving  organizational  or  instructor  support, 
becoming  acquainted  with  team  members,  establishing  effective  communication,  building  trust,  and  developing  effective  online 
organization  strategies  (DeNigris  & Witchel,  2000;  Duarte  & Snyder,  1999). 

Receiving  Support 

The  literature  often  cites  organizational  and  management  support  as  the  major  factor  in  team  success  (Berry,  Avergun,  & 
Russ-Eft,  1993;  Haywood,  1998).  In  educational  teams,  a lack  of  support  from  the  instructor  causes  anxiety  and  frustration 
among  students  and  adversely  affects  team  effectiveness  (Kara  & Kling,  2000).  Oliver,  Omari,  and  Herrington  (1998) 
recommended  that  online  instructors  should  scaffold  their  support  to  students.  They  describe  scaffolding  as  providing  increased 
support  at  critical  times,  such  as  at  the  beginning  of  instruction  when  students  must  learn  new  information  or  skills,  and  tapering 
off  as  students  become  more  experienced. 

The  element,  “support,”  encompasses  providing  encouragement,  information,  and  resources;  responding  to  team  requests 
promptly;  helping  to  provide  team  direction;  acting  as  an  arbitrator;  and  backing  the  decisions  of  teams  (Duarte  & Snyder,  1999; 
Eales-White,  1997;  Haywood,  1998).  In  knowing  what  constitutes  receiving  support,  team  members  can  play  a more  active  role 
in  obtaining  it. 

Getting  Acquainted 

Berliner  (1991)  wrote,  “education — even  when  carried  out  with  personal  computers — is  an  inherently  social  process”  (p,  50), 
Although  building  relationships  in  online  environments  is  difficult,  it  is  essential  to  team  effectiveness  (DeNigris  & Witchel, 
2000;  Haywood,  1998).  Research  has  indicated  a positive  link  between  team  member  relations  and  team  performance  in  both 
short-term  teams  (Druskat  & Kayes,  2000)  and  in  long-term  teams  (Druskat,  1996;  Goodman  & Leyden,  1991).  The  familiarity 
that  individual  members  develop  within  their  teams  helps  them  to  predict  each  other’s  behaviors  and  match  their  strengths  and 
interests  to  tasks  (Cannon-Bowers,  Salas,  & Volpe,  1995). 

The  element,  “getting  acquainted”  encompasses  sharing  cultural  information  which  includes  sharing  beliefs,  values, 
assumptions,  and  opinions;  personal  information  that  includes  sharing  interests,  hobbies,  work  life,  family  life,  personal  web  site, 
hours  of  availability,  personal  expectations  of  the  team,  types  of  computer  connections,  equipment,  and  skills  (Duarte  & Snyder, 
1999;  Haywood,  1998). 

Establishing  Communication 

Much  of  human  communication  is  inherently  ambiguous.  In  face-to-face  situations,  however,  people  are  more  apt  to  resolve 
these  ambiguities.  As  Hara  and  Kling  (2000)  found  in  a study  of  graduate  students  who  were  taking  an  online  course,  resolving 
communication  ambiguities  can  be  much  more  difficult  in  synchronous  and  asynchronous  online  situations  where  the  primary 
means  of  communication  is  written  text.  In  a separate  study  of  graduate  students  enrolled  in  an  online  educational  technology 
course,  Talley  (1997)  found  two  underlying  problems  specific  to  online  communication  and  distance  education:  1)  students  who 
easily  communicate  face-to-face  found  online  contact  more  difficult  due  to  limited  typing  skills,  and  2)  synchronous  discussions 
required  a speed  of  response  and  attentiveness  that  was  demanding,  while  students  generally  prefer  time  to  reflect  on  ideas  before 
responding  to  them. 

Establishing  clear  communication  is  fundamental  to  all  aspects  of  online  teams.  For  example,  teams  must  communicate 
effectively  in  order  to  establish  clear  and  specific  goals  and  objectives  so  that  they  may  function  effectively  as  a team  (Larson 
and  LaFasto,  1989).  Yukl  (1994)  maintains  that  teams  that  do  not  clearly  communicate  their  goals  will  be  fraught  with 
disagreement  about  priorities  and  processes  for  accomplishing  objectives. 
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Three  ways  to  foster  effective  online  communication  include:  1)  limiting  interference  which  entails  being  prepared  for  online 
team  meetings,  being  timely,  using  technology  effectively,  and  asking  questions  when  messages  are  unclear  (Duarte  & Snyder, 
1999;  Haywood,  1998);  2)  encouraging  open  communication  which  entails  being  informal,  incorporating  humor,  being  honest 
and  thoughtful,  and  providing  motivational  and  positive  messages  to  teammates  (Duarte  & Snyder,  1999;  Hara  & Kling,  2000); 
and  3)  building  rapport  among  teammates  which  entails  occasionally  engaging  in  team  activities  or  discussions  that  are  not  work- 
related  (Barker  & Franzak,  1997,  Oliver,  Omari,  & Herrington,  1998). 

Building  Trust 

Geber  (1995)  determined  that  face-to-face  contact  was  necessary  to  establish  trust  within  teams.  In  a study  of  virtual 
organizations.  Handy  (1995)  corroborated  Geber’s  conclusion  when  he  proposed  that  trust  may  not  be  possible  in  global  virtual 
teams  because  it  requires  “touch,”  that  is,  direct  face-to-face  exchanges.  Conversely,  Jarvenpaa,  Knoll,  and  Leidner  (1998)  found 
a positive  relationship  between  the  levels  of  trust  and  the  amount  of  cohesiveness,  satisfaction,  and  perceived  effectiveness 
among  online  team  members.  According  to  Jarvenpaa  and  Leidner  (1998),  although  the  usual  cues  used  to  convey  a sense  of 
trust,  such  as  warmth,  attentiveness,  and  other  expressive  behaviors,  are  somewhat  lacking  in  the  virtual  realm,  experienced 
online  team  members  display  trust  behaviors  in  other  ways.  That  particular  trust,  however,  is  delicate  and  provisional. 

While  developing  trust  in  a virtual  environment  requires  a more  conscious  and  planned  effort  (Duarte  & Snyder,  1999),  once 
it  is  developed,  trust  enhances  group  learning  and  development  (Braaten,  1974;  Mann,  1975)  and  it  allows  teams  to  manage 
conflict  more  effectively  to  become  more  productive  and  creative  (Dee,  1995).  Conflict  is  a reality  of  team  experience  and  the 
willingness  and  ability  to  resolve  personal  conflict  is  crucial  to  team  success  (Hequet,  1994;  McClure  & Werther,  1993). 
Although  several  researchers  recommended  the  golden  rule,  “treat  others  as  you  would  like  to  be  treated”  for  resolving  conflict 
and  building  trust  (Dee,  1995;  DuBrin,  1995;  Gardenswartz  & Rowe,  1994).  Kezar  (1998)  and  Manz,  Neck,  Mancuso,  and  Manz 
(1997)  suggested  a somewhat  modified  golden  rule  that  embraced  the  diversity  found  in  online  teams.  They  urge  team  members 
to  discover  how  other  people  on  their  team  want  to  be  treated  and  then  act  accordingly. 

In  online  teams,  the  element,  “building  trust”  encompasses  acting  with  integrity  toward  teammates,  respecting  others, 
committing  to  the  team  effort,  resolving  conflict  constructively,  being  reliable,  and  being  honest  (Duarte  & Snyder,  1999; 
Haywood,  1 998;  lacono  & Weisband,  1 997). 

Getting  Organized 

The  research  of  Langer  (1997)  and  Druskat  & Kayes  (2000)  revealed  an  inverse  relationship  between  clearly  defined  project 
goals  and  learning.  Specifically,  a decrease  in  structure  and  specifics  of  a project  results  in  an  increase  in  learning  and  vice  versa. 
Langer  (1997)  describes  this  phenomenon  as  the  need  for  “mindfulness”  or  thought  and  attention  to  changing  ideas  and 
circumstances,  which,  she  argues,  increases  learning.  Conversely,  other  studies  involving  project  teams  revealed  that  developing 
clear  plans,  goals,  and  priorities  is  positively  associated  with  team  efficiency  (Ancona  & Caldwell,  1992;  Ko  & Rossen,  2001) 
and  performance  (Porter  & Lilly,  1996). 

Duarte  and  Snyder  (1999)  described  a virtual  environment  as  inherently  chaotic  and  advocate  the  use  of  clear  procedures  and 
organization  guidelines  within  online  teams.  In  their  book  on  virtual  teams,  they  outline  several  strategies,  tools,  and  techniques 
to  help  online  teams  be  more  organized  and,  they  argue,  be  more  satisfied  and  successful.  For  example,  they  use  checklists  and 
worksheets  to  highlight  critical  success  factors  found  in  effective  teams;  they  provide  sample  agendas  to  help  confirm  team 
missions;  and  they  use  scenarios  and  exercises  to  encourage  spontaneous  and  reflective  thinking  to  develop  problem-solving 
skills. 

In  online  teams,  the  element,  “getting  organized”  encompasses  selecting  a team  leader,  recognizing  and  rewarding  team 
accomplishments,  facilitating  team  meetings,  developing  team  norms,  instituting  workflow  procedures,  creating  time  lines,  and 
selecting  the  appropriate  technology  and  method  for  team  interactions  (Duarte  & Snyder,  1999;  Haywood,  1998). 


Conclusion 

Research  that  emphasizes  the  value  to  learners  of  online  instruction  that  includes  collaboration  (Druskat  and  Kayes,  2000) 
coupled  with  findings  concerning  the  discrepancies  and  poor  quality  of  online  resources  available  to  instructors  (Hara  and  Kling, 
2000),  underscore  the  need  for  developing  more  resources  on  online  teambuilding. 

The  literature  review  also  calls  attention  to  the  need  for  further  research  of  virtual  teams.  Hara  and  Kling  (2000)  encouraged 
further  research  on  the  impact  of  support  on  students  enrolled  in  online  classes.  They  point  to  problems  and  disappointments  of 
students  who  lack  the  support  they  need  to  be  successful  in  online  courses.  Based  upon  this  collection  of  research,  guidance  from 
instructors,  content  experts,  and  peers,  a web-based  instructional  module  on  teambuilding  was  developed  to  aid  both  students  and 
instructors  who  are  involved  in  online  courses  which  have  team  projects. 

Methodology 

During  the  Summer  2000,  two  investigators  collaborated  to  develop  a web-based  teambuilding  instructional  module  (module) 
to  be  used  as  part  of  an  instructional  design  study.  The  investigators  developed  instruments  to  measure  various  aspects  of  learners 
before  and  after  the  learners  completed  the  module.  The  purpose  of  the  module  was  to  improve  online  collaboration  skills  and 
attitudes  of  people  who  produced  online  projects  as  teams.  The  investigators  worked  together  to  implement  the  module  and  to 
gather  data  focusing  on  teambuilding  skill  levels  of  online  learners  and  on  general  attitudes  toward  teambuilding  instruction  and 
the  collaboration  process. 
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Online  Teambuilding  Instructional  Module 

The  online  teambuilding  instructional  module  consisted  of  an  online  PowerPoint  presentation  and  an  activity  worksheet.  It 
provided  practical  information  and  practice  with  elements  found  in  successful  online  teamwork.  These  elements  included 
information  on  receiving  organizational  or  instructor  support,  becoming  acquainted  with  team  members,  establishing  effective 
communication,  building  trust,  and  developing  effective  online  organization  strategies.  The  instructional  module  created  for  this 
study  followed  a modified  version  of  Dick  and  Carey’s  (1996)  Instructional  System  Design  Model. 

The  module  was  web-based  and  was  accessible  to  learners  via  a virtual  teambuilding  (VTB)  web  site  designed  by  the 
investigators.  Learners  who  did  not  have  high-speed  Internet  access  were  provided  with  a duplicate  version  of  the  teambuilding 
PowerPoint  presentation  that  was  downloadable  to  their  computer  desktop. 

The  web  site  housing  the  module  also  included  a Pre-Course  Survey  and  Post-Teamwork  Survey.  These  surveys  measured 
learner  satisfaction  with  his  or  her  team  projects,  and  the  skills  they  gained  for  online  collaboration  following  the  instruction  they 
would  receive  from  the  module.  Data  on  prior  experience  with  relevant  online  collaborative  technologies  was  also  gathered  using 
these  surveys  and  was  to  be  used  to  group  learners  of  the  study  into  teams  for  purposes  of  collaborating  on  team  projects. 
Objectives 

There  were  four  objectives  for  this  study.  The  first  objective  was  to  explore  whether  or  not  online  learners  valued  the  VTB 
instruction  they  received  as  preparation  for  online  team  projects.  This  objective  was  measured  in  terms  of:  1)  online  learner 
satisfaction  with  the  two  components  that  comprised  the  instruction:  an  online  PowerPoint  presentation  on  virtual  teambuilding 
and  a teambuilding  activity  worksheet,  and  2)  online  learner  perceptions  of  instruction  as  contributing  to  their  individual  personal 
growth  as  learners  and  educators. 

The  second  objective  was  to  explore  how  online  learners  who  received  specific  instruction  on  online  teambuilding  perceived 
their  actual  collaborative  experiences  as  members  of  a virtual  team.  This  objective  was  measured  in  terms  of  online  learner 
satisfaction  with  individual  team  experiences  encompassing  elements  found  in  successful  online  teams:  receiving  support,  getting 
acquainted,  building  trust,  building  communication,  and  getting  organized. 

The  third  objective  was  to  determine  the  online  collaboration  skills  online  learners  gained  after  they  had  completed  the 
module.  This  objective  was  measured  by  comparing  results  from  individual  pre-course  surveys  with  post-teamwork  surveys  that 
contained  specific  questions  about  knowledge  and  skill  levels  when  working  in  online  collaborative  teams. 

The  fourth  objective  was  to  determine  whether  learners  believed  that  they  would  use  the  online  teamwork  skills  and 
knowledge  they  gained  from  the  instructional  module  in  future  online  collaborative  teams.  This  objective  was  measured  by  the 
responses  students  gave  concerning  their  attitudes  toward  using  the  online  team  skills  they  developed  in  future  online 
collaborative  situations. 

Target  Audience 

The  target  audience  for  the  teambuilding  instructional  module  is  adult  learners  who  are  engaged  in  online,  collaborative 
projects  as  part  of  a virtual  team.  The  ultimate  intended  purpose  of  the  module  is  to  be  available  to  any  person  who  has  Internet 
access  at  anytime  and  at  any  location.  Additionally,  while  it  was  written  in  English,  the  overall  theme,  the  language,  and  graphics 
used  throughout  the  module  were  specifically  designed  to  be  inclusive  of  other  cultures. 

Participants 

This  study  addressed  itself  to  a sample  drawn  from  a group  of  graduate  students  who  were  enrolled  in  an  online  course.  This 
course  was  being  offered  at  the  University  of  Hawaii,  Manoa  (UHM)  for  the  Fall  2000  Semester.  The  instructor  for  this  course 
had  extensive  experience  with  the  content  of  the  course  and  with  distance  learning.  Course  content,  schedules,  and  feedback  were 
given  via  an  online  university  network  called  Web  Course  Tools  (WebCT).  This  particular  class  was  selected  because  it 
represented  the  target  population  in  that  it  contained  a mix  of  male  and  female  adults  from  various  ethnicities  who  were  involved 
in  online  collaborative  projects. 

There  were  a total  of  27  students  enrolled  in  the  online  course — 10  male  and  17  female.  The  cultural  backgrounds  of  the 
students  were  Chinese,  Swedish,  Hispanic,  Japanese,  Filipino,  and  Caucasian  American.  This  module  was  administered  to  24  of 
the  27  enrolled  students.  Two  students  opted  not  to  participate  and  one  student  was  disqualified  because  she  was  also  one  of  the 
investigators.  Although  only  the  results  of  24  of  the  27  students  were  considered  as  part  of  the  study,  the  whole  class  received  the 
same  treatment. 

Procedures 

This  module  was  administered  to  all  students  enrolled  in  the  Fall  2000  online  class.  Before  the  class  began,  the  instructor  and 
an  investigator,  in  the  role  of  Teaching  Assistant,  sent  out  a surface  mail  letter  and  an  email  to  prospective  students  to  provide 
them  with  general  course  information  as  well  as  to  describe  the  VTB  web  site  and  how  to  access  it. 

In  order  to  incorporate  the  online  teambuilding  module,  the  instructor  designed  an  introductory  unit.  This  unit  contained  the 
first  graded  assignments  for  the  course.  Assigning  grades  to  the  Teamwork  Unit  and  online  team  projects  provided  incentives  to 
the  learners. 

For  the  first  teambuilding  session,  students  were  to  meet  in  an  online  chatroom  in  WebCT  with  their  instructor  and  the  course 
Teaching  Assistant.  It  was  mandatory  for  every  student  to  be  present,  but  they  had  the  option  of  meeting  in  a morning  or  evening 
session.  There  were  no  other  mandatory  online  meetings  scheduled. 

Access  to  the  course  site,  which  contained  a link  to  the  VTB  web  site,  was  permitted  a few  days  before  class  actually  began. 
The  VTB  site  outlined  six  steps  that  students  would  need  to  follow  in  order  to  complete  the  teambuilding  process. 
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Step  1 was  to  fill  out  an  electronic  Pre-Course  Survey  form.  A link  was  provided  to  the  survey.  The  survey  contained  items 
concerning  demographics,  types  of  computer  and  Internet  connections,  student  online  experience  levels,  and  experience  levels  of 
students  with  virtual  teams. 

Although  students  were  required  to  participate  in  the  Pre-Course  Survey,  several  items  on  the  survey  were  optional.  At  the 
bottom  of  the  survey  form,  students  were  notified  that  information  from  their  surveys  was  requested  to  be  used  in  the 
teambuilding  study.  Besides  investigative  purposes,  the  information  from  the  surveys  was  also  used  to  establish  heterogeneous 
teams.  For  example,  to  provide  students  with  a greater  chance  of  success  when  placed  on  teams,  the  investigators  used  the  Pre- 
Course  Survey  data  to  optimize  the  placement  of  students  onto  teams  in  the  following  ways:  1)  to  equip  the  online  teams  with  the 
optimum  number  of  three  people  (Stadtlander,  1 998),  team  assignments  were  planned  that  way;  2)  to  facilitate  transferring  data 
among  team  members,  students  were  matched  according  to  computer  types,  that  is,  PC  or  Macintosh;  3)  to  provide  a range  of 
experience  levels  and  genders,  student  teams  were  deliberately  mixed. 

After  completing  the  Pre-Course  Survey,  students  were  instructed  to  view  the  twenty -minute  PowerPoint  presentation 
entitled,  “Recipes  for  Satisfying  and  Successful  Virtual  Teams,”  that  was  developed  by  the  investigators.  It  was  listed  as  Step  2 
of  the  teambuilding  process.  They  could  view  the  presentation  online  or  download  it  to  view  it  from  their  computer  desktops.  At 
the  end  of  the  presentation,  students  were  given  an  opportunity  to  print  a copy  of  the  presentation  for  future  use. 

In  keeping  with  the  cooking  theme  of  the  presentation,  the  elements  of  satisfying  and  successful  teams  were  broken  down 
into  “ingredients.”  Haywood  (1998),  and  Berry,  Avergun,  and  Russ-Eft  (1993)  established  “support”  as  the  most  essential 
ingredient  for  satisfying  team  experiences,  so  a checklist  was  developed  for  the  instructor  to  go  through  with  the  class  in  a virtual 
chatroom  describing  how  she  would  support  the  teams.  This  online  class  chat  was  labeled  Step  3 of  the  teambuilding  process. 
The  checklist  provided  a general  topic  outline  for  the  hstructor  to  follow  as  she  talked  with  the  students.  The  instructor 
committed  to  support  the  teams  by  being  available  to  act  as  an  arbitrator,  providing  key  information,  and  helping  to  guide  teams 
toward  the  project  goals.  She  also  pledged  to  be  available  when  called  upon  for  assistance,  but  not  to  exert  undue  influence  on  the 
creativity  or  bonding  of  team  members.  At  the  end  of  the  chat  session,  students  were  provided  with  a list  of  their  teammates  for 
the  two  assignments  requiring  teams. 

Step  4 of  the  teambuilding  process  involved  students  making  arrangements  with  their  teammates  to  meet  in  a virtual 
chatroom  to  engage  in  teambuilding  activities  together  using  the  worksheet  as  a guide.  They  were  encouraged  to  do  this  as  soon 
as  possible  because  they  only  had  one  week  to  complete  the  activities.  The  activities  were  provided  to  the  students  as  a printable 
activity  worksheet  available  from  the  VTB  web  site.  Students  were  directed  to  read  through  the  activities  before  meeting  with 
their  teammates  so  that  they  would  become  familiar  with  what  each  of  the  activities  involved  and  have  some  time  to  reflect  on  the 
questions.  It  was  anticipated  that  these  directions  would  facilitate  the  first  online  team  meeting  and  make  it  as  productive  as 
possible. 

Step  5 of  the  teambuilding  process  involved  students  posting  individual  results  of  their  online  team  meeting  including 
reflections,  comments,  and  questions  they  had  regarding  the  teambuilding  activities  and  the  teambuilding  process  in  general  in  an 
online  Teambuilding  Forum, 

Once  the  two  required  collaborative  projects  were  completed,  students  were  directed  to  go  to  the  VTB  web  site  to  take  a Post- 
Teamwork  Survey.  This  was  labeled  Step  6 of  the  teambuilding  process.  The  survey  was  used  to  explore  student  perceptions  of 
their  team  experiences,  the  role  of  the  module  in  their  perceptions  of  their  team  experiences,  the  role  of  the  module  in  increasing 
their  online  collaboration  skills,  the  likelihood  that  they  would  apply  the  skills  they  learned  in  future  online  teams.  Finally,  at  the 
end  of  the  Post-Teamwork  Survey,  a comment  section  was  provided.  Students  were  urged  to  provide  feedback  about  the 
teambuilding  instruction  and  activities,  the  personal  experiences  they  had  in  the  assigned  online  teams,  and  their  satisfaction  with 
their  team  experiences. 

Instruments 

Three  research  instruments  were  used  to  gather  demographic  data  and  to  measure  student  satisfaction  levels.  These  included  a 
Pre-Course  Survey,  Post-Teamwork  Survey,  and  electronic  postings  to  an  online  Teambuilding  Forum  where  students  discussed 
experiences  with  the  online  instructional  module  and  team  experiences  in  general.  For  purposes  of  confidentiality  and  analysis, 
the  electronic  postings  were  condensed  to  comments  related  specifically  to  student  perceptions  about  the  module  and  team 
experiences. 

Data  Gathering  Process 

Data  generated  by  two  online  surveys,  arranged  chat  sessions,  and  electronic  Teambuilding  Forum  postings  were  used  to 
obtain  data  relating  to  skills  gained  and  satisfaction  levels  of  students  who  developed  online  team  projects.  Participants  released 
these  data  for  the  purposes  of  this  study. 

Data  Analysis  and  Results 

Four  objectives  were  measured  for  this  study  using  quantitative  and  qualitative  data.  Multiple-choice  questions  were  used  to 
measure  quantitative  data  and  were  collected  from  two  surveys:  the  pre-course  survey  and  the  post-teamwork  survey.  Comments 
from  the  online  learners  were  used  to  provide  qualitative  data  and  were  collected  from  chat  sessions  and  the  electronic 
Teambuilding  Forum  postings. 

Survey  Questions 

The  first  objective  was  to  explore  whether  or  not  students  valued  the  VTB  instruction  they  received  as  preparation  for  online 
team  projects.  Overall,  responses  indicated  that  this  objective  was  met  in  terms  of  student  satisfaction  with  the  two  components 
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that  comprised  the  instructional  module,  and  student  perceptions  of  instruction  as  contributing  to  their  individual  personal  growth 
as  learners  and  educators.  Twenty -three  out  of  the  twenty-four  students  indicated  that  they  valued  the  VTB  instructional  module. 

The  second  objective  was  to  explore  how  students  who  received  specific  instruction  on  online  teambuilding  perceived  their 
actual  collaborative  experiences  as  members  of  a virtual  team.  An  overwhelming  majority  of  the  student  responses  indicated  that 
they  were  satisfied  with  individual  team  experiences  encompassing  elements  found  in  successful  online  teams:  receiving  support, 
getting  acquainted,  building  trust,  building  communication,  and  getting  organized. 

The  third  objective  was  to  determine  the  online  collaboration  skills  students  gained  after  they  had  completed  the  module. 
Twenty-two  of  the  students  out  of  twenty-four  acknowledged  that  they  gained  skills  and  knowledge  essential  to  online 
collaboration. 

The  fourth  objective  was  to  determine  whether  students  believed  that  they  would  use  the  online  teamwork  skills  and 
knowledge  they  learned  from  the  instructional  module  in  future  online  collaborative  teams.  More  than  twenty  of  the  students 
indicated  that  indeed  they  would  use  these  skills  and  knowledge. 

Student  Comments 

To  substantiate  some  of  the  numerical  data  gathered,  the  investigators  asked  students  to  comment  on  various  aspects  of  the 
module  and  their  online  team  experiences.  The  following  themes  emerged  from  the  data:  receiving  support,  getting  acquainted, 
establishing  communication,  building  trust,  getting  organized,  teambuilding  presentation,  teambuilding  activities,  teambuilding 
module,  and  overall  team  experience. 

Receiving  Support 

There  was  only  one  comment  that  related  specifically  to  student  perceptions  of  the  online  support  they  received  and  it  was 
positive:  “I  enjoyed  this  course  and  appreciate  the  hard  work  and  dedication  of  the  teaching  assistant  and  the  instructor.” 

Getting  Acquainted 

A total  of  1 1 comments  related  specifically  to  student  perceptions  of  how  well  their  team  became  acquainted.  Again,  these 
comments  were  all  positive  ranging  from,  “[I]  felt  very  fortunate  to  have  two  good  teammates”  to  “GO  TEAM!” 

Establishing  Communication 

There  were  a total  of  nine  student  comments  concerning  communication.  One  student  wrote,  “Our  backup  communication 
plan  is  [first]  external  email  and  then  by  telephone.”  Other  student  comments  were  somewhat  more  revealing  of  their  attitudes 
toward  communication.  For  example,  a student  commented  on  the  value  of  communication  to  the  team,  “We  ended  our  chat  by 
talking  about  the  importance  of  keeping  communication  lines  open  by  checking  email  daily  and  to  call  each  other  if  necessary.” 
Finally,  since  the  module  presented  instruction  on  how  to  communicate  more  efficiently  using  acronyms  and  more  expressively 
using  emoticons,  many  students  commented  on  how  they  felt  about  the  instruction  in  these  terms.  Two  typical  comments 
included:  “We  introduced  some  clever  and  amusing  emoticons  and  acronyms”  and  “Our  ‘favorite’  emoticons  and  acronyms:  ;-) 
LOL  :o)  :Q  TTYL  BRB  :-)  AKK.” 

Building  Trust 

There  were  a total  of  three  comments  relating  specifically  to  perceptions  of  trust  developed  in  teams  and  all  were  positive.  A 
typical  comment  was,  “...everyone  was  reliable  and  committed  to  the  team.” 

Getting  Organized 

Eleven  students  commented  on  their  perceptions  of  team  organization.  A few  comments  indicated  a feeling  that  students  were 
satisfied  with  how  their  team  organized  their  work.  For  example,  one  student  wrote,  “We  were  able  to  work  well  together.”  The 
majority  of  the  comments,  however,  were  neutral,  indicating  methods  of  team  organization  only:  “[We  used]  a sequential  order 
and  our  editing  path  was  determined,”  and  “Whenever  we  get  any  files  from  any  teammates,  we  will  send  a quick  reply  to  inform 
of  our  receiving.” 

Teambuilding  Presentation 

A total  of  nine  students  commented  on  their  perceptions  of  the  PowerPoint  Teambuilding  Presentation.  The  majority  of 
comments  were  positive:  “All  agreed  that  the  teambuilding  PowerPoint  Presentation  was  well  designed,  had  excellent  tips,  and 
would  serve  as  a great  reference  for  upcoming  team  projects.”  “The  PowerPoint  Presentation  is  a helpful  guide  for  teambuilding. 
It  was  well  organized  and  informative.”  There  was  one  slightly  negative  comment,  however,  concerning  the  length  of  the 
presentation:  “The  presentation  is  pretty  long,  but  it  is  really  useful.” 

Teambuilding  Activities 

There  were  a total  of  1 1 comments  that  related  specifically  to  student  perceptions  of  the  teambuilding  activities.  Of  all  the 
comments  made  by  students,  this  particular  set  of  comments  was  the  most  revealing  of  student  attitudes.  The  majority  of 
comments  were  positive,  with  only  one  comment  being  neutral.  Typical  comments  included:  “I  really  enjoyed  our  teambuilding 
chat”  and  “The  [teambuilding]  exercises  worked  well  to  get  us  going.” 

Complete  Module 

There  was  only  one  student  comment  related  to  the  complete  module  and  it  was  positive:  “The  whole  presentation  is  really 
easy  to  understand.” 

Overall  Team  Experience 

Only  one  comment  was  related  to  student  perceptions  of  the  overall  team  experience  and  it  was  exceedingly  positive:  “The 
overall  team  experience  definitely  built  my  confidence  to  participate  in  similar  activities  in  the  future.” 
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Discussion 

From  student  responses  relating  to  satisfaction  with  the  entire  module  and  its  specific  components,  overall  student  attitudes 
toward  the  instructional  module  were  favorable.  The  PowerPoint  presentation  component  received  a slightly  better  rating  than 
the  Teambuilding  Activity  Worksheet  component. 

The  responses  of  the  students  relating  to  their  individual  personal  growth  as  learners  and  educators  demonstrated  the  value  of 
the  module.  Overall,  their  opinions  indicated  that  they  felt  their  personal  learning  increased,  their  personal  creativity  increased, 
and  their  personal  collaboration  skills  improved  after  receiving  teambuilding  instruction  from  the  module  and  participating  in 
online  teams  for  the  team  projects.  The  comments  from  students  describing  their  enthusiasm  going  into  their  teamwork  included 
specific  language  and  ideas  taken  from  the  teambuilding  module  they  had  just  completed.  For  example,  students  commented  on 
specific  file  naming  systems  they  would  use,  types  of  editing  paths  they  would  implement,  and  ways  they  would  communicate 
and  build  trust.  It  is  evident  that  students  had  clear  intentions  derived  from  the  module  of  what  would  make  their  online 
teamwork  more  satisfying  and  successful  going  into  the  team  projects. 

The  students’  satisfaction  with  the  instruction  of  the  online  teambuilding  module  combined  with  their  reports  of  increased 
learning,  increased  creativity,  and  improved  collaboration  skills  help  to  strengthen  the  finding  of  Pascarella  et  al,  (1998)  that 
student  satisfaction  with  instruction  ultimately  leads  to  success.  Although  Pascarella’ s study  involved  only  face*to*face 
instruction,  this  positive  relationship  between  student  satisfaction  with  instruction  and  their  subsequent  success  appears  to  apply 
as  well  in  online  courses  using  any  type  of  instruction. 

Student  responses  and  comments  on  the  Post-Teamwork  Survey  indicate  that  they  did  indeed  perceive  their  online  team 
experiences  as  satisfying  and  successful.  The  students  rated  their  experiences  according  to  the  elements  found  in  successful 
online  teams:  getting  acquainted,  receiving  support,  establishing  communication,  building  trust,  and  getting  organized.  Students 
ranked  receiving  support  as  the  most  important  element  for  their  success  as  online  team  members.  This  finding  supports  the  work 
of  Berry,  Avergun,  and  Russ-Eft  (1993)  and  Haywood  (1998)  who  asserted  that  receiving  support  was  the  major  factor  in  team 
success. 

Contrary  to  the  expectations  of  the  investigators  who  assumed  that  the  module  would  be  more  rewarding  to  the  students  with 
less  online  technology  experience,  students  with  more  experience  seemed  to  value  the  module  more  than  students  with  less 
experience.  One  possible  explanation  for  the  higher  ratings  of  the  students  with  more  experience  may  be  that  these  students  felt 
more  secure  using  an  online  instructional  module  and  were  able  to  glean  more  from  it.  This  explanation  lends  further  support  to 
Sherry’s  (2000)  finding  that  greater  experience  with  technology  reduces  anxiety  and  fosters  positive  attitudes  toward  online 
technology. 

Half  of  the  comments  from  the  students  spoke  directly  to  the  applicability  of  the  module  indicating  they  valued  the  skills  they 
learned  enough  to  use  them  in  their  future  roles  as  educators  and  collaborators.  Additionally,  the  positive  responses  students 
made  on  the  Post  Teamwork  survey  reflected  that  indeed  they  would  apply  their  skills  towards  future  situations  involving  online 
collaboration.  This  may  substantiate  Rooney’s  (2000)  conclusion  that  online  collaboration  is  a skill  necessary  for  the  2P^  century 
workplace. 

Implications 

The  findings  of  this  investigation  have  several  implications:  1)  the  online  teambuilding  module  used  in  this  study  does  have 
an  overall  positive  effect  on  team  attitudes;  2)  students  who  develop  online  collaborative  projects  and  undergo  online 
teambuilding,  instruction  that  includes  teambuilding  activities  have  a high  satisfaction  rate  with  their  online  teams;  3)  students 
who  learn  online  teambuilding  skills  do  apply  them  to  their  online  collaborative  work;  4)  students  value  the  online  teambuilding 
skills  they  learned  to  the  extent  that  they  indicated  that  they  would  apply  to  future  online  collaborative  work;  5)  instructors  who 
assign  online  collaborative  projects  should  consider  incorporating  teamwork  instruction  in  their  online  classes;  and  6)  the 
elements  found  in  successful  teams,  receiving  support,  getting  acquainted,  establishing  communication,  building  trust,  and 
getting  organized,  are  also  elements  found  in  teams  satisfied  with  their  online  teamwork. 

Future  Studies 

Since  a number  of  the  students  from  this  study  wrote  positive  comments  about  their  teammates  and  their  experiences,  the 
investigators  believe  that  this  may  have  implied  that  the  manner  in  which  teams  were  formed  was  done  successfully.  A study  of 
the  combination  of  characteristics  used  to  form  the  teams  could  lead  to  interesting  findings. 

Additionally,  as  of  this  writing,  the  investigators  have  received  a number  of  inquiries  about  the  module  from  business  entities. 
These  inquiries  warrant  future  research  be  conducted  beyond  the  academic  environment. 

Finally,  support  plays  a critical  role  in  the  success  or  failure  of  a team  (Berry,  Avergun,  & Russ-Eft,  1993;  Haywood,  1998), 
For  this  particular  study,  the  investigators  developed  a checklist  for  the  instructor  that  emphasized  the  important  ways  that  she 
could  offer  her  personal  support  to  the  online  teams.  In  the  future,  however,  a web-based  information  guide  or  pamphlet  might  be 
distributed  to  instructors  describing  specific  ways  that  they  can  support  online  teamwork. 

The  researchers  for  this  investigative  study  developed  workshops  to  instruct  faculty  members  at  the  University  of  Hawaii  at 
Manoa  and  at  Hawaii  Pacific  University  on  how  they  might  support  virtual  team  collaboration  for  their  online  courses.  Overall, 
the  workshops  received  positive  feedback  from  the  attendees.  Evaluations  from  these  workshops  and  others  like  it  may  also 
contribute  to  a database  for  further  research. 

VTB  Web  Site  Access 

The  overwhelming  positive  response  from  students  combined  with  the  enthusiasm  of  educators  and  business  professionals 
have  led  the  researchers  to  currently  work  towards  marketing  their  VTB  module.  However,  at  present,  they  do  allow  educators 
full  access  to  it  in  return  for  feedback  on  its  usefulness  and  suggestions  for  its  improvement.  To  view  a web  site  that  provides 
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samples  taken  directly  from  the  full  VTB  web  site,  please  visit:  <http://members.home.net/vtbsolutions>.  You  may  also  email  the 
researchers  to  request  permission  for  access  to  the  full  VTB  web  site  at:  <vtbsolutions@hotmail.com>.  Please  include  your  name, 
title,  institution,  and  how  you  would  like  to  use  the  web  site. 
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Introduction 

Recent  research  has  shown  that  self-regulated  learning  is  an  important  aspect  of  student  academic  performance  in  the 
classroom.  Self-regulated  learning  can  be  defined  as  a student’s  use  of  specified  strategies  to  achieve  academic  goals  on  the  basis 
of  self-efficacy  perceptions.  Students  practicing  self-regulation  behaviors  initiate  and  direct  their  own  efforts  to  acquire 
knowledge  and  skill  rather  than  relying  on  teachers,  parents,  or  others.  In  general,  self-regulated  learning  consists  of  three 
essential  elements:  commitment  to  academic  goals,  self-efficacy  perceptions,  and  self-regulated  learning  strategies  (Zimmerman, 
1989). 

Student  academic  goals  are  the  underlying  reasons  or  purposes  for  the  students'  learning  behaviors.  They  represent  the 
meaning  that  students  assign  to  achievement  situations.  These  goals  provide  a sort  of  cognitive  organizational  structure  that 
encompasses  how  students  define  their  successes  and  failures,  affective  reactions,  and  future  behaviors  (Urdan,  1997). 
Researchers  on  achievement  motivation  have  found  that  different  types  of  goal  orientations  elicit  different  motivational  processes 
(Ames  & Archer,  1988). 

Academic  goals  are  most  often  described  as  either  mastery  or  performance  goals.  Students  possessing  mastery  goals  are 
considered  to  be  intrinsically  motivated,  primarily  focusing  on  learning  or  mastering  the  course  material.  Because  they  value  the 
learning  process  itself,  these  students  often  seek  out  challenging  assignments  and  put  forth  more  effort  to  learn  the  material.  They 
also  tend  to  use  more  effective  learning  strategies  while  studying.  In  contrast,  students  with  performance  goals  are  considered  to 
be  extrinsically  motivated.  These  students  tend  to  focus  on  the  outcome  of  their  learning  and  are  primarily  interested  in  earning  a 
good  grade  in  the  course,  or  gaining  social  esteem  (Dweck,  1986;  Pintrich  1995).  Since  they  are  mostly  concerned  with  the 
reward  that  comes  after  they  have  learned  the  material,  they  tend  to  use  less  effective  learning  strategies.  For  students  with 
performance  goals,  learning  the  material  is  often  seen  as  a means  to  an  end  rather  than  an  end  in  itself 

Although  student  goals  provide  direction  and  incentive  for  academic  work,  a second  self-regulation  element  affecting  student 
achievement  is  the  students’  beliefs  about  his  or  her  ability.  Belief  in  ones’  ability  to  successfully  perform  a particular  task  is 
known  as  self-efficacy.  Bandura  (1986)  stated  that  self-efficacy  beliefs  influence  an  individual’s  willingness  to  attempt  a 
particular  task,  the  level  of  effort  he  or  she  will  employ,  and  his  or  her  persistence  in  accomplishing  the  task.  Self-efficacy  is 
particularly  important  because  of  its  two-fold  effect  on  the  other  components  of  self-regulation.  Not  only  does  self-efficacy 
influence  the  type  of  goals  students  set  for  themselves  but  it  also  affects  the  amount  of  effort  they  invest  in  working  toward  these 
goals  (Pintrich,  1995). 

Previous  research  on  self-efficacy  has  indicated  that  student  behaviors  can  often  be  better  predicted  by  their  beliefs  about 
their  capabilities  than  by  what  they  are  actually  capable  of  accomplishing.  Results  have  shown  that  people’s  beliefs  help 
determine  what  they  do  with  the  knowledge  and  skills  that  they  have  (Pajares  & Miller,  1994).  Typically,  students  with  high -self 
efficacy  are  confident  in  their  skills  and  abilities  to  do  well  and  have  been  shown  to  participate  more  in  learning  activities,  show 
greater  effort  and  persistence,  and  achieve  higher  levels  of  academic  performance  than  students  with  low  self-efficacy  (Pintrich  & 
De  Groot,  1990;  Schunk,  1991).  Even  when  experiencing  difficulty,  students  with  high  self-efficacy  tend  to  work  longer  and 
harder  than  do  students  with  low  self-efficacy.  On  the  other  hand,  students  with  low  self-efficacy  frequently  show  less  persistence 
and  may  attempt  to  avoid  the  learning  situation  altogether  (Hagen  & Weinstein,  1995).  Lack  of  self-efficacy  has  also  been 
coupled  with  the  debilitating  affect  of  high  test-anxiety  (Bandura,  1986). 

A third  element  of  self-regulation  consists  of  student’s  learning  strategies.  Self-regulated  learning  strategies  are  the  behaviors 
and  actions  students  use  to  acquire  desired  information  or  skills.  They  include  such  methods  as  organizing  and  applying  new 
information,  self- monitoring  one’s  performance,  seeking  assistance,  and  managing  time  and  study  environments  (Pintrich,  Smith, 
Garcia  & McKeachie,  1991;  Zimmerman,  1989;  Zimmerman  & Martinez-Pons,  1986).  Students'  use  of  self- regulated  learning 
strategies  depends  not  only  on  their  knowledge  of  strategies  but  also  on  their  academic  goals  and  self-efficacy  perceptions. 
Students  with  learning  goals  tend  to  use  deep  processing  strategies  that  enhance  their  understanding  of  concepts.  They  attempt  to 
integrate  information  and  monitor  their  comprehension  (Pintrich  & Garcia,  1991).  Conversely,  students  with  performance  goals, 
tend  to  use  strategies  that  promote  only  short-term  and  surface  level  processing,  like  memorizing  and  rehearsing  (Graham  & 
Golan,  1991). 

In  much  of  the  previous  research  on  self-regulated  learning,  the  focus  has  been  on  determining  the  foundational  elements  of 
the  construct  and  the  relationship  between  these  elements.  The  results  of  this  work  have  indicated  that  self-regulatory  processes 
are  linked  with  content  domains,  and  individuals  learn  how  to  apply  these  skills  in  a given  learning  or  applied  context 


o 

ERLC 


312 


22A 


(Zimmerman,  1998).  Determining  the  specific  self-regulatory  processes  associated  with  successful  learning  in  particular  content 
domains  is  an  important  next  step  in  this  line  of  research. 

Computer  Literacy  courses  are  offered  by  many  colleges  and  universities  and  are  often  taken  by  students  from  various 
academic  majors.  For  many  students,  this  course  is  a requirement  of  their  degree  programs.  For  others,  the  importance  and 
applicability  of  the  content  information  are  influential  factors  in  their  desire  to  take  the  course.  Very  often,  student  enrollment  in 
these  courses  may  be  as  high  as  two-  to  three-hundred  students  per  semester.  Because  Computer  Literacy  courses  are  so  prevalent 
and  the  content  a necessity  to  many  college  students,  it  is  important  to  determine  the  relationship  of  motivation  and  learning 
strategies  affecting  learning  and  performance  in  these  courses.  It  is  also  important  to  discover  the  various  approaches  and 
methodologies  undergraduate  students  report  they  use  to  learn  the  course  material. 

The  purpose  of  this  study  was  to  determine  the  relationship  among  students'  reports  about  their  goal  orientation,  self-efficacy 
and  self- regulated  strategy  use  and  their  academic  performance  in  a Computer  Literacy  course  as  indicated  by  course  grade.  We 
also  investigated  students'  reports  about  their  most  preferred  and  utilized  study  techniques  and  the  techniques  they  used  to 
monitor  their  learning  in  this  course. 

Method 

Subjects 

All  participants  in  this  study  were  students  in  a general  studies  Computer  Literacy  course  at  a large  university  in  the 
southwest.  Of  the  291  participants,  193  were  female  and  98  male.  The  majority  of  the  participants  were  education  (27%), 
communication  (18%),  or  broadcasting  (1 1%)  majors.  In  total,  26  different  academic  majors  were  represented.  Four  percent  were 
freshman,  27%  sophomores,  47%  juniors,  21%  seniors,  and  1%  graduate  students.  Students  ranged  in  age  from  18  years  to  50 
years,  with  an  overall  average  age  of  22. 

Procedures 

The  Computer  Literacy  course  was  a multi-section  course  consisting  of  a lecture  class  and  lab.  The  lecture  portion  of  the 
course  met  in  a large  lecture  hall  twice  a week  for  50  minutes,  while  the  lab  section  met  in  a PC  computer  lab  once  a week  for  a 
period  of  one  hour  and  50  minutes.  In  total,  there  were  17  lab  sections  for  this  course.  Data  were  collected  at  the  end  of  the  fall 
2000  semester  during  lab  sections  in  which  students  were  completing  their  semester  final  exam.  In  each  lab  section,  the 
investigator  described  the  survey,  "Strategies  Used  for  Learning  in  a Computer  Literacy  Course".  Students  completed  the  survey 
after  finishing  their  final  exam.  Participation  in  the  study  was  voluntary.  Students  were  given  two  extra  credit  points  toward  their 
course  grade  for  completing  the  survey. 

Materials 

The  participants  completed  the  survey  "Strategies  Used  for  Learning  in  a Computer  Literacy  Course".  This  survey  consisted 
of  three  sections.  The  first  section  included  demographic  questions  as  well  as  selected  response  questions  regarding  the  lowest 
grade  they  would  be  happy  with  in  this  course,  and  how  many  hours  a week  they  study  for  this  course.  Participants  were  also 
asked  to  respond  to  a series  of  yes  or  no  questions  aimed  at  discovering  their  reasons  for  taking  this  course.  Examples  of  these 
questions  were,  "The  course  is  required  for  my  major",  "The  content  seems  interesting",  "It  is  an  easy  elective",  and  "It  will 
improve  my  career  prospects". 

The  second  section  of  the  survey  included  73  motivation  and  learning  strategies  questions  adopted  from  the  Motivated 
Strategies  for  Learning  Questionnaire  (MSLQ)  (Pintrich  et.al.,  1991).  The  MSLQ  is  a self-report  instrument  designed  to  assess 
college  students'  motivational  orientations  and  their  use  of  different  learning  strategies.  The  motivation  section  of  the  MSLQ 
consists  of  six  sub-scales  with  items  designed  to  assess  students'  goals  and  value  beliefs  for  a course,  their  beliefs  about  their 
skills  to  succeed  in  a course,  and  their  anxiety  about  tests  in  a course.  The  learning  strategy  section  consists  of  nine  sub-scales 
with  items  regarding  students'  use  of  different  cognitive  and  metacognitive  strategies  as  well  as  management  of  various  resources. 
Students  rate  themselves  on  a 7-point  Likert  scale  (1  = not  true  of  me,  to  7 = very  true  of  me). 

The  various  sub-scales  on  the  MSLQ  can  be  used  together  or  separately.  Two  learning  strategy  sub-scales.  Rehearsal,  and 
Help  Seeking,  were  not  used  in  this  study  due  to  the  lower  Cronbach  alpha  scores  published  by  Pintrich  et.al.  (1991)  for  these 
scales  (.69,  and  .52  respectively). 

The  third  section  of  the  survey  consisted  of  eight  questions,  two  selected-response  and  six  open-ended,  focusing  on  student 
study  habits.  The  selected-response  questions  asked  students  if  they  study  differently  for  this  class  than  for  their  other  classes  and 
who  is  responsible  for  their  success  in  learning,  themselves  or  their  instructor.  The  open-ended  questions  asked  participants  to 
describe  two  ways  that  they  study  for  this  class,  two  ways  that  they  study  for  their  other  classes,  how  they  check  their 
understanding  of  the  material  while  studying  for  this  course,  what  is  their  major  strength  as  a learner,  what  is  their  major 
weakness  as  a learner,  and  what  they  think  would  help  them  become  better  learners. 

Data  Analysis 

Using  the  method  developed  by  Pintrich  et.al.  (1991),  the  MSLQ  sub-scale  scores  for  each  participant  were  constructed  by 
taking  the  mean  of  the  items  that  make  up  that  scale.  For  example,  intrinsic  goal  orientation  has  four  items.  An  individual's  score 
for  intrinsic  goal  orientation  was  computed  by  summing  the  four  items  in  the  sub-scale  and  taking  the  average.  There  are  some 
negatively  worded  items  and  the  ratings  were  reversed  before  an  individual's  score  was  computed.  The  statistics  reported 
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represent  the  positive  wording  of  all  the  items.  In  general,  a higher  score  of  4,  5,  6,  or  7 for  a sub-scale  mean  score  indicates  that 
the  student  feels  the  items  were  a fairly  good  representation  of  their  motivational  orientation  or  learning  strategies  used  in  this 
course. 

Multiple  regression  analysis  for  two  unordered  sets  of  predictors  was  used  to  evaluate  how  well  the  use  of  specific  motivation 
and  learning  strategies  predicted  course  grade.  The  Holms  Method  was  used  to  control  for  Type  I error. 

The  responses  to  each  open-ended  question  were  analyzed  and  categorized  by  discemable  themes.  The  number  of  responses 
in  each  thematic  category  was  then  calculated.  Responses  to  the  selected-response  questions  were  compiled  and  summarized  by 
frequency  of  occurrence. 

Results 

Three  sets  of  analyses  were  conducted  and  results  are  organized  accordingly.  The  first  set  consisted  of  multiple  regression 
analyses  examining  the  MSLQ  responses  and  their  relationship  to  course  grade.  In  the  second  set  of  analyses,  frequencies  of 
responses  and  the  thematic  categories  to  the  eight  study -habit  questions  were  determined.  In  the  last  set  of  analyses,  the  students' 
answers  to  the  selected-response  general  course  questions  were  analyzed  and  summarized. 

Results  from  MSLQ  Assessment  of  Student  Motivational  Orientations  and  Learning  Strategies 
Table  1 displays  the  means  and  standard  deviations  for  course  grade  and  scores  on  the  MSLQ  sub-scales.  Significant  sub-scale 
mean  scores  will  be  discussed  along  with  mean  scores  for  individual  items  on  these  sub-scales. 


Table  1 Mean  Scores  and  Standard  Deviations  on  Course  Grade  and  MSLQ  Sub-scale  Summaries 


Variables 

Mean 

Standard 

Deviation 

Course  Grade 

2.73 

.98 

Motivation  Scales 

Intrinsic  Goal  Orientation 

4.62 

1.10 

Extrinsic  Goal  Orientation 

5.00 

1.23 

Task  Value 

5.10 

1.18  s 

Control  of  Learning  Beliefs 

5.18 

1.06 

Self-Efficacy  for  Learning  and  Performance 

5.32 

1.07 

Test  Anxiety 

3.87 

1.46 

Learning  Strategy  Scales 

Elaboration 

4.21 

1.16 

Organization 

3.71 

1.33 

Critical  Thinking 

3.49 

1.24 

Metacognition 

4.01 

.94 

Time  and  Study  Environment  Management 

4.31 

1.10 

Effort  Regulation 

4.25 

.85 

Peer  Learning 

3.06 

1.52 

Note:  Sub-scale  mean  scores  can  range  from  1 to  7. 

The  possible  mean  scores  for  the  MSLQ  sub-scales  can  range  from  one  to  seven.  The  selection  of  a one  for  an  item  on  a sub- 
scale indicated  that  the  student  believed  the  item  was  not  at  all  true  of  them,  whereas  a selection  of  seven  indicated  that  the 
student  believed  the  item  was  very  true  of  them.  The  scores  for  all  the  individual  items  on  the  sub-scale  were  then  averaged 
together  to  determine  the  mean  score  for  the  sub-scale. 

The  range  of  final  course  grades  was  from  A through  E.  The  numeric  value  associated  with  each  grade  was  A = 4,  B = 3,  C = 
2,  D = 1,  E = 0.  Final  course  grades  resulted  in  the  following  distribution:  A=  65  (22%),  B = 120  (42%),  C = 75  (26%),  D = 24 
(8%),  and  E = 7 (2%). 

Motivation  Sub-scale  Results 

In  response  to  the  sub-scale  items  on  the  motivation  scale,  participants  rated  extrinsic  goal  orientation  and  self-efficacy  fairly 
high,  as  indicated  by  the  Extrinsic  Goal  Orientation  and  Self-Efficacy  for  Learning  and  Performance  sub-scale  mean  scores.  The 
mean  score  for  the  Extrinsic  Goal  Orientation  sub-scale  was  5.0  and  Self-Efficacy  for  Learning  and  Performance  was  5.3. 
Additionally,  participants  appear  to  not  worry  about  course  tests  as  indicated  by  a mean  score  of  3.8  on  the  Test  Anxiety  sub- 
scale. 

Extrinsic  goal  orientation  sub-scale  results. 

There  were  four  items  on  the  Extrinsic  Goal  Orientation  sub-scale,  with  three  items  focusing  on  the  importance  of  course 
grades  and  one  item  focusing  on  the  approval  of  others.  Mean  response  scores  for  each  of  the  three  items  asking  students  to  rate 
the  importance  of  earning  high  course  grades  were  fairly  high,  with  each  item  mean  score  over  5.0.  These  items  asked  students  to 
rate  their  satisfaction,  value,  and  desire  to  earn  a grade  higher  grade  than  their  classmates.  Based  on  the  mean  scores  for  each 
item,  it  appears  that  earning  a good  grade  was  a goal  of  many  students. 

The  one  item  on  the  sub-scale  focusing  on  the  importance  of  earning  a good  grade  in  order  to  receive  approval  from  others 
had  a mean  score  of  4.3.  The  mean  score  is  approximately  at  the  midpoint  of  the  scale  range,  indicating  that  for  some  students, 
earning  a good  grade  was  important  in  order  to  prove  their  ability  to  others. 
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Self-efficacy  for  learning  and  performance  sub-scale  results. 

There  were  eight  items  on  the  Self-efficacy  for  Learning  and  Performance  sub-scale,  with  five  items  focusing  on  the  students' 
judgment  about  his  or  her  ability  to  accomplish  the  tasks  for  the  course,  and  three  items  focusing  on  the  students'  expectation  for 
success  in  the  course. 

Mean  response  scores  for  the  five  items  focusing  on  the  students'  beliefs  about  being  able  to  accomplish  the  tasks  for  the 
course  were  positive  and  ranged  from  4.7  to  6.1  on  the  seven-point  scale.  These  items  asked  students  to  rate  their  beliefs  in  their 
ability  to  understand  both  basic  and  complex  course  material,  and  their  confidence  in  performing  well  on  course  assignments  and 
tests. 

Mean  response  scores  for  each  of  the  three  items  focusing  on  the  students'  expectancy  for  success  were  also  very  positive  and 
ranged  from  5.1  to  5.6.  These  items  asked  students  to  rate  their  beliefs  on  being  able  to  earn  an  excellent  grade,  and  their  beliefs 
in  their  overall  ability  to  do  well  in  the  course. 

Test  anxiety  sub-scale  results. 

There  were  five  items  on  the  Test  Anxiety  sub-scale,  with  three  items  focusing  on  worry  or  negative  thoughts  during  test 
taking  and  two  items  focusing  on  physiological  arousal  aspects  of  anxiety,  such  as  upset  feelings,  and  rapid  heart  beat 

The  mean  response  scores  for  the  three  items  focusing  on  worry  were  approximately  at  the  mid-point  of  the  seven  point  scale, 
ranging  from  3.2  to  4.2.  These  mean  scores  seem  to  indicate  that  for  most  students,  they  were  not  worrying  about  the  possibility 
of  a poor  performance  or  even  failure  during  test  taking. 

The  mean  response  scores  for  the  items  focusing  on  the  physiological  aspects  of  anxiety  were  3.6  and  4.2.  These  mid-range 
mean  scores  indicated  that  most  students  were  not  upset  or  did  not  have  uneasy  feelings  during  test  taking. 

Learning  Strategy  Scale  Results 

In  response  to  the  learning  strategy  items,  participants  rated  elaboration  fairly  high  and  peer  learning  fairly  low  as  indicated 
by  the  Elaboration  and  Peer  Learning  sub-scale  mean  scores.  The  mean  score  for  the  Elaboration  sub-scale  was  4.21  and  Peer 
Learning  was  3.06. 

Elaboration  sub-scale  results . 

There  were  six  items  on  the  Elaboration  scale  all  focusing  on  study  techniques  that  help  students  integrate  and  connect  new 
information  with  prior  knowledge.  Mean  response  scores  for  these  items  ranged  from  a low  mean  score  of  3.0  to  a fairly  high 
mean  score  of  5.0. 

The  mean  response  score  for  the  item  asking  students  whether  they  write  brief  summaries  of  course  readings  had  a low  score 
of  3.0,  indicating  that  most  students  did  not  use  this  study  technique.  The  remaining  items  on  this  sub-scale  asked  students  if 
they  try  to  connect  the  information  learned  in  this  course  to  prior  knowledge  or  to  other  courses  had  higher  scores  of  4.2  to  5.0, 
indicating  that  many  students  used  these  methodologies  when  studying. 

Peer  learning  sub-scale  results . 

There  were  three  items  on  the  Peer  Learning  scale  all  focusing  on  whether  students  worked  with  classmates  to  complete 
assignments  or  enhance  their  understanding  of  course  content.  Mean  response  scores  for  all  three  items  were  fairly  low  on  the 
seven-point  scale,  ranging  from  3.3  to  3.8.  These  low  scores  seem  to  indicate  that  students  did  not  prefer  to  work  with  classmates 
in  order  to  learn  the  course  material. 

Analyses  to  Determine  Relationship  Among  Motivational  Orientations,  Learning  Strategies,  and  Course  Grade 

Two  multiple  regression  analyses  were  conducted  to  predict  final  course  grade  from  students'  self-reported  motivation  and 
use  of  learning  strategies.  One  analysis  included  the  six  motivation  strategies  as  predictors  (intrinsic  goal  orientation,  extrinsic 
goal  orientation,  task  value,  control  of  learning  beliefs,  self-efficacy  and  test  anxiety),  while  the  second  analysis  included  the 
seven  learning  strategies  predictors  (elaboration,  organization,  critical  thinking,  metacognition,  environment  regulation,  effort 
regulation,  and  peer  learning).  For  both  analyses,  the  Holms  Method  was  used  to  correct  for  Type  I error.  These  two  sets  of 
predictors  are  unordered.  The  regression  equation  with  the  motivation  strategies  was  significant,  = .14,  adjusted  = .12,  F(6, 

284)  = 7.70,  p <.001.  The  regression  equation  with  the  learning  strategies  was  also  significant,  = .13,  adjusted  F?  = .\\,  F {1 , 

283)  = 6.27,  p <.001. 

Indices  indicating  the  relative  strength  of  the  individual  predictors  are  presented  in  Table  2.  Of  the  motivation  components, 
extrinsic  goal  and  self-efficacy  were  positively  related  to  course  grade,  while  test  anxiety  was  negatively  related  to  course  grade. 
Of  the  learning  strategies,  elaboration  was  positively  related  to  course  grade  and  peer  learning  was  negatively  related  to  course 
grade. 
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Table  2 The  Bivariate  and  Partial  Correlations  of  the  Predictors  with  Course  Grade 


Predictors 

Correlation  between 
each  predictor  and 
course  grade 

Correlation  between 

each  predictor  and 
course  grade  controlling 
for  all  other  predictors 

Motivation  Scales 

Intrinsic  Goal  Orientation 

.12 

-.03 

Extrinsic  Goal  Orientation 

.22*** 

.19 

Task  Value 

.09 

-.05 

Control  of  Learning  Beliefs 

.09 

-.04 

Self-Efficacy  for  Learning  and 

Performance 

.30** 

.17 

Test  Anxiety 

-.18* 

-.16 

Learning  Strategy  Scales 

Elaboration 

.13  *** 

.16 

Organization 

.04 

-.04 

Critical  Thinking 

-.07 

-.15 

Metacognition 

.11 

.08 

Time  and  Study  Environment 

Management 

.22 

.14 

Effort  Regulation 

‘.03 

.02 

Peer  Learning 

- 19**** 

-.21 

* p <.0125,  **  p<.01,  ***  p<.008,  ****  p<.007 


Next,  in  order  to  determine  moderating  effects  of  one  significant  variable  on  another  significant  variable,  we  created  five 
interaction  terms.  Development  of  the  interaction  term  was  based  on  both  the  statistical  significance  of  each  variable  and 
conceptual  importance.  The  five  interaction  terms  were  created  by  developing  cross-products  of  the  following  variable 
combinations:  self-efficacy  and  peer  learning,  extrinsic  goal  and  peer  learning,  extrinsic  goal  and  self-efficacy,  extrinsic  goal  and 
test  anxiety,  self-efficacy  and  test  anxiety.  Of  the  five  interaction  terms,  only  two  were  significant. 

The  first  interaction  term  created  by  the  cross  product  of  self-efficacy  and  peer  learning  was  significantly  related  to  course 
grade,  F(3,  287)  = 14.09,  p < .001.  Students  who  had  high  self-efficacy  who  collaborated  with  their  peers  were  likely  to  have 
lower  grades.  In  contrast,  students  with  low  self-efficacy  who  collaborated  with  peers  were  more  likely  to  have  higher  grades. 

The  second  interaction  term  created  by  the  cross  product  of  extrinsic  goal  and  peer  learning  was  also  significantly  related  to 
course  grade,  F (3,287)  = 1 1.27,  p <.001.  Students  with  high  extrinsic  goal  orientation  who  collaborated  with  their  peers  were 
likely  to  have  lower  grades.  In  contrast,  students  with  low  extrinsic  goal  orientation  who  collaborated  with  their  peers  were  likely 
to  have  higher  grades. 

Responses  to  Questions  About  Students^  Study  Habits 

In  addition  to  responding  to  the  MSLQ  items,  students  were  also  asked  to  respond  to  two  selected-response  questions  and  six 
open-ended  questions  focusing  on  their  study  habits.  Their  responses  were  analyzed  by  frequency  of  occurrence.  Not  all 
participants  answered  all  of  the  questions  in  this  section,  possibly  due  to  time  constraints  or  simply  lack  of  interest  in  responding. 
Because  of  this,  the  total  number  of  responses  for  each  question  may  not  equal  the  total  number  of  participants.  Summaries  of  the 
responses  are  provided  in  Table  3.  The  responses  for  each  question  are  listed  in  rank  order  of  occurrence,  beginning  with  the 
highest-ranking  response.  The  numbers  provided  indicate  the  total  responses.  The  percentages  are  based  on  the  total  number  of 
responses  for  the  particular  question.  Results  of  the  analyses  on  the  two-selected  response  questions  will  be  presented  first,  and 
will  then  be  followed  by  the  results  of  the  analyses  on  the  six  open-ended  questions. 
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Table  3 Summary  of  Responses  to  Study  Habit  Questions 


Do 

you 

As 

a 

What 

What 

How  do  you 

What  is 

What  is 

What  would 

study 

student. 

methods  do 

methods  do 

check  your 

your  major 

your  major 

help  you 

different-ly 

who  do  you 

you  use  to 

you  use  to 

under- 

strength as 

weakness  as 

become  a 

for 

this 

think 

is 

study  for 

study  for 

standing  of 

a learner? 

a learner? 

better 

class 

than 

respon-sible 

this  course? 

your  other 

course 

(108) 

(77) 

learner? 

most 

of 

for  your 

(222) 

courses? 

material? 

(97) 

your 

other 

success 

in 

(200) 

(105) 

classes? 

learning? 

(150) 

(150) 

No 

78 

I am  119 

Read  text 

Read  text 

Apply 

Ability  to 

Procras- 

Study 

(52%) 

(79%) 

and  notes 

and  notes 

information 

memorize 

tinate, 

schedule  23 

106(48»/o) 

115(57%) 

32  (30%) 

26  (24%) 

unmoti- 

vated, 

lazy 

33  (43%) 

(24%) 

Yes 

72 

My 

Apply 

Outline 

Quiz  myself 

Visual 

Low 

Disci-pline 

(48%) 

instructor 

informa- 

readings 31 

29  (28%) 

learner 

attention 

23  (24%) 

12  (8%) 

tion 

(16%) 

22  (20%) 

span 

51  (23%) 

28  (36%) 

Both 

19 

Use  course 

Study  with 

I don't 

Ability  to 

Lack  of 

Hands-on, 

(13%) 

study  guide 

peers 

16(15%) 

compre- 

time 

real  world 

31  (10%) 

21  (10%) 

hend  and 

10(13%) 

applica- 

under-stand 

tions 

19(18%) 

17(18%) 

Use 

Use 

Make  sure 

Hands-on 

Not  good  in 

More  time 

fiashcards 

fiashcards 

it  is 

learner 

lectures 

14(14%) 

17(7%) 

20(10%) 

memo-rized 

10(10%) 

18(17%) 

6 (8%) 

Study  with 

Memo-rize 

Discuss- 

Persis-tence 

Better  study 

peers 

material 

ions  with 

and  dedica- 

skills 

9 (4%) 

9 (5%) 

Teaching 

tion 

11  (11%) 

Assistants 

10(10%) 

13(12%) 

Highlight 

Attend  class 

Discuss- 

Quick 

Read  more 

text 

4 (2%) 

ions  with 

learner 

9 (9%) 

8 (4%) 

peers 
8 (7%) 

10  (9%) 

Note:  Number  of  responses  varies  between  questions  because  some  participants  did  not  provide  responses  while  others  provided 
multiple  responses.  Percentages  based  on  total  number  of  responses  for  each  question. 

Selected-response  Question  Results 

The  first  selected-response  question  asked  students  if  they  studied  differently  for  this  Computer  Literacy  course  than  for  their 
other  courses.  Students  were  to  respond  by  circling  either  “Yes”  or  "No”.  Of  the  150  participants  responding  to  this  question,  78, 
or  52%,  circled  "Yes",  indicating  that  they  studied  differently,  and  seventy-two,  or  48%  circled  "No**,  indicating  that  they  studied 
the  same  way. 

The  second  question  asked  students  who  they  thought  has  responsibility  for  their  success  in  learning.  Students  were  to 
respond  by  circling  "I  am"  or  "My  instructor  is".  Again,  150  students  responded.  The  vast  majority  of  the  students,  1 19  or  79%, 
circled  *'I  am**  indicating  that  they  are  responsible.  A small  group  of  students,  12  or  8%,  circled  "My  instructor"  indicating  that 
they  feel  the  instructor  is  responsible  for  their  success  in  learning,  and  19,  orl3%,  wrote  in  that  both  they  and  the  instructor  are 
responsible  for  their  success  in  learning. 

Open-ended  Question  Results 

The  first  open-ended  item  in  this  section  asked  students  to  list  two  ways  that  they  studied  for  the  Compiler  Literacy  course. 
Reading  the  text  and  notes  was  the  most  frequently-listed  study  technique,  with  106  responses  or  48%,  followed  by  applying 
information  learned  in  lecture  to  the  lab  class,  with  51  responses,  or  23%.  Studying  with  peers  was  listed  only  9 times,  or  4%. 

The  second  open-ended  question  asked  students  to  list  two  ways  that  they  studied  for  their  other  courses.  Again,  the  most 
frequently -listed  study  technique  mentioned  by  students  was  reading  the  text  and  notes,  with  1 15  responses,  or  56%.  The  next 
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most  frequently  occurring  response  was  outlining  readings,  listed  31  times,  or  15%.  Studying  with  peers  was  listed  21  times, 
accounting  for  10%  of  the  responses. 

The  third  open-ended  question  asked  students  to  describe  how  they  check  their  understanding  of  the  Computer  Literacy 
course  material.  The  responses  to  this  question  indicated  that  there  were  two  methods  that  most  students  used  to  check  their 
understanding.  Thirty-two  students,  or  30%,  indicated  that  applying  the  lecture  information  by  working  on  the  computer  helped 
them  to  determine  their  understanding  of  the  material,  29  students,  or  28%,  stated  that  they  quizzed  themselves,  and  16,  or  15%, 
stated  that  they  didn’t  check  their  understanding. 

The  fourth  open-ended  question  asked  students  what  they  considered  to  be  their  strength  as  a learner.  These  results  were 
mixed,  with  three  characteristics  being  mentioned  most  often.  In  total,  108  students  responded,  with  26  participants,  or  24%, 
indicated  their  ability  to  memorize  was  their  strength.  Twenty -two  students,  or  20%,  stated  that  their  strength  was  based  on  the 
fact  that  they  were  visual  learners,  and  19  students,  or  1 8%,  cited  their  ability  to  comprehend  and  understand. 

The  fifth  open-question  asked  students  what  they  considered  to  be  their  weakness  as  a learner.  Most  responses  centered 
around  two  main  themes,  procrastination  and  low  attention  span.  Of  the  seventy -seven  students  who  responded  to  this  question, 
33  students,  or  43%,  indicating  that  procrastination,  lack  of  motivation  and  laziness  was  their  weakness,  and  28,  or  36%  of 
students  indicated  that  their  weakness  was  due  to  their  low  attention  span.  Ten  students,  or  13%,  indicated  that  they  didn't  have 
enough  time  to  dedicate  toward  studying. 

The  final  open-ended  question  asked  participants  what  they  thought  would  help  them  to  become  a better  learner.  From  the 
responses,  it  appears  that  there  are  four  factors  students  felt  could  possibly  influence  their  learning.  Of  the  97  students  that 
responded,  23,  or  24%,  indicated  a study  schedule  would  be  helpful,  and  23  indicated  that  they  needed  to  be  more  disciplined. 
Seventeen  students,  or  18%,  stated  that  they  needed  more  hands-on,  real  world  applications,  and  14  students,  or  14%,  needed 
more  time  in  their  daily  lives  to  dedicate  toward  school. 

Responses  to  General  Course  Questions 

Participants  were  also  asked  to  respond  to  a series  of  selected  response  questions  regarding  the  lowest  grade  they  would  be 
happy  with,  and  how  many  hours  a week  they  study  for  this  course.  They  were  also  asked  to  respond  yes  or  no  to  a series  of  nine 
items  aimed  at  discovering  their  reasons  for  taking  this  course. 

Lowest  grade  acceptable 

Participants  were  asked  to  indicate  the  lowest  course  grade  that  would  be  acceptable  to  them.  A,  B,  C,  D,  or  E.  For  each 
participant,  the  actual  grade  earned  was  then  compared  to  the  lowest  grade  acceptable.  Summary  of  the  responses  and  the 
comparison  between  the  lowest  grade  acceptable  and  actual  grade  earned  are  provided  in  Table  4. 


Table  4 Comparison  of  Lowest  Grade  Acceptable  to  Actual  Grade  Earned 


Lowest  Grade 
Acceptable 

Participants  Indicating 
This  as  Lowest  Grade 
Acceptable 

Actual  Grade  Earned  by  Participants 

A 

B 

C 

D 

E 

A 

78  (27%) 

46  (59%) 

23  (29%) 

7 (9%) 

2 (3%) 

- 

B 

161  (55%) 

17(11%) 

85  (53%) 

43  (27%) 

14  (8%) 

2(1%) 

C 

51  (18%) 

2 (4%) 

11  (21%) 

25  (49%) 

8(16%) 

5(10%) 

N = 290. 


All  participants  wanted  to  earn  a grade  higher  than  C.  In  total,  1 56  students,  or  54%,  earned  the  grade  they  indicated  would 
be  the  lowest  grade  acceptable,  104  students,  or  36%,  earned  a grade  lower  than  that  which  was  acceptable,  and  30  students,  or 
10%,  earned  a grade  higher  than  their  lowest  grade  acceptable. 

Number  of  weekly  study  hours 

Participants  were  also  asked  how  many  hours  a week  they  study  for  this  course.  They  were  given  five  possible  choices  to 
select  from;  0 hours,  1-3  hours,  4-6  hours,  7-8  hours,  and  more  than  9 hours.  Response  totals  and  percentages  are  provided  in 
Table  5. 


Table  5 Reported  Number  of  Study  Hours  per  Week  Dedicated  to  the  Computer  Literacy  Course 


Hours  per  week 

Total  Responses 

0 

40  (14%) 

1 -3 

206  (71%) 

4-6 

37(13%) 

7-8 

5 (2%) 

9 or  more 

2(1%) 

N = 290 


In  general,  206  students,  or  71%,  indicated  that  they  dedicated  between  one  to  three  hours  per  week  studying  for  this  course 
and  37  students,  or  13%,  indicated  that  they  dedicated  four  to  six  hours  per  week  studying  for  this  course.  Forty  students,  or  14%, 
responded  that  they  did  not  study  at  all  for  this  course. 

Reasons  for  enrolling  the  Computer  Literacy  course 

The  last  question  in  this  section  of  the  survey  asked  students  about  the  reasons  they  had  for  taking  this  course.  They  were 
asked  to  respond  yes  or  no  to  a series  of  nine  items  aimed  at  discovering  their  purpose.  They  were  to  indicate  all  reasons  that 
were  applicable  to  them.  Response  totals  and  percentages  are  shown  in  Table  6. 


Table  6 Reasons  Students  Enrolled  in  the  Computer  Literacy  Course 


Reason 

Total  Responses 

Will  be  useful  to  me  in  other  courses 

248  (85%) 

Required  for  academic  major 

233  (80%) 

Will  improve  my  academic  skills 

211  (73%) 

Fit  into  my  schedule 

212(73%) 

Will  improve  my  career  prospects 

205  (70%) 

Content  seems  interesting 

191  (66%) 

Was  recommended  by  my  advisor 

170  (58%) 

Is  an  easy  elective 

106  (36%) 

Was  recommended  by  a friend 

71  (24%) 

The  responses  indicated  that  most  students,  248  or  85%,  thought  this  course  would  be  helpful  to  them  in  other  courses,  and 
for  233  students,  or  80%,  this  course  was  a requirement  of  their  academic  major.  Many  students,  211,  or  73%,  felt  the  course 
would  improve  their  academic  skills  and  205  students,  or  70%,  felt  the  course  would  improve  their  career  prospects.  One- 
hundred  ninety-one  students,  or  66%,  took  the  course  because  they  thought  the  content  seemed  interesting. 

Discussion 

The  purpose  of  this  study  was  to  determine  the  relationship  among  student  self-reports  of  their  academic  goal  orientation, 
self-efficacy,  self-regulated  learning  strategy  use  and  their  academic  performance  in  a Computer  Literacy  course  as  indicated  by 
course  grade.  Also  investigated  were  students'  most  preferred  and  utilized  study  techniques  and  the  methods  they  used  to  monitor 
their  learning  in  this  course. 

The  results  of  this  study  portray  a complex  combination  of  the  motivation,  and  learning  strategies  utilized  by  college  students 
in  a Computer  Literacy  course.  Overall,  the  results  appear  to  indicate  that  these  students  held  both  extrinsic  and  intrinsic  goal 
orientations  at  the  same  time.  For  many  students,  earning  a high  grade  was  very  important  to  them,  and  many  took  the  course 
because  they  thought  the  content  would  be  valuable  and  interesting.  These  students  also  reported  that  they  have  both  high  self- 
efficacy  and  low  test-anxiety,  they  utilize  elaboration  learning  strategies  and  prefer  to  not  study  with  their  peers.  Approximately 
half  of  the  students  earned  the  grade  that  they  indicated  was  the  lowest  grade  acceptable  to  them,  but  about  one-third  of  the 
students  earned  a poorer  grade  than  the  lowest  grade  acceptable  to  them.  The  majority  of  students  reported  that  they  spent 
between  one  aid  three  hours  per  week  studying  for  this  course,  however,  many  indicated  that  more  discipline  and  a study 
schedule  would  help  them  become  better  learners 

In  terms  of  achievement  goals,  findings  in  this  study  indicated  that  extrinsic  goal  orientation  was  positively  related  to  course 
grade.  This  finding  is  similar  to  results  from  a previous  study  focusing  on  college  students’  goal  orientations  and  use  of  self- 
regulation strategies  in  the  classroom.  In  their  study,  Pintrich  & Garcia  (1991)  found  that  having  an  extrinsic  goal  orientation. 
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such  as  a commitment  to  earning  high  grades,  may  actually  help  students  focus  not  only  on  learning  the  course  material,  but  also 
may  assist  them  in  maintaining  their  self-efficacy. 

In  the  current  study,  self-efficacy  was  also  positively  related  to  course  grade.  From  this  finding,  it  appears  that  students  had  a 
combination  of  extrinsic  goal  orientation  and  high  self-efficacy,  which  may  have  caused  them  to  persist  in  their  learning  of  the 
course  material  to  achieve  their  desired  academic  goal.  When  individuals  with  either  extrinsic  or  intrinsic  goal  orientation  have 
high  self-efficacy  beliefs  their  behavior  is  quite  similar  (Miller,  Behrens,  Greene,  & Newman,  1993),  Individuals  with  high  self- 
efficacy  are  confident  in  their  ability  to  succeed  at  a task,  tend  to  accept  the  challenge  of  the  task  and  will  persevere  in  an  effort  to 
successfully  complete  it. 

Self-efficacy  beliefs  also  influence  the  amount  of  stress  and  anxiety  individuals  experience  as  they  engage  in  a task  and  the 
level  of  accomplishment  they  realize.  In  the  current  study,  students  reported  high  self-efficacy  beliefs,  therefore,  it  is  not 
surprising  that  they  also  indicated  they  had  low-test  anxiety.  Individuals  with  a strong  sense  of  competence  approach  difficult 
tasks  as  challenges  to  be  mastered  rather  than  dangers  to  be  avoided.  Conversely,  individuals  with  low  self-efficacy  beliefs  may 
feel  that  things  are  tougher  than  they  really  are,  a belief  that  fosters  stress,  depression  and  a narrow  vision  of  how  to  solve  a 
problem  (Pajares,  1997). 

Student  selection  of  learning  strategies  used  to  accomplish  a task  is  also  dependent  on  both  beliefs  of  goal  orientation  and 
self-efficacy.  Results  from  previous  studies  have  indicated  that  extrinsically  motivated  students  tend  to  use  short-term  and 
surface-level  processing  strategies,  such  as  memorizing  and  rehearsing  strategies  (Miller,  et,  al,,  1993),  Additionally,  individuals 
with  performance  goals  are  less  concerned  with  learning,  and  the  use  of  learning  strategies  requires  effort  which  implies  a lack  of 
ability,  an  inference  that  performance  oriented  students  do  not  wish  to  make  (Ames  & Archer,  1988), 

In  the  current  study,  the  learning  strategy  of  elaboration  was  positively  related  to  course  grade.  Elaboration  learning  strategies 
are  generally  considered  to  be  utilized  by  students  with  intrinsic  goal  orientations.  For  students  that  are  intrinsically  motivated, 
their  goal  is  to  learn  the  course  material,  therefore,  learning  strategies  such  as  elaboration  are  often  used  to  enable  them  to  make 
new  information  more  meaningful  by  connecting  it  with  prior  knowledge  (Weinstein  & Mayer,  1986),  However,  results  from 
research  by  Pintrich  and  Garcia  (1991)  found  that  students  with  either  intrinsic  or  extrinsic  goal  orientations  both  reported 
substantial  use  of  cognitive  and  self-regulated  learning  strategies,  such  as  elaboration  and  organization.  It  appears  that  a high 
level  of  concern  for  grades  may  actually  lead  to  better  cognitive  engagement. 

From  earlier  motivational  research,  it  was  generally  believed  that  students  have  either  a mastery  or  performance  goal  (Meece 
& Holt,  1993).  Yet  recent  research  on  the  relationship  between  mastery  and  performance  goals  has  indicated  that  these  two  types 
of  goals  are  independent  of  one  another  rather  than  opposite  of  one  another.  This  independence  means  that  it  is  possible,  and 
perhaps  even  likely,  for  students  to  have  both  mastery  and  performance  goals  at  the  same  time  (Pintrich  & Garcia,  1991), 

Interestingly,  results  of  the  current  study  also  hdicated  that  students  valued  the  information  they  were  learning  in  the 
Computer  Literacy  course.  Though  intrinsic  goal  orientation  was  not  significantly  related  to  course  grade,  student  responses  to 
the  items  focusing  on  their  reasons  for  taking  the  course  indicated  that  a majority  of  students  enrolled  because  they  thought  the 
course  material  was  interesting.  Students  that  enroll  in  courses  because  they  find  the  content  interesting  or  enjoyable  are 
intrinsically  motivated.  They  are  taking  the  course  for  its  inherent  satisfaction  rather  than  because  it  may  lead  to  a separable 
outcome  (Ryan  & Deci,  2000).  Based  on  these  findings  it  appears  the  students  in  the  Computer  Literacy  course  had  a 
combination  of  intrinsic  and  extrinsic  goal  orientations. 

Though  results  of  the  current  study  appear  to  be  consistent  with  the  finding  of  Pintrich  & Garcia  (1991),  it  is  also  important 
to  consider  the  course  context  in  which  the  learning  strategies  were  being  used.  In  their  responses  on  the  Elaboration  sub-scale 
items  on  the  MSLQ  portion  of  the  survey,  students  in  the  current  study  indicated  that  they  used  information  from  lectures  and 
readings  to  accomplish  learning  tasks  and  activities.  The  Computer  Literacy  course  was  comprised  of  a lecture  and  lab  session, 
therefore,  it  is  not  surprising  that  students  would  use  information  learned  in  lecture  to  accomplish  lab  activities.  These  results 
seem  to  suggest  that  good  strategy  use  in  the  classroom  may  be  conditional  and  contextual  to  the  learning  situation,  and  may  not 
be  solely  dependent  on  student  goal  orientation  and  self-efficacy  beliefs.  Future  research  may  be  necessary  to  further  clarify  these 
results. 

Another  characteristic  of  extrinsically  motivated  students  is  their  desire  to  demonstrate  their  ability,  or  hide  their  perceived 
lack  of  ability.  The  fear  of  appearing  incompetent  can  cause  students  to  use  behaviors  that  they  feel  might  protect  their  sense  of 
self-worth  (Archer,  1994).  Results  of  the  current  study  indicated  that  peer  learning  was  negatively  related  to  course  grade.  This 
result  is  not  surprising  since  students  held  high  extrinsic  goal  orientations,  A characteristic  of  many  extrinsically  motivated 
students  is  demonstrating  competence  to  others.  Because  of  this,  perhaps  these  students  did  not  want  to  appear  incapable  in  front 
of  others.  When  students  pursue  goals  that  concern  maintaining  a certain  image  in  front  of  others,  they  interpret  the  need  for  help 
as  a threat  to  self-worth.  (Archer,  1994). 

An  interesting  finding  from  the  current  study  indicated  that  students  with  low  self-efficacy  beliefs  that  studied  with 
classmates  were  likely  to  have  higher  grades.  Comparatively,  students  with  high  self-efficacy  that  studied  with  classmates  were 
likely  to  have  lower  grades.  These  results  appear  to  be  in  contrast  to  previous  research  focusing  on  the  effects  of  goal  orientation 
and  help  seeking.  Results  Irom  previous  studies,  found  that  students  who  had  low  self-efficacy  beliefs  were  more  likely  to  feel 
threatened  when  asking  their  peers  for  help  and  were  more  likely  to  avoid  those  types  of  activities.  These  students  feel  that  their 
need  for  help  indicates  that  they  lack  ability,  therefore,  they  are  less  likely  to  seek  assistance  (Ryan  & Pintrich  ,1997;  Ryan, 
Gheen,  & Midgley,  1998). 

Another  interesting  finding  of  the  present  study  also  indicated  that  students  with  high  extrinsic  goal  orientation  that  studied 
with  their  classmates  were  likely  to  earn  lower  grades.  Those  students  with  low  extrinsic  goal  orientation  that  studied  with  their 
classmates  were  likely  to  earn  higher  grades. 


Also  investigated  in  this  study  were  the  reported  study  techniques  utilized  by  students  in  the  Computer  Literacy  course,  and 
what  they  felt  would  help  them  become  more  successful  in  their  learning.  It  is  interesting  to  find  that  approximately  half  of  the 
students  in  this  study  indicated  that  they  study  the  same  way  for  this  course  as  they  do  their  other  courses,  and  half  stated  that 
they  study  differently.  The  majority  of  students  indicated  that  the  study  strategies  they  used  in  the  Computer  Literacy  course  were 
reading  their  textbook  and  lecture  notes,  then  applying  that  information  to  computer  activities.  Most  students  also  monitored  their 
understanding  of  course  material  while  trying  to  apply  this  information  to  the  computer  activities. 

Results  from  previous  research  have  indicated  that  use  of  various  learning  strategies  may  be  conditional  and  contextualized. 
Students,  therefore,  need  to  understand  the  situations  when  certain  learning  strategies  may  be  more  or  less  effective  (Pintrich  & 
Garcia,  1994),  When  encountering  a learning  situation  for  the  first  time,  students  may  not  know  how  to  think  within  that 
discipline.  Pintrich  (1995)  suggests  that  in  order  for  students  to  become  successful  self-regulated  learners,  teachers  should  help 
students  become  aware  of  how  to  think,  learn,  and  reason  within  the  particular  discipline.  Perhaps  this  would  be  beneficial  for 
students  in  the  Computer  Literacy  courses. 

For  many  students  in  this  course,  it  appears  that  learning  techniques  may  need  to  be  improved  in  order  to  promote  more 
successful  learning.  There  may  possibly  be  variations  to  study  strategies  that  could  enhance  learning  in  these  types  of  courses. 
The  majority  of  students  indicated  that  they  felt  responsible  for  their  success  in  learning,  however,  only  half  of  them  earned  the 
grade  that  was  the  lowest  grade  acceptable  to  them,  with  many  others  earning  a poorer  grade.  Students  also  indicated  that  they 
believed  they  could  be  more  successful  if  they  had  a study  schedule  and  more  discipline.  It  may  be  beneficial,  therefore,  to 
provide  students  with  appropriate  strategies  for  learning  the  course  material  and  assisting  them  in  establishing  suitable  study 
schedules. 

The  results  of  this  study  highlight  the  motivation  and  learning  strategies  most  related  to  course  grade  in  a Computer  Literacy 
course.  This  study  not  only  provides  information  on  students’  learning  goals  and  their  use  of  self-regulated  learning  strategies, 
but  it  also  gives  insight  to  how  undergraduate  students  view  learning  and  the  methodologies  they  use  to  study.  The  information 
provided  from  this  study  may  assist  teachers  of  undergraduate  students  in  Computer  Literacy  courses,  as  well  as  other  computer 
based  courses. 

Results  of  this  investigation  suggest  several  avenues  for  future  research.  First,  measurements  in  the  current  study  were 
gathered  at  only  one  point  in  time.  It  may  be  useful  to  replicate  this  study  on  a longitudinal  basis  to  further  examine  the 
relationships  between  the  motivation  and  learning  strategies  that  prove  to  be  most  effective.  Second,  it  would  also  be  beneficial 
to  look  further  into  the  interactions  of  the  significant  variables  to  determine  the  interplay  and  influence  they  may  have  on  student 
learning  and  performance.  Third,  in  this  study,  student  self-report  measures  were  used.  It  would  be  informative  to  use  other  data 
gathering  measures  such  as  interviews  and  observations.  Finally,  additional  research  may  be  needed  to  determine  how  classroom 
context  and  structures  impact  the  motivational  and  learning  strategies  used  by  students. 
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Abstract 

Does  the  introduction  of  hypertext  and  hypermedia  into  college  instruction  impact  students ' ability  to  regulate  their  own 
learning  processes?  The  metacognition  “calibration  of  comprehension  ” research  paradigm  is  used  to  investigate  this  question. 
Interviews  with  experimental  subjects  provide  additional  insights  into  the  study  process. 

Introduction 

To  be  academically  successful,  college  students  must  effectively  allocate  study  effort  among  multiple  courses  based  on  the 
requirements  of  each  course.  An  important  study  self-regulation  skill  is  the  ability  to  answer  the  pragmatic  question  “Do  I know 
this  subject  matter  well  enough  to  take  the  test?’’  College  students  have  developed  this  skill  - to  varying  degrees  - through  years 
of  studying  paper-based  textbooks.  In  today’s  college  environment,  Web  and  CD-ROM  instructional  materials  require  students 
to  study  materials  displayed  on  a computer  screen  and  organized  in  a nonlinear  structure.  Do  these  nonlinear  hypertext  and 
hypermedia  instructional  materials  impact  students’  ability  to  accurately  assess  their  own  test  readiness  and  thus  to  effectively 
regulate  their  study  processes?  If  so,  then  the  promotion  of  hypermedia  instructional  materials  in  the  college  environment  may 
create  unintended  stumbling  blocks  to  academic  success. 

Literature  Review 

The  research  literature  to  date  does  not  address  this  issue.  Although  numerous  studies  have  examined  the  use  of  hypertext  and 
hypermedia  as  instructional  media,  there  is  no  body  of  literature  addressing  impacts  of  hypermedia  on  self-regulation  of  the 
learning  process.  Some  studies  do,  however,  suggest  there  may  be  cause  for  concern. 

Conklin  (1987)  noted  two  fundamental  issues  with  hypermedia:  disorientation  and  cognitive  overhead.  Hypermedia  reading 
requires  much  greater  mental  effort  in  managing  the  reading  process  (compared  to  the  simple  page -turning  of  print 
environments);  this  mental  activity  can  divert  mental  resources  from  the  intellectual  activity  of  reading  and  learning  (Dede  & 
Palumbo,  1991).  Hypermedia  users  may  lack  the  navigational  skills  needed  to  be  successful  in  hypertext -based  learning  (Lawless 
& Kulikowich,  1996;  Schroeder  & Grabowski,  1995).  Domain  knowledge  of  the  individual  reader  is  a key  determinant  of  a 
reader’s  ability  to  successful  learning  in  the  hypertext  environment  (Beishuizen  et  al.,  1994;  Lawless  & Kulikowich,  1996). 
Reading  text  from  the  computer  screen  generally  requires  more  time  (Belmore,  1985;  Gould,  Alfaro,  Finn,  Haupt,  & Minuto, 
1987;  Grice,  Ridgeway,  & See,  1991;  Kearsley,  1988).  Some  experiments  have  found  poorer  comprehension  with  computer- 
based  text  (Belmore,  1985;  Feldmann  & Fish,  1988;  Fish  & Feldmann,  1987;  Reinking  & Schreiner,  1985). 

Although  silent  on  the  question  of  hypermedia  impacts,  reading  and  cognition  researchers  have  investigated  learning  self- 
regulation. During  the  past  15  years,  a number  of  research  studies  have  explored  learning  self-regulation  using  a research 
paradigm  known  as  “calibration  of  comprehension”  (Lin  & Zabrucky,  1998).  In  the  current  research,  the  calibration  of 
comprehension  paradigm  has  been  adapted  to  investigate  study  self-regulation  in  a hypermedia  learning  environment. 

In  a typical  calibration  experiment,  subjects  read  expository  text  and  then  are  asked  to  predict  their  performance  on  a simple 
objective  test  over  the  materials  read.  Actual  test  performance  is  compared  to  self-assessed  predicted  performance  using  a 
correlation  coefficient.  Subjects  able  to  accurately  predict  their  performance  are  considered  “highly  calibrated”  regardless  of  their 
performance  on  the  test.  Likewise,  subjects  who  do  not  predict  their  performance  accurately  are  considered  “poorly  calibrated.” 
Calibration  of  comprehension  research  has  shown  correlation  between  predicted  performance  and  actual  performance  ranging 
from  virtually  zero  to  greater  than  r = .60  (Lin  & Zabrucky,  1998).  Calibration  research  has  also  identified  several  ways  in  which 
research  designs  can  maximize  the  probability  of  detecting  calibration  if  it  is  indeed  taking  place.  These  guidelines  were 
followed  in  the  current  research  project: 

1.  Since  posttest  performance  predictions  may  be  influenced  by  subjects’  prior  knowledge  of  the  topic  (Glenberg  & Epstein, 
1987)  and  by  subjects’  interest  in  the  topic  (Glenberg  et  al.,  1982;  Lin  et  al.,  1997),  some  method  of  assessing  these  subject 
attributes  may  be  useful  in  the  study. 

2.  Text  should  be  of  moderate  difficulty  for  the  research  subject  population  (Weaver  & Bryant,  1 995;  Weaver  et  al.,  1 995). 

3.  Posttests  should  have  more  than  one  question  per  text  segment.  Weaver  (1990)  found  four  questions  per  text  to  produce 
significantly  more  accurate  indications  of  calibration  than  a single  question  per  text. 

4.  Since  subjects  are  better  able  to  inventory  their  understanding  and  retention  of  facts  than  they  are  their  ability  to  recognize 
logical  inferences,  posttest  questions  should  deal  with  the  recognition  of  facts  and  ideas  (Glenberg  et  al.,  1987;  Pressley  et 
al.,  1987). 
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Experiment 

Presentation  technology  (paper  or  computer)  and  content  structure  (linear  or  nonlinear)  were  independent  variables  in  this 
2x2  factorial  design  quantitative  study.  As  illustrated  in  Figure  1,  instructional  materials  were  differentially  formatted  to  create 
the  four  experimental  treatments.  Content  was  identical  in  each  treatment  and  consisted  of  eight  topics.  The  instructional 
materials  (and  corresponding  test  questions)  were  adapted  from  a well-established  curriculum  and  test  bank. 


Presentation  Technology 

Paper 

(p) 

Computer 

(Q 

Content  Structure 

Nonlinear 

(N) 

NP  Environment 

Nonlinear  text  in  paper  form  (printed 
nonlinear  WWW  site). 

NC  Environment 

Nonlinear  text  on  computer  (hypermedia). 

Linear 

(L) 

LP  Environment 

Linear  text  in  paper  form  (book). 

LC  Environment 

Linear  text  on  computer. 

Figure  1.  Experimental  treatments  created  by  varying  presentation  technology  and  content  structure. 

Experimental  subjects  (undergraduate  students  at  a small  private  college)  were  randomly  assigned  to  the  four  treatment 
groups.  Each  treatment  group  had  17  subjects.  After  studying  the  treatment  instructional  materials,  subjects  predicted  test 
performance  on  each  of  the  eight  topics.  Upon  completion  of  an  objective  posttest,  a comprehension  calibration  coefficient  (the 
dependent  variable)  was  calculated  for  each  subject  by  correlating  the  eight  performance  predictions  with  the  actual  test  scores  on 
the  eight  topics  using  the  Pearson  product -moment  correlation. 

As  noted  above,  the  research  literature  suggests  that  subject  interest  in  the  study  topic,  subject  expertise  in  the  topic,  and 
subject  motivation  to  perform  in  the  posttest  may  all  be  covariates  with  the  calibration  coefficient.  Length  of  study  time  might 
also  reasonably  be  a covariate.  Based  on  researcher-recorded  reading  times  and  subject  self-reported  measures  of  interest, 
expertise,  and  motivation,  no  significant  covariance  was  found. 

To  assess  the  impact  of  presentation  technology  and  content  structure  on  subject  calibration  of  comprehension,  data  generated 
in  the  experiments  were  subjected  to  hypothesis  testing: 

1 . HqI  : There  was  no  significant  difference  between  the  calibration  coefficients  for  the  computer  technology  treatment  and  the 
paper  technology  treatment.  (Rejection  of  this  hypothesis  would  mean  the  presentation  technology  influences  calibration.) 

2.  Hq2:  There  was  no  significant  difference  between  the  calibration  coefficients  for  linear  structure  treatment  and  the  nonlinear 
structure  treatment.  (Rejection  of  this  hypothesis  would  mean  the  linear/nonlinear  Sructure  of  the  instructional  materials 
influences  calibration.) 

3.  Hq3:  There  was  no  significant  interaction  effect  between  the  technology  and  structure  treatments  as  measured  by  the 
calibration  coefficient.  (If  hypothesis  Hq3  were  to  be  rejected,  then  three  more  hypotheses  would  be  tested.) 

4.  Ho3a:  There  was  no  significant  difference  between  the  calibration  coefficients  for  the  linear  paper  treatment  and  the 
nonlinear  computer  treatment.  (This  hypothesis  compared  a typical  book  format  to  nonlinear  computer  hypermedia.) 

5.  Ho3b:  There  was  no  significant  difference  between  the  calibration  coefficients  for  the  nonlinear  computer  treatment  and  the 
nonlinear  paper  treatment.  (This  hypothesis  compared  calibration  when  reading  from  a website  to  calibration  when  reading 
from  a printed  copy  of  the  website.) 

6.  Hq3c:  There  was  no  significant  difference  between  the  calibration  coefficients  for  the  nonlinear  computer  treatment  and  the 
linear  computer  treatment.  (This  hypothesis  compared  two  different  design  approaches  for  hypermedia.) 

Experimental  Results 

The  dependent  variable,  referred  to  as  the  “calibration  coefficient”,  was  a Pearson  Product -Moment  Correlation  calculated 
between  subjects’  self-predicted  performance  on  eight  fallacy  topics  with  their  actual  posttest  scores  on  those  topics.  The  mean 
value  of  the  calibration  coefficient  is  0.09  which  is  significantly  greater  than  zero,  t(67)  - 1.95,  £.<0.05.  The  median  value  of  the 
calibration  coefficient  variable  is  0.15.  Figure  2 displays  the  distribution  of  the  calibration  coefficient  for  the  experimental 
subjects. 
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Figure  2.  Distribution  of  dependent  variable. 

An  analysis  of  variance  was  conducted  to  determine  if  treatments  or  treatment  interactions  affected  subjects’  ability  to  predict 
test  performance  and  thus  regulate  their  study  processes.  As  noted  in  Table  1,  treatment  effects  were  not  statistically  significant 
nor  were  there  statistically  significant  interaction  effects.  Thus,  the  first  three  hypotheses  are  not  rejected  and  the  second  three 
hypotheses  are  not  tested. 

Table  I,  ANOVA  with  calibration  coefficient  as  dependent  variable 


Source 

SS 

df 

MS 

F 

Structure 

.176 

1 

.176 

1.251 

Technology 

.020 

1 

.020 

.146 

2-way  Interaction 

.050 

1 

.050 

.354 

Residual 

9.005 

64 

.141 

Post  hoc  power  analysis  with  p<0.05  indicates  a power  level  less  than  0.20 

Several  aspects  of  the  experimental  data  warranted  further  investigation: 

1.  The  mean  of  the  calibration  coefficients,  while  statistically  greater  than  zero,  was  disappointingly  low.  The  0.9  mean  is 
reminiscent  of  the  earliest  calibration  research  and  was  expected  to  be  higher  since  the  research  meticulously  followed 
guidelines  developed  in  the  calibration  research  literature. 

2.  The  distribution  of  the  calibration  coefficient  shown  in  Figure  2 is  disturbing.  It  has  the  general  appearance  of  a random 
normal  distribution  centered  on  zero  and  shows  a large  number  of  “negatively  calibrated”  subjects.  Negative  calibration 
coefficients  imply  that  subjects  consistently  score  poorly  on  topics  they  think  they  know  well  and  vice  versa.  Negative 
correlation  is  not  addressed  in  the  calibration  of  comprehension  literature  and  has  no  obvious  ties  to  real  world  learning 
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experiences.  The  negative  correlation  found  in  the  calibration  coefficient  distribution  suggests  a major  random  process 
underlying  the  calibration  coefficient. 

3.  The  test  questions  (along  with  the  instructional  texts)  have  an  extensive  track  record  at  the  host  institution.  The  grade 
distributions  typical  for  the  posttest  questions  were  well  known.  Posttest  scores  were  lower  and  more  disperse  than  expected. 

4.  Reliability  analysis  and  item  analysis  indicated  problems  with  the  test  questions.  Since  extensive  past  use  of  the 
instructional  materials  and  test  questions  suggest  these  materials  are  effective  and  appropriate  together,  then  low  reliability 
and  item  analysis  scores  indicate  some  other  problem  in  the  study  methodology. 

Taken  together,  these  observations  pointed  to  extensive  random  guessing  by  subjects  during  the  posttest  phase  of  the  experiment. 
The  randomizing  influence  of  posttest  guessing  would  produce  the  symptoms  noted  above. 

Interviews 

To  obtain  further  insights  into  the  experimental  results,  two  subjects  from  each  of  the  four  treatment  groups  were  selected  for 
interviews.  For  each  treatment  group,  one  subject  had  a high  posttest  score  and  a relatively  large  positive  calibration  coefficient. 
The  second  subject  for  each  treatment  group  had  a high  posttest  score  and  a relatively  large  negative  calibration  coefficient. 
(Only  subjects  with  high  posttest  scores  were  selected  to  make  sure  the  interviewed  subjects  were  actively  engaged  in  the 
experiment’s  learning  task.) 

A thirteen-question  telephone  interview  was  conducted  with  each  of  the  eight  identified  research  subjects.  The  questions 
addressed  several  issues  arising  from  analysis  of  the  experimental  data: 

1.  Effort.  Seven  of  the  eight  interviewed  subjects  admitted  they  would  have  put  much  more  effort  into  studying  the 
instructional  materials  if  a course  grade  had  been  at  stake.  Six  of  the  eight  subjects  estimated  guessing  at  20%-40%  of  the 
posttest  questions;  one  estimated  guessing  at  60%  of  the  questions. 

2.  Stopping  criteria.  Interviewed  subjects  were  ask  to  describe  their  study  “stopping  criteria”  - how  they  decided  when  to  stop 
studying-  for  both  real  world  study  tasks  and  for  the  experimental  task.  Only  three  of  the  eight  interviewed  subjects  used 
the  same  criteria  in  the  experiment  as  they  typically  used  when  studying  for  college  courses. 

3.  Anti-calibration.  None  of  the  eight  interviewed  subjects  reported  study  self-regulation  difficulties  that  resemble  anti- 
calibration. 

Based  on  these  interviews,  it  is  clear  that  subjects  were  not  motivated  to  study  the  experimental  materials  in  the  same  way  or 
to  the  same  extent  they  study  actual  academic  materials.  It  can  not  be  assumed  then,  that  the  subjects  engaged  their  normal 
calibration  skills  either.  The  extensive  guessing  during  the  experimental  posttest  introduced  a large-scale  random  influence  to  the 
experimental  data.  Thus,  it  is  not  clear  to  what  extent  the  experimental  data  accurately  reflects  student  calibration  in  real  world 
settings. 

Other  Findings 

The  interviews  revealed  four  types  of  study  stopping  criteria: 

1.  Process  criteria.  Several  subjects  described  study  “rituals”  involving  reading/re-reading  practices,  note-taking,  or  other 
study  strategies.  Process-oriented  students  tended  to  stop  studying  once  the  study  process  was  complete. 

2.  Feel  good  criteria.  Some  subjects  reported  studying  until  they  felt  they  knew  the  materials. 

3.  Feel  bad  criteria.  Other  subjects  reported  studying  until  they  felt  like  they  weren’t  getting  anything  more  out  of  the 
studying. 

4.  Time  criteria.  Some  subjects  reported  studying  until  they  ran  out  of  time  and  the  test  was  given. 

Subjects  reported  a variety  of  ergonomic  and  interface  concerns.  Eye  strain  and  navigational  confusion  were  mentioned  by 
subjects  reading  from  the  computer  screen.  One  subject  noted  her  “body  got  bored”  reading  the  hypertext  - she  found  the 
instructional  material  interesting,  but  became  physically  restless  having  to  sit  in  one  position  in  front  of  the  computer  screen. 
Reading  from  a book  would  allow  her  to  change  physical  positions  and  read  for  longer  periods  of  time.  Subjects  noted  they  were 
unable  to  highlight  text  on  the  computer  like  they  did  in  textbooks.  One  subject  who  visualized  the  book  during  testing  found  he 
could  not  use  the  same  memory  technique  when  reading  from  the  computer.  Two  subjects  suggested  hypermedia  would  be  an 
excellent  tool  for  reviewing  materials  already  read. 

Figure  3 presents  one  of  the  most  interesting  observations  from  the  experimental  data.  The  scattergram  shows  individual 
subjects  plotted  by  posttest  score  performance  (x-axis)  and  predicted  performance  (y-axis).  Regression  lines  for  each  of  the  four 
treatment  groups  are  shown,  along  with  the  R^  for  each  regression  line. 

In  examining  Figure  3,  it  is  obvious  that  the  LP  treatment  (the  treatment  representing  traditional  studying  from  a textbook)  is 
noticeably  different  from  the  other  treatments.  The  regression  line  and  R^  for  the  LP  treatment  suggests  that  students  who  scored 
better  on  the  posttest  were  also  better  able  to  predict  their  performance  than  students  who  did  not  mastered  the  material.  This  is 
intuitively  appealing,  regardless  of  whether  this  indicates  that  better  calibrated  students  learn  more  or  that  students  who  have 
learned  more  are  better  able  to  assess  their  knowledge.  Note,  however,  that  the  other  treatments  show  no  such  relationship.  Since 
the  discussion  above  has  already  established  that  the  artificial  experimental  setting  heavily  influenced  the  research  data,  one 
cannot  presume  Figure  3 necessarily  represents  impacts  of  the  treatments  in  real  world  study  situations.  However,  these  results 
do  suggest  further  study  is  warranted. 
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Summary 

The  present  study  is  a pioneering  foray  into  an  unexplored  issue-the  calibration  of  comprehension  in  hypermedia 
environments.  In  particular,  this  study  was  designed  to  determine  if  hypermedia  instructional  materials  would  impact  college 
students’  ability  to  assess  their  own  readiness  for  testing. 

Two  basic  characteristics  of  hypermedia-content  structure  and  presentation  technology  -were  used  as  variables  to  define  a 
2x2  experimental  design.  The  content  structure  variable  could  be  either  linear  (L)  or  nonlinear  (N);  the  presentation  technology 
variable  could  be  either  paper-based  (P)  or  computer-based  (C).  The  variables  define  four  treatment  categories;  linear  paper- 
based  materials  (LP),  linear  computer-based  materials  (LC),  nonlinear  paper-based  materials  (NP),  and  nonlinear  computer-based 
materials  (NC). 

After  reading  the  instructional  materials,  but  before  seeing  the  posttest  questions,  subjects  were  asked  to  predict  their  test 
performance  on  questions  for  each  of  the  eight  topics  contained  in  the  instructional  materials.  The  posttest  generated  eight 
different  scores— one  for  each  of  eight  topics  contained  in  the  experimental  materials.  For  each  subject,  a Pearson  Product- 
Moment  Correlation  (Pearson  r)  was  calculated  by  pairing  the  eight  posttest  scores  with  the  eight  score  predictions.  These 
Pearson  r values -called  calibration  coefficients-served  as  the  dependent  variable  in  the  experiment. 

Although  statistically  significant  calibration  was  detected,  analysis  of  variance  found  no  statistically  significant  treatment  or 
interaction  effects.  Data  analysis  and  interviews  suggest  that  subjects  did  not  study  at  the  same  level  of  effort  or  use  the  same 
study  strategies  as  used  in  real-world  academic  test  preparation.  This  lack  of  preparation  resulted  in  greater  levels  of  guesswork 
during  the  posttest  and  thus  increased  randomness  in  the  experimental  data.  Since  students  may  not  have  used  the  same  study 
strategies  in  the  experiment  as  they  use  in  actual  academic  coursework,  it  is  not  certain  whether  the  findings  of  the  experiment 
can  be  applied  to  actual  college  coursework  environments.  However,  additional  statistical  analysis  suggests  that  learning  with 
unfamiliar  media  may  impact  calibration  of  comprehension  for  some  students  and  that  further  investigation  is  needed.  Future 
research  seeking  to  address  calibration  in  academic  settings  should  be  incorporated  into  academic  coursework  where  possible. 
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Figure  3.  Scattergram  of  calibration  coefficients  and  posttest  scores  with  regression  lines  by  treatment. 
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Background 

The  pedagogical  shifts  embodied  in  a series  of  standards  published  by  the  National  Council  of  Teachers  of  Mathematics 
(NCTM)  emphasize  a departure  from  the  teaching  and  learning  approaches  typical  to  American  classrooms  (NCTM,  1989,  1991, 
1995,  2000).  National  and  worldwide  assessments  such  as  the  TIMSS  report  (Cochran,  1999  and  NAEP  (U.S.  Department  of 
Education,  2001)  confirm  that  there  is  a need  to  carefully  examine  and  improve  our  educational  practices  in  order  to  remain  a 
leader  in  worldwide  learning  and  in  worldwide  industry.  In  order  to  meet  the  needs  of  the  educational  system,  we  need  to  rethink 
the  role  and  format  of  teacher  professional  development  experiences  (e.g.,  NCSMT,  2000;  NPEAT,  2000;  National  Commission 
on  Teaching  & America’s  Future,  1996;  Renyi,  1996;  Sparks,  1999). 

Several  leaders  in  professional  development  and  mathematics  have  clearly  defined  a working  plan  for  improving  mathematics 
teachers’  content  knowledge  and  pedagogical  content  knowledge,  as  these  two  factors  seem  to  be  critical  factors  in  student 
learning.  Calls  for  professional  development  that  occurs  over  a long  period  of  time,  that  emphasizes  teacher  thinking  and 
development  of  reflective  dispositions,  and  that  pushes  teachers  to  learn  more  in  their  content  areas  have  become  pervasive  in  the 
professional  development  literature  (e.g..  Ball,  1994;  Hawley  & Valli,  1999;  Krajcik,  Blumenfeld,  Marx,  & Soloway,  1994).  As 
pointed  out  by  Kilpatrick,  Swafford,  and  Findell  (2001): 

Teachers’  professional  development  should  be  high  quality,  sustained,  and  systematically  designed  and  deployed  to  help 

all  students  develop  mathematical  proficiency.  Schools  should  support,  as  a central  part  of  teachers’  work,  engagement  in 
sustained  efforts  to  improve  their  mathematics  instruction.  This  support  requires  the  provision  of  time  and  resources  (p.  12).  This 
statement  is  certainly  a call  for  professional  development  experiences  that  depart  from  the  “make  and  take”  model  that  is 
commonly  associated  with  teacher  workshops. 

Based  on  the  growing  need  for  constructivist,  learner-centered  environments  (McCombs  & Whisler,  1997),  National 
Partnership  for  Excellence  and  Accountability  in  Teaching  (NPEAT)  has  “revisioned”  what  professional  development  should 
look  like  and  be  (NPEAT,  2000).  Based  on  current  research,  and  in  alignment  with  other  proposals  for  improving  professional 
development,  the  NPEAT  Research-Based  principles  provide  a guide  for  professional  development.  These  principles  include: 

• The  content  of  professional  development  (PD)  focuses  on  what  students  are  to  learn  and  how  to  address  the  different 
problems  students  may  have  in  learning  the  material. 

• Professional  development  should  involve  teachers  in  the  identification  of  what  they  need  to  learn  and  in  the  development 
of  the  learning  experiences  in  which  they  will  be  involved. 

• Most  professional  development  should  be  organized  around  collaborative  problem  solving. 

• Professional  development  should  be  continuous  and  on-going,  involving  follow-up  and  support  for  further  learning  — 
including  support  from  sources  external  to  the  school  that  can  provide  necessary  resources  and  new  perspectives. 

• Professional  development  should  provide  opportunities  to  gain  an  understanding  of  the  theory  underlying  the  knowledge 
and  skills  being  learned.  (NPEAT,  2000) 

In  short,  teachers  should  take  charge  of  their  learning,  be  provided  with  motivational  and  challenging  ways  to  learn,  and  should 
have  the  opportunity  to  decide  what  is  most  relevant  for  their  students  (Hawley  & Valli,  1999).  NPEAT  also  asserts  that  the  most 
successful  professional  development  occurs  in  a culture  of  change. 

InterMath 

One  attempt  at  creating  a professional  development  environment  that  aims  to  meet  these  goals  is  the  InterMath  project. 
InterMath  is  comprised  of  a 15-week  technology -rich  workshop,  a website  with  mathematical  investigations  and  tools  to  support 
learning,  an  ongoing  support  system  that  is  designed  to  provide  continued  support  for  teachers  who  have  participated  in  the 
workshop,  and  a number  of  other  tools  and  opportunities  to  allow  teachers  to  take  away  what  they  need  for  their  own  success. 
InterMath  was  created  to  help  address  a critical  problem  with  middle  school  mathematics  — many  of  the  teachers  are  not 
knowledgeable  enough  in  their  content  area  or  in  content  pedagogy  (SREB,  1998).  This  is  because  of  the  number  of  middle 
grades  teachers  who  are  either  teaching  out  of  field  or  who  have  a generalist  background.  For  many  subject  areas,  the  generalist 
degree  does  not  offer  a rich-enough  content  background  for  the  teachers  to  support  students  in  the  classrooms  being  called  for. 
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The  goals  of  InterMath  include  the  improvement  of  teachers’  mathematical  skills  and  knowledge  through  open-ended 
explorations;  an  understanding  and  ability  to  use  software  to  support  the  development  of  mathematical  thinking;  and  the  creation 
of  a community  of  teachers  who  support  each  other  in  implementing  the  explorations-based  approach  in  their  classroom.  In 
implementation,  there  is  considerable  room  for  teachers  to  choose  their  own  path  to  success  - they  select  which  problem(s)  they 
want  to  work  in  each  of  the  critical  content  areas;  they  select  the  approach  they  want  to  use  to  solve  the  problem;  and  ultimately, 
the  teachers  decide  the  depth  of  learning  they  take  from  the  class  by  choosing  to  explore  more  challenging  problems,  or  add 
extensions  to  the  problems. 

It  is  our  view  that  InterMath  provides  one  approach  to  learner-centered  professional  development.  We  provide  the  workshop 
across  a full  semester,  but  include  tools  to  allow  the  continuous  growth  of  an  online  community.  The  teachers  are  given 
considerable  freedom  to  choose  the  mathematics  that  is  most  relevant  to  their  situation,  they  are  provided  with  a number  of  tools 
from  which  to  select,  and  they  decide  what  is  important  in  each  problem  when  they  prepare  their  write-ups  of  their  work.  While 
many  of  the  teachers  who  come  to  InterMath  are  not  necessarily  seeking  a reform-based  approach  to  mathematics,  they  leave 
with  a deepened  understanding  of  the  aspects  of  the  NCTM  standards  that  help  define  a quality  mathematics  experience.  In  our 
interviews  with  teachers,  we  have  also  found  that  they  develop  their  understanding  of  learning  by  experiencing  this  approach. 

This  Study 

The  purpose  of  this  paper  is  to  examine  two  InterMath  pilot  workshops  and  how  they  were  able  to  meet  the  goals  of  learner- 
centered  professional  development.  We  focus  on  the  experiences  of  the  learners,  offer  insights  from  each  case  as  well  as  across 
cases,  and  provide  suggestions  to  improve  the  experience  for  later  learners.  The  data  reported  here  came  only  from  the  workshops 
and  interviews  with  the  teachers  and  the  instructors. 

These  studies  were  conducted  by  participant  observer  graduate  students  in  each  workshop.  The  observers  collected  field  notes 
and  helped  the  workshop  participants.  Another  observer  (Orrill)  visited  the  larger  workshop  three  times  during  the  course  as  well. 
In  those  visits,  the  goal  was  to  gain  a non-participant  view  of  the  learning  environment. 

Other  data  collected  for  this  study  include  tape-recorded  interviews  with  several  participants  (8  in  one  class,  4 in  the  other) 
and  both  instructors.  There  were  also  informal  interviews  conducted  half-way  through  the  course  in  the  larger  group.  Student 
work,  published  on  the  Web,  was  also  considered  as  it  provided  insight  into  the  teachers’  mathematical  thinking  and  technology 
skills.  A survey  that  asked  our  participants  to  discuss  the  importance  of  and  their  comfort  with  using  technology  in  mathematics 
was  also  evaluated. 

For  the  purposes  of  this  study,  three  weeks  of  fieldnotes  were  selected  from  each  class.  They  came  from  early  in  the  semester 
(week  2 or  3),  mid-semester  (week  6),  and  later  in  the  semester  (week  12-14).  All  students  and  instructor  interviews  were 
considered.  Only  the  webpages  for  the  interviewed  students  were  analyzed. 

The  analysis  process  involved  coding  and  sorting  data.  We  found  several  emergent  categories  that  came  up  repeatedly  and 
used  those  as  a framework  for  our  thinking.  Those  included:  Support,  Interaction,  Barriers,  Presentation,  and  Adoption.  Each  case 
is  briefly  discussed  below  with  a cross-case  analysis  following. 

Case  1 

Description 

One  of  the  two  InterMath  pilot  workshops  took  place  near  Atlanta,  GA.  Throughout  the  workshop,  approximately  26-28 
participants  were  enrolled.  These  participants  were  all  full-time  middle  school  teachers  in  the  school  district  and  also  University 
of  Georgia  graduate  students  seeking  a master’s  or  education  specialist  degree  in  middle  school  mathematics  education.  Even 
though  the  participants  were  all  certified  to  teach  mathematics,  some  were  currently  teaching  subjects  other  than  math.  The 
teachers  participated  in  the  InterMath  workshop  as  their  first  experience  in  a middle  school  program  cohort  established  between 
their  school  district  and  UGA’s  mathematics  education  department.  While  being  a member  of  the  cohort  was  by  choice  (and 
acceptance),  there  was  no  choice  of  course  selection  once  the  teachers  joined  the  cohort. 

The  workshop  instructor  was  a full-time  professor  in  the  mathematics  education  department  at  the  University  of  Georgia  and 
also  one  of  the  InterMath  developers.  There  were  two  mathematics  education  graduate  assistants  who  served  as  participant 
observers  during  the  workshop.  The  project  manager,  who  visited  the  class  three  times  during  the  semester  to  act  as  an  outside 
observer,  also  made  additional  observations. 

The  workshop  took  place  at  the  county’s  board  office  computer  lab.  There  were  approximately  29  computers  in  the  lab.  The 
computers  lined  three  walls  of  the  room  to  form  a U-shape.  In  the  middle  of  the  lab,  there  were  six  long  tables  that  faced  an 
overhead  projector  screen  in  the  front  of  the  room.  There  were  no  computers  at  these  middle  tables.  The  participants  themselves 
chose  to  sit  at  the  long  tables,  rather  than  at  the  computers  during  the  lecture  portion  of  each  class  period.  On  the  first  table,  the 
instructor  set  up  a laptop  computer  to  connect  to  the  projector.  He  faced  the  participants,  who  sat  in  the  latter  five  rows,  as  he 
taught.  No  students  ever  sat  in  the  front  row. 

The  class  met  weekly  in  the  evening.  During  the  first  hour  portion  of  each  class,  the  instructor  taught  in  a traditional  lecture- 
style  manner.  He  demonstrated  how  to  explore  InterMath  problems  using  software  such  as  Geometer’s  Sketchpad,  NuCalc,  and 
Excel.  The  instructor  asked  a few  questions  during  his  demonstrations,  but  there  was  little  student  involvement.  The  class  would 
then  shift  to  the  computers  to  work  on  problems,  write-ups,  and  webpages.  For  the  remainder  of  the  class,  the  participants 
explored  the  investigations  using  various  software  programs  and  wrote  up  their  findings  to  post  to  their  individual  web  pages. 
The  instructor  and  graduate  assistants  walked  around  the  room  to  assist  the  participants  with  technological  and  mathematical 
questions,  as  they  requested  help. 
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In  our  analysis  of  data,  we  found  the  following  trends  in  this  setting. 

• Over-reliance  on  the  instructor 

The  participants  seemed  to  perceive  the  instructor  and  graduate  assistants  as  experts.  They  relied  on  the  instructor,  rather 
than  each  other  for  technological  and  mathematical  support.  Moreover,  they  seemed  to  view  the  main  instructor  as  the 
“owner”  of  the  class.  Even  after  seeking  help  from  the  graduate  assistants,  the  participants  often  wanted  the  instructor’s 
approval.  For  example,  one  participant  was  exploring  an  investigation  where  he  needed  to  find  the  maximum  volume  for  a 
box.  The  participant  asked  one  of  the  graduate  assistants  how  he  could  incorporate  technology  in  the  investigation.  More 
specifically,  he  wanted  to  know  what  technology  he  could  use.  The  graduate  assistant  discussed  some  of  his  options. 
Instead  of  exploring  these  routes  on  his  own  and  finding  multiple  representations  of  the  problem,  the  participant  told  the 
graduate  assistant  that  he  was  going  to  ask  the  instructor  which  way  he  should  explore  the  investigation.  The  participant 
was  seeking  a “correct  process”  for  solving  the  investigation  from  the  instructor.  He  only  wanted  to  explore  the  problem 
the  way  the  instructor/  “owner”  saw  it. 

The  instructor  seemed  to  put  himself  in  the  ownership  role  through  the  manner  in  which  he  structured  the  class.  He 
directed  the  workshop  conversations  and  selected  which  problems  to  investigate.  He  sought  little  input  from  the 
participants  about  exploring  the  problems  he  had  chosen.  The  participants  were  placed  in  a passive  role  during  the  first  half 
of  the  workshop.  In  the  observations,  many  were  reported  as  off-task  during  the  lecture/demonstration  portion  of  the 
workshop.  During  the  second  portion  of  each  meeting,  participants  chose  to  work  individually  on  their  write-ups  with  little 
communication  with  other  participants.  Yet,  in  the  interviews,  the  participants  relayed  the  feeling  that  there  was  not 
enough  support  available  during  the  workshop.  This  feeling  further  illustrates  the  assertion  that  the  participants  did  not  turn 
to  each  other  for  support,  but  rather  saw  the  instructors  as  their  only  source  of  support.  In  observations,  the  second  half  of 
the  workshop  was  described  as  being  very  quiet  other  than  the  sound  of  clicking  and  the  graduate  assistants  talking  to  the 
participants.  We  would  have  liked  to  have  seen  more  interaction  and  collaboration  among  the  students  during  the  entire 
class  period. 

• View  of  InterMath 

The  data  distinguished  three  categories  of  the  participants’  views  of  the  goals  and  purposes  of  InterMath.  In  the  first 
category,  participants  saw  InterMath  as  a “make  and  take”  activity  to  take  into  their  middle  school  mathematics  classrooms. 
They  selected  investigations  for  their  own  use  based  on  the  knowledge  level  of  their  students,  rather  than  their  own 
knowledge  level.  Because  of  this,  the  participants  failed  to  push  themselves  to  increase  their  own  mathematical 
understandings.  These  participants  had  the  misconception  that  InterMath  was  for  their  middle  school  students,  rather  than  a 
challenge  for  themselves  personally.  This  idea  influenced  problem  selection  and  depth  of  exploration  of  a problem,  which 
was  evidenced  in  some  of  the  workshop  discussions  and  by  the  participants’  webpages.  In  one  class,  a participant  voiced 
the  concern  that  the  investigations  seemed  very  hard  for  middle  school  students.  The  instructor  explained  that  the 
investigations  were  meant  for  the  teachers  and  that  the  teachers  would  have  to  adapt  them  if  they  chose  to  use  them  with 
middle  school  students.  Despite  this  explanation,  participants  continued  to  cling  to  the  idea  that  the  investigations  were 
suitable  for  their  middle  school  students  with  little  modification  or  thought  into  how  to  present  such  an  activity  to  this  age 
level. 

A second  group  seemed  to  view  InterMath  as  technology.  They  wanted  to  learn  how  to  use  the  software  tools,  but  took 
little  interest  in  using  the  tools  to  develop  mathematical  understandings.  In  workshop  observations,  these  participants 
became  excited  when  using  the  technology  and  learning  something  new  on  the  computer.  However,  very  little  of  their  focus 
was  placed  on  learning  new  mathematical  concepts  and  making  connections.  For  example,  during  a one  of  the  workshop 
classes,  one  of  the  graduate  assistants  showed  a participant  which  button  to  push  to  display  all  the  Excel  functions  she  might 
have  needed  to  create  a spreadsheet.  The  participant  said  something  to  the  nature  of  “woo-hoo!  I’m  finally  excited  about 
something  in  here!”  This  participant  apparently  wanted  the  InterMath  workshop  to  teach  her  to  use  the  technology  more 
efficiently,  rather  than  using  it  to  deepen  her  understanding  of  mathematics. 

The  last  group  saw  InterMath  as  an  opportunity  to  enhance  their  mathematical  understandings.  In  the  interviews,  these 
participants  stressed  the  learning  of  mathematics  over  the  learning  of  the  technology.  Unfortunately,  few  participants  held 
this  view  of  InterMath.  One  explanation  may  be  that  the  participants  seemed  to  have  a low  mathematical  knowledge  base. 
The  participants  particularly  seemed  to  experience  difficulties  in  the  discipline  of  geometry  and  thus  had  trouble  making 
mathematical  connections  and  multiple  representations  that  were  crucial  in  the  investigations.  However,  these  were  the 
students  who  seemed  most  interested  in  further  exploration  of  the  mathematics  and  also  the  most  reflective  about  their  own 
mathematical  ability. 

• InterMath  adoption  to  the  classroom 

Some  of  the  participants  had  already  begun  to  use  InterMath  in  their  classroom  before  the  end  of  the  workshop. 
Surprisingly,  there  seemed  to  be  little  or  no  adaptation  of  the  InterMath  investigations  when  the  teachers  took  them  into  the 
middle  school  classroom,  as  evidenced  in  workshop  observation  discussions.  This  is  ironic  since  a number  of  teachers 
mentioned  in  class  that  they  felt  like  the  InterMath  investigations  were  not  appropriate  for  the  middle  school  level.  Late  in 
the  semester,  one  of  the  participants  pulled  one  of  the  graduate  assistants  to  the  side  and  shared  with  her  what  she  had  been 
doing  in  her  middle  school  classroom.  She  had  assigned  her  students  to  choose  three  InterMath  investigations  directly  from 
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the  website  to  work  on  and  to  write-up  in  two  weeks.  Apparently,  the  participant  made  no  modifications  on  the 
investigations  for  her  middle  school  students.  Even  more,  the  participant  seemed  to  make  no  attempt  to  guide  the  students 
as  to  which  investigations  they  might  attempt.  The  teacher  made  no  reference  to  having  her  middle  school  students  work  in 
cooperative  groups,  but  rather  asked  that  each  student  turn  in  their  own  independent  writo-up. 

The  manner  in  which  some  of  the  participants  incorporated  InterMath  into  their  classrooms  is  not  surprising  considering 
how  many  participants  viewed  InterMath  as  a mathematical  development  tool  for  middle  school  students,  but  not  for 
themselves.  In  addition,  instructing  the  middle  school  students  to  work  independently  parallels  the  way  the  participants 
themselves  worked  through  the  investigations  during  the  workshop.  Many  of  the  investigations  were  not  meant  for  middle 
school  students,  but  rather  to  challenge  the  middle  school  teachers’  own  conceptions  of  mathematics  and  how  it  is  taught. 

In  the  post-workshop  surveys,  most  of  the  participants  perceived  an  increase  in  their  ability  to  incorporate  technology  and 
mathematics  in  the  classroom.  However,  interviews  reveal  that  the  participants  felt  a need  for  more  pedagogical  support  in 
taking  InterMath  into  the  classroom. 

Case  2 

Overview 

The  UGA  pilot  of  the  InterMath  workshop  was  held  at  a center  designed  for  supporting  teacher  professional  development 
with  technology  on  the  University  of  Georgia  campus.  The  classroom  was  made  up  of  four  rows  of  two  tables  that  all  faced  an 
overhead  screen.  Each  table  held  four  computers,  and  an  instructor's  workstation  sat  at  the  back  of  the  room.  The  workshop  met 
one  evening  per  week  during  the  Spring,  2001  semester. 

The  class  began  with  seven  students;  however,  by  the  end  of  the  fifteen  weeks,  there  were  only  four  participants  in  regular 
attendance.  Two  of  these  participants  taught  eighth  grade  pre-algebra  and  algebra  at  a rural  middle  school.  The  other  two 
participants  came  from  a private  middle  school;  one  was  a sixth  grade  mathematics  teacher  and  the  other  was  the  technology 
support  person  who  also  had  a mathematics  education  background. 

A professor  from  the  mathematics  department  at  the  University  of  Georgia  led  the  workshop,  and  two  graduate  assistants,  one 
from  UGA’s  mathematics  education  department  and  one  from  the  instructional  technology  department,  attended  regularly  in  order 
to  assist  the  instructor  and  participants  and  to  collect  research  data.  A third  graduate  assistant,  also  from  the  instructional 
technology  department,  attended  the  first  few  workshops  and  supported  the  participants  in  learning  web  publishing. 


What  the  participants  DID  learn 

There  were  some  overarching  successes  in  this  pilot.  First,  the  participants  learned  how  to  use  technology  to  create  and  post 
write-ups  of  their  mathematical  investigations.  Specifically,  the  participants  learned  how  to  use  computer  software  that  included 
web  composers  and  FTP  clients.  On  average,  the  participants  posted  seven  write-ups  to  the  internet.  These  write-ups  often 
included  links  to  spreadsheets  and/or  dynamic  geometry  files. 

Second,  the  participants  learned  to  identify  and  to  appreciate  certain  aspects  of  reform-based  issues  in  mathematics  teaching 
and  learning.  As  evidenced  through  their  final  interviews,  the  participants  noted  the  value  of  problem  solving,  learning  through 
collaboration  and  communication,  finding  multiple  solutions  and  answers,  and  asking  extension  questions.  For  example,  when 
asked  what  students  in  an  ideal  mathematics  classroom  would  be  doing,  one  participant  commented,  "Well,  after  all  this,  problem 
solving."  Anther  participant  said  that  an  ideal  classroom  to  her  would  be  one  in  which  the  students  are  "asking  questions  and 
they're  showing  their  classmates  what's  happening  and  sharing  ideas  and  thoughts  and  communicating  with  each  other."  A third 
participant  mentioned  that  the  most  important  things  she  learned  from  the  InterMath  experience  were  "The  importance  of 
thinking  and  not  just  computation.  ...  And  collaboration."  She  also  stated,  "I've  even  told  my  kids  that  there  are  lots  of  ways  to 
find  an  answer  and  oftentimes  the  answer's  not  the  important  part."  It  was  clear  that  mathematics  and  mathematics  education 
pedagogy  were  key  issues  to  these  participants. 

What  the  participants  DID  NOT  learn 

There  were  also  some  critical  areas  in  which  learning  did  not  occur  as  expected.  Improvement  in  these  areas  is  a focus  in 
preparing  for  the  next  InterMath  workshop.  First,  the  participants  did  not  seem  to  greatly  expand  their  mathematics  content 
knowledge.  Approximately  61%  of  the  write-ups  posted  were  about  investigations  that  were  taken  from  the  Algebra  or  Number 
Concepts  units  on  the  InterMath  website.  These  units  correspond  to  the  majority  of  the  topics  that  are  covered  in  middle  grades 
mathematics.  Only  25%  of  the  write-ups  focused  on  Geometry  problems  and  only  7%  on  Data  Analysis  problems.  One 
participant  mentioned  that  after  she  and  her  partner  struggled  with  a problem  that  was  hard,  they  would  simply  "close  that  one  up, 
and  we'd  do  another  one."  Issues  of  lack  of  perseverance  and  an  unwillingness  to  try  new  areas,  possibly  relating  to  efficacy, 
were  prevalent. 

Second,  the  participants  did  not  become  comfortable  with  using  a variety  of  mathematical  software  in  doing  their 
investigations.  Approximately  86%  of  write-ups  indicated  that  the  authors  used  spreadsheets  to  help  them  with  the 
investigations.  Not  surprisingly,  spreadsheets  were  the  only  software  the  teachers  had  considerable  experience  with  when  they 
began  the  workshop.  Only  1 8%  of  write-ups  illustrated  use  of  geometry  software,  and  only  4%  mentioned  the  use  of  graphing 
software.  One  participant  stated  that  she  and  her  partner  "felt  more  comfortable  using  a spreadsheet.  And  it's  just  because... that's 
what  we  could  maneuver  better  with." 

Finally,  the  participants  did  not  mention  any  form  of  proof  in  their  write-ups  (it  should  be  noted,  however,  that  the  instructor 
did  not  emphasize  the  importance  of  including  proofs  in  the  write-ups).  Most  of  the  participants  used  numeric  patterns  or 
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measurements  to  justify  their  solutions  to  the  investigations.  No  one  offered  any  conceptual  explanations  or  tried  to  rationalize 
why  the  numeric  patterns  or  measurements  must  have  given  the  correct  answer.  More  disturbing,  they  also  did  not  seek  to  use 
extensions  to  push  their  thinking  and/or  their  students’  thinking  further,  even  though  that  issue  was  an  explicit  focus  of  the 
instructor.  The  instructor  commented  that  even  when  the  participants  did  write  extensions,  they  never  tried  to  solve  them.  This 
fact  may  relate  to  the  same  issues  that  prevented  attempts  at  difficult  problems. 

Cross-Case  Analysis 

Several  findings  emerged  that  spanned  across  both  cases.  There  were  also  some  findings  within  each  case  that  we  were 
unable  to  reconcile.  For  example,  we  are  not  sure  why  we  lost  three  of  seven  students  in  Case  2.  While  they  reported  they  each 
left  for  personal  reasons,  the  fact  that  all  three  were  from  one  school  raises  questions  about  how  to  keep  busy  teachers  engaged  in 
the  experience.  We  can  speculate  about  the  role  of  groups,  the  need  for  proper  location,  etc.;  however,  there  is  no  clear  way  to 
determine  whether  that  trend  is  one  we  need  to  attend  to  in  future  cases. 

For  our  cross-case  analysis,  we  adhered  to  the  findings  framework  introduced  previously.  We  looked  at  what  we  found  in 
each  case,  what  was  true  in  both  cases,  and  what  we  thought  might  be  a reasonable  assertion  based  on  the  evidence  provided. 

Support  & Interaction 

We  found  that  support  and  interaction  became  very  intertwined  in  our  cross-case  analysis.  This  fact  was  because  most  of  the 
interactions  between  students  and  between  instructors  and  students  were  focused  on  helping  the  learners  be  successful  in  what 
they  were  doing.  We  noted  that  there  were  two  distinct  kinds  of  interactions:  affective  (those  aimed  at  providing  positive 
feedback  or  other  information  to  keep  the  students  motivated)  and  intellectual  (those  interactions  that  provided  the  information 
learners  needed  in  order  to  move  on  with  the  problem  on  which  they  were  working).  Based  on  our  interviews  and  observations, 
the  affective  interactions  were  particularly  important  between  participants.  Several  times  the  learners  commented  that  they  felt 
behind  or  inadequate  until  they  began  talking  with  the  other  participants  or  until  they  began  to  find  out  from  the  support  staff  that 
others  were  having  the  same  kinds  of  problems.  In  more  than  one  case,  this  “same  boat”  effect  prevented  our  participants  from 
dropping  out  of  the  workshop. 

Another  support/interaction  issue  that  appeared  was  the  overwhelming  number  of  procedural  questions  that  were  asked  by  the 
participants.  In  both  workshops,  until  around  the  middle  of  the  semester,  the  questions  all  focused  on  how  to  use  particular  pieces 
of  software.  Later,  we  saw  some  movement  to  more  process-oriented  thinking,  but  the  procedural  questions  never  faded  entirely. 
This  finding  raises  a number  of  questions  about  supporting  the  teachers  in  getting  the  learning  that  we  had  hoped  for  from  the 
workshop  and  about  who  needs  to  provide  support  and  what  that  support  should  look  like.  In  Case  1 , we  had  about  30  students 
with  three  support  people  (two  graduate  assistants  and  one  instructor)  in  Case  2,  we  had  one  instructor,  one  to  two  graduate 
assistants,  and  another  graduate  assistant  who  acted  primarily  as  a researcher,  but  ended  with  only  four  students.  Even  with,  or 
perhaps  because  of,  this  presence  of  knowledgeable  others,  the  teachers  resisted  engaging  with  each  other  for  problem  solving, 
instead  turning  to  those  perceived  as  owning  information.  This  phenomenon  leaves  an  open  question  about  whether  the  students 
perceived  that  InterMath  allowed  them  to  understand  the  theory  and  skills  they  were  learning  - after  all,  if  they  still  felt  they 
needed  to  seek  instructor  guidance  rather  than  relying  on  themselves  or  their  peers,  it  is  likely  that  they  still  held  the  traditional 
idea  that  math  is  about  right  answers  and  that  the  teacher’s  role  is  to  have  those  answers. 

Finally,  while  we  provided  every  opportunity  for  collaborative  learning,  few  teachers  chose  to  engage  in  it.  Even  in  those 
instances  where  teachers  worked  as  pairs  or  trios,  they  tended  to  each  work  their  own  problem  and  rely  oh  each  other  only  when 
they  were  confused  or  unable  to  continue.  We  also  found  that  among  the  teachers  who  did  work  together,  almost  every  group 
included  teachers  who  worked  together  in  the  same  schools.  These  findings  combined  lead  to  two  insights:  first,  teachers  seem  to 
work  with  people  who  they  already  know  and  feel  safe  with;  second,  teachers  are  not  naturally  predisposed  to  working  in  groups. 
This  second  point  may  explain  many  teachers’  reluctance  to  include  groupwork  in  their  classrooms-  which  reinforces  the  need 
for  the  professional  development  environment  to  model  the  desired  classroom  environment. 


Barriers  & Difficulties 

There  were  two  main  barriers  across  the  two  cases:  technology  and  goals.  The  technology  problem  was  one  of  both 
participant  inexperience  with  the  tools  we  were  using  and  hardware  problems  that  were  exasperated  by  participant  inexperience 
with  the  tools.  The  technology  difficulties  were  so  severe  that  almost  half  of  each  workshop  was  spent  with  students  struggling  to 
make  webpages  and  publish  them.  This  amount  of  time  was  particularly  alarming  given  that  the  webpage  aspect  of  the  class  was 
only  a tool  for  portfolio  generation.  The  technologies  that  were  of  greatest  interest  were  tools  that  allowed  mathematical 
visualization  and  oqjloration  (e.g..  Geometer’s  Sketchpad,  Excel,  and  NuCalc/Graphing  Calculator).  Further,  the  web 
development  goal  was  a tiny  one  as  compared  to  the  mathematical  aims  of  the  InterMath  experience. 

The  barrier  caused  by  goals  was  an  interesting  one.  The  problem  was  that  the  participant  goals  and  the  workshop  goals  were 
not  always  in  alignment.  In  our  follow-up  interviews  and  surveys,  for  instance,  a large  number  of  teachers  indicated  that  learning 
technology  was  their  perceived  goal  for  InterMath.  Those  who  reported  this,  it  should  be  noted,  were  also  quite  happy  with  their 
experience.  However,  that  was  not  our  intended  goal.  What  we  had  hoped  was  to  allow  teachers  to  think  about  teaching  and 
learning  mathematics  in  a different  way  - certainly  technology  was  a part  of  that  vision,  but  not  the  central  component. 

Another  large  group  of  teachers  seemed  to  think  that  the  InterMath  workshop  provided  an  opportunity  to  become  familiar  with  a 
tool,  the  InterMath  website,  that  could  be  used  in  middle- grades  classrooms.  While  there  were  some  problems  that  certainly  could 
be  useful  for  middle  school  students,  the  purpose  and  intention  of  the  site  was  to  enhance  teacher  mathematical  understanding. 


o 

ERIC 


333 


342 


Because  teachers  saw  the  site  as  being  a tool  for  use  in  their  own  classrooms,  many  completed  only  problems  they  felt  their 
students  could  complete.  This  meant  that  many  of  them  did  not  challenge  their  own  mathematical  abilities  at  all. 

On  one  hand,  because  the  participants  were  able  to  define  and  follow  their  own  goals,  they  were  pleased  with  the  outcome. 
On  the  other  hand,  we  have  concerns  about  the  kind  and  quality  of  learning  given  that  teachers  did  not  seem  concerned  with  their 
mathematical  development. 

Adoption 

Our  final  major  finding  in  the  cross-case  analysis  was  a disturbing  trend  among  the  teachers  who  implemented  the  InterMath 
problems  in  their  classroom  to  structure  their  students’  learning  experiences  exactly  as  their  workshop  experience  had  been 
structured.  This  was  alarming  for  a number  of  reasons.  First,  it  demonstrated  little  reflection  on  the  part  of  the  teachers  about  their 
students’  abilities  in  mathematics.  They  allowed  students  to  randomly  choose  problems  from  sets  that  covered  a number  of  topics 
and  varied  in  conceptual  difficulty  tremendously.  Further,  the  teacher  participants  had  complained  throughout  the  workshop,  in 
both  cases,  that  there  was  not  enough  structuring  because  there  were  not  clear  guidelines  for  assignments,  etc.  Yet,  they  reported 
implementing  this  same  kind  of  approach  for  their  students  who  did  not  have  the  maturity  or  life-experience  upon  which  to  draw 
to  cope  in  this  extremely  open-ended  environment.  In  short,  it  seemed  that  the  teachers  borrowed  InterMath  rather  than  adopting 
it.  It  may  be  argued  that  this  is  the  first  step  of  adoption,  but  it  is  complicated  because  we  no  longer  have  the  opportunity  to 
support  these  teachers  in  their  efforts. 

Further,  post  survey  results  indicated  that  the  teachers  were  not  yet  comfortable  with  the  implementation.  This  was  perhaps 
corroborated  by  the  teachers  who  talked  about  using  demonstration  techniques  to  implement  InterMath  in  their  classrooms  or 
those  who  said  they  needed  more  practice  themselves  before  they  could  implement.  While  these  problems  are  somewhat  different 
from  the  wholesale  adoption  approach  with  no  attention  to  philosophy,  they  still  prevent  the  students  from  having  a successful 
experience  with  mathematical  explorations. 

Conclusion 

In  conclusion,  we  offer  our  suggestions  for  improvement  for  this  kind  of  workshop  experience. 

First,  it  is  vital  to  the  success  of  the  workshop  that  we  solve  the  technology  problem.  Unless  we  find  a way  to  shift  teacher 
focus  away  from  the  procedural  aspects  of  using  the  technology,  we  will  not  be  able  to  support  them  in  their  content  knowledge 
development.  While  the  learner-centered  professional  development  principles  do  indicate  that  teachers  should  be  in  charge  of 
setting  their  goals,  it  seems  critical  that  the  learning  of  procedure  needs  to  somehow  be  removed  from  the  prominent  position  it 
held  in  our  pilot  workshops.  Some  ways  of  supporting  teachers  in  the  technology  area  might  include  setting  prerequisites  for 
taking  the  workshop,  creating  a forms-driven  webpage  publishing  approach,  or  rethinking  the  role  of  online  portfolios  in  the 
InterMath  experience.  We  could  also  provide  special  technology  skills  workshops  for  the  participants.  From  a more  systemic 
view,  it  seems  that  schools  need  to  support  a higher  level  of  technology  literacy  among  their  teachers.  All  of  our  participants  had 
completed  some  kind  of  basic  technology  training,  yet  they  did  not  know  how  to  accomplish  their  goals  either  conceptually  or 
procedurally. 

For  the  goal  alignment  problem  there  are  several  potential  solutions.  First,  consistent  with  the  NPEAT  standards,  more 
professional  development  opportunities  should  be  aimed  at  supporting  teacher  conceptual  development  rather  than  activity 
generation.  Further,  we  learned  that  simply  telling  the  participants  about  the  intentions  of  the  workshop  was  not  enough.  They 
need  to  be  challenged  through  the  structuring  of  the  workshop  to  push  themselves.  Further,  they  need  to  have  the  opportunity  not 
only  to  own  the  goals  they  are  aiming  for,  but  also  to  own  the  workshop  itself  Participants  need  to  feel  that  they  can  work  with 
the  instructor(s)  to  steer  the  professional  development  program  for  their  success.  Finally,  we  need  to  look  for  ways  to  let  the  need 
for  technology  arise  out  of  the  mathematics  so  that  the  teachers  first  explore  what  mathematics  and  mathematical  knowledge  are, 
then  look  for  ways  to  solve  the  problems.  The  technology  should  be  one  of  a host  of  tools  that  they  are  comfortable  with  and  able 
to  use  in  the  learning  environment. 

Finally,  the  adoption  issue.  From  our  experience  in  these  two  classes,  it  seems  reasonable  that  a first  step  toward  more 
meaningful  adoption  by  the  teachers  would  be  to  attack  the  problem  head  on  - to  discuss  ways  to  implement  InterMath  with  the 
teachers,  to  look  at  ways  problems  might  be  modified  to  more  appropriately  meet  the  needs  of  middle  school  students,  and  to 
discuss  the  value  and  learning  that  might  come  from  using  the  investigations.  A second  approach  might  be  to  model  a classroom 
approach  to  using  the  problems  with  the  students.  In  this  way,  we  would  provide  a safe  environment  in  which  teachers  can  think 
through  the  issues  involved  with  implementation  before  affecting  students.  Finally,  our  ongoing  goal  of  creating  a lesson  plan 
database  may  help  with  the  adoption  process.  The  database  will  provide  InterMath  participants  with  access  to  tested  lesson  plans 
that  other  teachers  have  used  in  their  own  classrooms. 

In  conclusion,  InterMath  offers  one  view  of  a research-based  professional  development  endeavor.  In  our  initial 
implementation  we  learned  much  about  the  successes  and  pitfalls  of  working  within  this  kind  of  framework. 
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Abstract 

The  Instructional  Systems  Technology  Distance  Master  of  Science  program  is  one  of  the  first  degrees  of  its  kind  to  be  offered 
at  Indiana  University.  Other  than  an  initial  on-campus  orientation,  it  can  be  completed  entirely  via  the  Internet.  The  first  course 
for  this  program  was  offered  Fall  of  2000  with  eighteen  students  enrolled.  This  same  semester,  a research  team  analyzed  the 
monetary  costs  and  pecuniary  benefits  of  this  program  in  a cost-benefit  analysis.  They  identified  the  costs  and  benefits  from  the 
perspective  of  the  department.  Using  these,  they  calculated  the  costs  to  benefits  ratio.  Although  the  team  concluded  that  the 
distance  program  had  a high  costs  to  benefits  ratio,  there  are  many  value  benefits  which  were  not  monetarily  included  in  the 
analysis.  This  report  also  offers  recommendations  for  further  cost-benefit  analyses  of  a distance  education  program. 

Introduction 

For  Fall  semester  2000  the  Instructional  Systems  Technology  (1ST)  department  in  the  School  of  Education  at  Indiana 
University  established  an  online  Distance  Master  of  Science  (DM)  program.  It  was  designed  and. developed  to  mirror  the  on- 
campus  1ST  Master  of  Science  program. 

As  part  of  the  on-campus  course  R563:  Business  and  Economics  of  Training  and  Development,  a team  of  six  students 
conducted  a cost-benefit  analysis  (CBA)  for  the  DM  program  in  the  Fall  of  2000.  Literature  shows  that  most  studies  of  online 
learning  environments  have  been  conducted  to  examine  educational  advantages  and  to  explore  effective  design  strategies  (Jung  & 
Rha,  2000).  In  contrast,  the  R563  team,  working  with  Dr.  Charles  Reigeluth,  Director  of  the  DM  program,  focused  their  analysis 
on  the  monetary  costs  and  benefits  to  the  1ST  department  and/or  the  School  of  Education.  For  simplicity,  these  will  be  referred  to 
as  “the  1ST  department”  or  “the  department.” 

By  presenting  a breakdown  of  the  developmental  and  ongoing  costs  and  benefits,  this  report  provides  information  from  which 
the  1ST  department  could  base  future  DM  program  decisions.  This  report  may  also  be  useful  for  others  who  are  responsible  for 
the  design,  development,  implementation,  and  maintenance  of  distance  education  programs  in  higher  education. 

Literature  Review 

Distance  Education 

According  to  Molenda  (1996),  “distance  education  refers  to  a program  of  some  duration,  leading  to  formal  recognition  of 
achievement,  in  which  the  learner  is  separated  from  the  instructor  and  in  which  special  arrangements  have  been  made  to  facilitate 
dialog  between  the  remote  students  and  an  instructor.”  Keegan’s  (1980)  definition  of  distance  education  is  still  widely  used 
today.  He  defines  the  principle  characteristics  of  distance  education  as  the  separation  of  teacher  and  learner;  influence  of  an 
educational  organization;  use  of  technical  media;  provision  for  two-way  communication;  and  possibility  of  occasional  meetings. 

In  general,  educators  believe  that  distance  education  is  subject  to  economies  of  scale  and  that  the  primary  benefit  of  this  form 
of  education  is  that  costs  can  be  distributed  over  a large  number  of  students.  It  is  seen  as  a probable  money-making  venture 
because  of  the  greater  number  of  students  who  could  potentially  enroll  versus  traditional  on-campus  programs.  The  higher  the 
revenue  overall,  the  lower  the  cost  per  student  would  be  (Inglis,  1999;  Whalen  & Wright,  1999).  While  this  potential  to  exploit 
economies  of  scale  exists  in  distance  education,  it  does  not  mean  that  only  programs  with  high  student  enrollment  can  be  cost 
beneficial  (Curran,  1995). 
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Cost-Benefit  Analysis 

When  conducting  a cost-benefit  analysis,  dollar  amounts  are  examined  for  both  the  costs  and  benefits.  The  costs  are  then 
compared  to  the  benefits  in  a costs  to  benefits  ratio.  This  allows  one  to  determine  the  extent  to  which  the  monetary  value  of  a 
program’s  benefits  outweigh  the  costs  (Sikorski  et  al,  1991).  If  the  decision  were  purely  economical,  a program  would  exist  only 
if  the  costs  to  benefits  ratio  were  less  than  one,  meaning,  monetarily,  benefits  have  exceeded  the  costs. 

An  advantage  of  cost-benefit  analyses  is  that  programs  can  be  directly  compared  no  matter  what  their  platform,  desired 
outcomes,  values,  and  delivery  systems  are.  Everything  is  converted  to  monetary  values  and  thus  can  be  directly  compared. 

The  primary  disadvantage  of  a cost-benefit  analysis  is  that  oftentimes  it  is  difficult,  if  not  impossible,  to  account  for  every 
foreseeable  cost  and  benefit  in  monetary  amounts  (Levin,  2000).  This  is  especially  true  for  certain  value-based  benefits,  which 
are  often  left  out  of  a CBA  (Cukier,  1997).  For  this  reason,  it  is  mportant  to  consider  other  cost-analyses,  such  as  cost- 
effectiveness,  cost-utility,  and  cost -feasibility  when  deciding  whether  or  not  to  pursue  a particular  program. 

Measuring  Costs 

As  part  of  a study  conducted  at  Marshall  University,  Morgan  (2000)  divided  costs  into  categories  to  help  determine  the  cost 
of  online  courses.  Based  on  this  study,  three  main  cost  categories  emerge: 

1.  Capital  and  recurrent  costs:  These  occur  on  an  ongoing  basis,  such  as  technology  support,  equipment  upgrades, 
indirect  costs,  and  course  maintenance. 

2.  Production  costs:  These  are  incurred  during  the  development  of  courses,  including  factors  like  providing  software  to 
students. 

3.  Delivery  costs:  These  costs  are  associated  with  teaching  a course,  such  as  instructor  salary,  course-related  mailings  to 
students,  and  opportunity  costs  related  to  teaching  a course  instead  of  doing  an  alternative. 

These  costs  can  be  broken  down  even  further  to  include  hidden  costs,  technology  specific  costs,  support  personnel  costs,  faculty 
training  costs,  and/or  administrative  costs  (Morgan,  2000). 

Whalen  and  Wright  (1999)  compared  the  capital,  production,  and  delivery  costs  of  online  courses  with  the  costs  of  equivalent 
courses  taught  in  the  classroom.  They  made  the  assumption  that  the  learning  outcomes  were  the  same.  Their  capital  costs 
included  the  cost  of  the  server  which  housed  all  courses.  They  divided  production  costs  into  six  areas:  instructional  and 
multimedia  design  costs;  the  cost  of  producing  text,  audio,  video,  graphics,  and  photographs;  the  costs  of  authoring  and 
delivering  software,  or  the  cost  of  licensing  and  delivering  commercial  software;  the  costs  of  testing  and  modifying  course 
content;  student  and  instructor  training  costs;  and  final  course  testing  costs. 

For  their  analysis,  they  also  divided  their  costs  into  fixed  and  variable  costs.  They  defined  fixed  costs  as  costs  that  remain  the 
same  regardless  of  the  output  and  variable  costs  as  those  that  vary  directly  with  the  amount  of  output — so  fixed  costs  are  the  same 
no  matter  how  many  students  are  in  a course,  while  variable  costs  increase  with  the  number  of  students. 

They  determined  that  online  courses  tend  to  have  higher  fixed  costs  than  classroom-based  courses,  but  that  these  costs  are 
offset  by  lower  variable  costs.  Due  to  the  reduction  in  course  delivery  time  and  the  potential  to  deliver  courses  to  a larger  number 
of  students,  they  found  online  courses  to  be  more  cost-beneficial  than  classroom  teaching. 

Whalen  and  Wright  chose  to  ignore  costs  that  would  have  been  incurred  had  a course  been  delivered  in  a classroom  (sunk 
costs)  in  their  analysis  on  online  courses.  They  identified  these  costs  as  instructor  salary  and  benefits,  equivalent  costs  of  course 
development,  course  materials,  administrative  support,  and  classroom  overhead. 

After,  gathering  cost  information  for  both  online  and  traditional  courses,  Whalen  and  Wright  used  costs  to  benefits  ratio 
analyses  to  determine  the  breakeven  number  of  students  required  to  recover  costs  over  five  years. 

In  a similar  study,  Bartolic-Zlomislic  and  Brett  (1999)  analyzed  costs  and  benefits  of  an  online  graduate  course  at  the 
University  of  Toronto.  Their  analysis  projected  that  the  online  course  would  likely  make  a profit  of  1,962  Canadian  dollars  per 
year.  They  also  calculated  that  1 9 students  would  be  needed  to  achieve  a breakeven  point. 

Measuring  Benefits 

When  he  estimated  monetary  value,  Cukier  (1997)  divided  benefits  into  three  categories: 

1.  Performance-driven  benefits:  These  include  cost  savings,  revenues,  and  other  income.  These  benefits  are  usually  the 
easiest  to  quantify  and  thus  lend  themselves  easily  to  a cost-benefit  analysis. 

2.  Value-driven  benefits:  From  a departmental  perspective,  these  may  include  time  efficiency,  flexibility,  consistency  in 
quality  of  delivery,  ease  of  access  to  the  technology,  the  quality  of  student-teacher  interaction,  the  rate  and  ease  by 
which  material  can  be  updated  and  changed,  the  appearance  of  being  at  the  cutting  edge  of  technology,  opportunities 
for  on-campus  students  to  produce  the  online  course,  and  ej^ansive  delivery  with  limited  interaction. 

3.  Value-added,  societal,  or  indirect  benefits:  These  include  reduction  in  capital  investments  (fewer  buildings  and 
parking  lots),  reduction  in  pollution,  increased  job  creation,  new  business  opportunities  (telephone  companies, 
publishers),  reductions  in  social  community  costs,  the  creation  of  secondary  markets,  time  savings,  revitalizing  a 
curriculum  and  faculty,  reaching  new  markets,  and  increasing  student  diversity. 

An  organization  may  adopt  a value-based  approach  to  a CBA  to  stress  the  importance  of  understanding  the  pedagogical  needs 
and  values  when  judgments  are  made  about  costs  and  benefits.  The  main  strength  of  a value-based  approach  is  that  it  allows  for  a 
subjective  definition  of  benefits,  therefore  makingit  a flexible  technique  (Cukier,  1997).  Such  an  organization  may  decide  that  a 
program  which  may  have  a high  costs  to  benefits  ratio  is  still  worth  pursuing  because  of  the  value  it  offers. 
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AH  benefits,  as  well  as  costs,  are  determined  on  a program  by  program  basis  as  each  organization  has  differing  pedagogical 
needs  and  values.  For  example,  one  possible  value-driven  benefit  for  an  online  program  might  be  expansive  delivery  with  limited 
interaction.  However,  for  another  organization  which  values  interaction,  this  would  be  viewed  as  a cost. 

Measurement 

The  data  were  collected  from  four  separate  sources: 

7.  Cohort  Study:  Data  concerning  faculty  salaries  were  taken  from  the  Cohort  Study  located  in  the  Dean  of  Faculties  at 
Indiana  University. 

2.  Questionnaire:  General  data  regarding  the  use  and  opinions  of  the  DM  program  were  gathered  using  a survey 
questionnaire.  It  was  distributed  to  and  filled  out  by  several  faculty  members,  staff  members,  and  graduate  assistants 
who  were  directly  involved  in  the  program. 

3.  Personal  Interviews:  From  the  respondents  of  the  questionnaire,  the  team  selected  individuals  to  be  interviewed.  They 
were:  Dr.  Charles  Reigeluth,  Director  of  the  DM  program;  Dr.  Robert  Appelman,  Head  of  Technical  Support;  Prof 
Elizabeth  Boling,  1ST  department  Chairperson;  Carthel  Everett,  Contract  and  Grants  Specialist  in  the  School  of 
Education;  Susie  Sloffer,  DM  program  graduate  assistant;  and  Bill  Dueber,  technical  support  graduate  assistant.  These 
interviews  focused  more  specifically  on  costs  and  benefits  regarding  the  DM  program. 

4.  Online  Resources:  Websites  on  various  distance  education  programs  at  Indiana  University  and  at  other  institutions 
added  to  the  general  knowledge  and  understanding  of  costing  a distance  education  program.  Of  these,  the  most 
beneficial  was  Morgan’s  (2000)  study. 


Methodology 

Gathering  Cost  Data 

Based  upon  Morgan’s  (2000)  study,  the  team  gathered  cost  information  for  the  first  academic  year  of  the  DM  program  and 
placed  it  into  the  three  categories: 

1.  Capital  and  recurrent  costs:  Server,  server  administration,  data  communications  charges,  maintenance,  equipment, 
technological  support,  and  indirect  costs. 

2.  Production  costs:  Faculty  training  for  online  course  tools,  course  development,  course  materials,  and  licensing 
software. 

3.  Delivery  costs:  Instructor  salary  and  fringe  benefits,  director  salary,  graduate  assistant  support,  adjunct  instructor 
wages,  and  opportunity  costs. 

Gathering  Benefit  Data 

The  team  organized  the  benefits  for  the  first  academic  year  of  the  DM  program  based  on  Cukier’s  (1997)  benefit  categories: 

1.  Performance-driven  benefits:  Funding  for  development,  student  tuition,  and  technology  fees. 

2.  Value-driven  benefits:  Time  efficiency,  flexibility,  ease  of  access  to  the  technology,  the  rate  and  ease  by  which  material 
can  be  updated  and  changed,  the  appearance  of  being  at  the  ‘^cutting  edge”  of  technology,  opportunities  for  on-campus 
students  to  produce  the  online  course,  and  expansive  course  delivery  to  capitalize  on  economies  of  scale. 

3.  Value-added,  societal,  or  indirect  benefits:  Reduction  in  capital  investments,  increased  job  creation,  time  savings, 
revitalizing  a curriculum  and  faculty,  reaching  new  markets,  and  increasing  student  diversity. 

Results 

Costs  and  Benefits 

The  results  of  the  data  are  summarized  as  follows  (see  Appendix  A for  the  detailed  breakdown): 


ACTUAL  COSTS  AND 

BENEFITS 

TOTALS 

Benefits 

$91,606.40 

Less:  Costs 

$177,159.41 

NET 

($85,553.01) 

Using  the  cost  and  benefit  data,  the  team  calculated  the  costs  to  benefits  ratio. 
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Costs  to  Benefits  Ratio 


Costs 

$177,159.41 

Benefits 

$91,606,40 

This  ratio  shows  that  costs  for  the  DM  program  are  nearly  twice  the  monetary  benefits. 

Discussion 

DM  Program  Costs  to  1ST 

It  is  clear  that  the  1ST  DM  program  is  not  a money  making  venture.  But,  it  must  be  remembered  and  emphasized  that  this 
program  was  not  launched  nor  was  ever  intended  to  be  a profit  producing  entity.  The  motivation  behind  the  DM  program  was, 
quite  simply,  to  investigate  the  possibilities  of  offering  an  online  program  that  would  mirror  the  experience  of  the  on-campus 
Master  of  Science  program.  Because  of  the  current  staffing  situation  and  the  labor-intensive  nature  of  delivering  an  equivalent 
online  degree,  the  enrollment  has  been  capped  at  20  students  p er  course. 

The  team  was  able  to  approximate  the  indirect  costs  using  percentage  breakdowns  provided  by  the  School  of  Education.  This 
data  was  then  inserted  into  the  cost  calculation  model  from  Marshall  University.  Some  of  the  categories  from  Dr.  Morgan’s 
model  did  not  apply  to  the  1ST  DM  program.  For  instance,  the  1ST  DM  program  was  given  the  go-ahead  from  the  School  of 
Education  on  July  1,  2000  to  begin  in  the  Fall  2000  semester.  The  department  had  approximately  one  month  to  prepare  the 
coursework.  Thus,  the  1ST  DM  program  generated  revenue  in  the  first  year.  Dr.  Morgan’s  model  devotes  the  first  year  of  a 
distance  education  program  strictly  to  development  with  no  student  enrollment.  Therefore,  there  were  no  monetary  figures  for 
performance-driven  benefits. 

Dr.  Morgan’s  model  also  calls  for  costs  of  technology  investment.  However,  the  1ST  DM  program  had,  and  has,  an  extensive 
existing  infrastructure  within  the  School  of  Education  and  Indiana  University.  Because  of  this,  the  department  did  not  have  to 
purchase  any  extra  equipment,  including  servers.  There  is  also  existing  technology  support  that  is  provided  for  by  the  indirect 
costs.  In  their  interviews,  professors  indicated  that,  in  the  near  future,  there  will  need  to  be  an  investment  into  technology  support 
and  maintenance  for  the  DM  program  to  continue.  These  are  costs  that  will  need  to  be  considered  in  the  future. 

In  this  analysis,  the  team  did  not  factor  in  opportunity  costs.  The  Department  has  not  assigned  monetary  values  to  the  cost  (or 
perhaps  benefit)  of  not  pursuing  other  teaching  opportunities  instead  of  the  DM  program.  According  to  some  survey  responses,  it 
takes  roughly  twice  the  time  to  prepare  for  and  teach  a DM  course  during  this  first  year.  The  university  currently  pays  the 
instructor  the  same  amount  for  teaching  an  online  course  as  it  would  for  an  on-campus  course.  The  underlying  assumption  is  that 
the  time  commitment  for  a distance  education  course  is  no  greater  than  an  on-campus  course.  This  additional  time  could  be  spent 
mentoring  students,  serving  on  dissertation  committees,  conducting  research,  and  publishing. 

DM  Program  Benefits  to  1ST 

Of  the  three  categories  of  benefits  that  the  team  used,  only  the  performance-driven  benefits  had  a specific  dollar  amount 
identified  by  the  department.  Value-driven  and  value-added  benefits  are  not  necessarily  tangible,  and  thus  more  difficult  to  assign 
a monetary  value.  This  does  not  mean  these  benefits  are  less  important.  To  the  contrary,  value-based  benefits  may  be  the  most 
important  benefits  in  a distance  education  program.  Moreover,  depending  on  the  pecuniary  values  assigned  to  value  benefits,  the 
total  benefits  may  far  outweigh  the  total  costs.  It  is  possible  that  this  is  the  case  in  the  1ST  DM  program. 

Performance-driven  benefits: 

• Revenue  generated:  Tuition,  DM  fees,  and  the  startup  money. 

Value-driven  benefits: 

• Institutional  prestige:  By  having  a distance  education  program,  the  reputation  of  Indiana  University,  the  School  of 
Education,  and  the  Instructional  Systems  Technology  department  as  being  on  the  cutting  edge  of  research  and 
technology  is  fortified. 

• Learning  and  applying  opportunities:  Students  in  the  on-campus  program  have  opportunities  to  assist  in  producing  the 
courses  for  the  DM  program.  This  provides  the  on-campus  students  with  an  opportunity  to  apply  the  skills  they  have 
learned  by  producing  authentic  online  courses. 

• Publication  possibilities:  The  1ST  DM  program  provides  numerous  possibilities  for  faculty  and  students  to  conduct 
research  that  may  lead  to  publication. 

• Anytime  and  anywhere.  Students  in  the  DM  program  can  complete  their  assignments  when  it  is  convenient  for  their 
schedules.  The  flexibility  offered  with  distance  education  is  not  possible  in  an  on-campus  course. 

Societal  or  value-added  benefits: 

• Increased  student  dversity:  The  DM  program  offers  students  that  have  career,  family,  or  geographic  constraints  the 
opportunity  to  obtain  a degree  from  the  1ST  department.  These  students  may  have  no  other  way  of  obtaining  this  degree. 
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• New  job  creation:  With  the  DM  program  in  place,  two  extra  Graduate  Assistant  positions  were  created  to  help 

handle  the  extra  workload.  These  positions  would  not  exist  without  the  DM  program.  Having  these  extra  positions 
available  may  attract  students  who  might  otherwise  attend  a different  institution. 

• Potential  to  revitalize  the  faculty  and/or  curriculum:  Having  a distance  education  program  can  be  a new  source  of 
motivation  for  faculty  and  staff  members  of  a department.  It  can  be  seen  as  something  new  and  exciting.  Also, 
converting  course  material  into  an  online  version  can  inspire  revisions  in  course  content. 

Recommendations 

One  of  the  most  important  steps  in  conducting  an  accurate  cost-benefit  analysis  is  converting  all  the  costs  and  all  the  benefits 
into  monetary  amounts.  This  is  very  difficult,  especially  for  the  value-based  benefits.  In  fact,  the  1ST  department  did  not  have 
figures  for  these  benefits.  When  conducting  a cost-benefit  analysis,  it  is  recommended  to  have  monetary  amounts  established  for 
all  costs  and  all  benefits.  Each  institution  should  determine  the  value  of  these  benefits,  independently. 

It  is  recommended  that  tuition  be  increased  if  the  DM  student  enrollment  is  to  remain  capped  at  20  students.  Presently,  all 
DM  students  pay  residential  tuition  rates.  This  could  be  raised,  especially  for  the  DM  students  who  are  not  residential  students. 
The  $30  DM  fee  could  also  be  raised.  These  changes  in  tuition  and  fees  would  lower  the  costs  to  benefits  ratio. 
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Abstract 

Thirteen  K12  teachers  participated  in  a technology  integration  professional  development  course  that  included  course 
components  such  as  peer  modeling,  peer  collaboration,  and  reflection  in  an  authentic  learning  context.  The  study  purpose  was  to 
explore  how  teachers'  beliefs,  practices,  and  self-efficacy  changed  in  this  learning  environment.  Preliminary  results  indicate 
evolving  teacher  beliefs  and  practices  as  related  to  these  course  components.  Results  also  indicate  a significant  increase  in 
teacher  selfeffiicacy,  based  on  pre-  and  post-course  survey  scores. 

Introduction  and  Background 

While  literature  shows  that  barriers  and  challenges  to  effective  technology  integration  exist  even  among  exemplary  users 
(Becker,  1994),  it  may  be  possible  to  address  some  of  these  barriers  through  professional  development  strategies.  For  example, 
research  suggests  that  peer  modeling  and  reflection  may  be  effective  strategies  to  move  teachers  along  the  technology  integration 
continuum  (Bandura,  1997;  Dwyer,  1996;  Gilmore,  1995;  Schunk,  Hanson  & Cox,  1987),  such  as  moving  from  a traditional  to  a 
constructivist  (or  integrated)  learning  environment  (Grabe  & Grabe,  1 998).  Peer  modeling  of  effective  teaching  and  technology 
integration  strategies  may  result  in  increased  teacher  confidence  and  competence  (Bandura,  1997;  Gilmore,  1995;  Pintrich  & 
Schunk,  1996). 

Further,  providing  models  of  exemplary  teachers  in  a staff  development  setting  may  facilitate  changed  teacher  beliefs  about 
technology  integration  through  structured  exploration  of  those  beliefs  ^rtmer,  Gopalakrishnan,  & Ross,  2000).  Seeing  other 
teachers  similar  to  oneself  in  a successful  technology  integration  capacity  may  cause  one  to  examine  and  possibly  revise  those 
beliefs. 

A related  professional  development  strategy  is  teacher  collaboration,  whereby  teachers  share  ideas  and  strategies  through 
discussion  with  computer-using  peers  (Dwyer,  Ringstaff,  & Sandholtz,  1991;  Hadley  & Sheingold,  1993).  Research  suggests 
that  peers  provide  emotional  and  technical  support  in  the  classroom  (Dwyer  et  al.,  1991;  Hadley  & Sheingold,  1993)  and  are 
found  in  exemplary -user  environments  (Becker,  1994).  If  such  strategies  are  developed  and  implemented,  we  may  increase  the 
likelihood  that  teachers  will  use  classroom  technology  to  enhance  the  critical  thinking  and  problem  solving  abilities  of  school 
children. 

Research  Purpose 

Thirteen  K-12  teachers  participated  in  a professional  development  course,  which  included  components  such  as  peer  modeling 
(including  a CD-ROM  model  of  exemplary  technology -using  teachers),  peer  collaboration,  and  reflection.  The  research  purpose 
was  to  explore  how  this  staff  development  model,  incorporating  such  components  as  reflection,  collaboration,  modeling 
facilitated  changes  in  (1)  teachers’  beliefs  about  technology  integration;  (2)  teachers’  technology  integration  practices;  and  (3) 
teachers’  self-efficacy  beliefs  about  incorporating  technology.  This  study  explores  how  teacher  beliefs,  practices,  and  self- 
efficacy  changed  in  this  authentic  learning  environment. 

The  study  was  guided  by  the  following  research  questions: 

1.  How  do  teachers'  beliefs  about  technology  integration  (e.g.,  role  of  the  teacher,  assessment)  change  using  reflection, 
collaboration,  and  modeling  in  a staff  development  program? 

2.  How  do  teachers'  technology  integration  practices  (e.g.,  assessment  strategies,  curricular  emphases)  change  using  this 
staff  development  model? 

3.  How  do  teachers'  self-efficacy  beliefs  about  integrating  technology  change  using  this  staff  development  model? 

This  paper  will  address  preliminary  results  obtained  from  initial  and  post-course  teacher  interviews,  course  assignments,  and 
a self-efficacy  survey  instrument. 
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Methodology 


Participants 

Thirteen  participants  in  the  technology  integration  professional  development  course  comprised  our  purposive  sample.  Twelve 
teachers  agreed  to  participate  in  the  study  at  various  levels  of  involvement.  Five  teachers  participated  in  the  study  in  a limited 
capacity,  agreeing  to  share  course  assignments  and  complete  surveys  specific  to  self-efficacy  relative  to  technology  integration. 
One  teacher  agreed  to  participate  in  interviews,  surveys  and  to  share  her  course  assignments.  Six  teachers  agreed  to  participate  in 
semi-structured  interviews,  observations  and  a self-efficacy  survey,  as  well  as  share  their  course  assignments. 

The  participants  came  from  four  private  schools  in  a Catholic  diocese  in  a Midwest  city  and  represented  a range  of  grades, 
levels,  and  content  taught.  School  demographics  of  the  course  participants  are  provided  in  Table  1.  Teacher  information  and 
demographics  on  participants  and  their  classes  are  provided  in  Table  2. 


Table  L School  Demographics  for 

Spring  200}  (Names  are  pseudonyms) 

Name 

Student 

Population 

Ethnic 

Makeup 

Grade  Level 

Hilltop 

285 

Approx.  4% 

Pre-K  to  3 

Fairview 

293 

Approx. 

9% 

Pre-K  to  6 

Middleton 

133 

Approx.  4% 

Grades  4-6 

Elm  Creek 

315 

Approx.  3% 

Grades  7-12 

Table  2.  Demographics  for  Research  Participants  ■ 

- Spring  2001  (Names  are  pseudonyms) 

Teacher 

Name 

Research 

Participation 

Level 

School 

Yrs 

Tchg 

Teacher 

Degrees 

Obtained 

Grade 

Level 

Content 

Taught 

Class 

Size 

Classroom  and  Lab 
Computer  Resources 

Caroline 

Extensive 

Hilltop 

23 

M.S., 

Elementary 

Education 

3rd 

All  subjects 

Began 

with 

22; 

ended 

with  19 

Mac  and  PC  available; 
3 classroom 
computers;  printers, 
digital  camera,  scanner 

Clara 

Extensive 

Hilltop 

9 

B.S., 

Elementary 
Education  and 
Language 
Arts 

1st 

All  subjects 

18 

Five  computers; 
scanner,  printer, 

2 IBM  computers, 
1 I-Mac,  2 Macs 

Greta 

Extensive 

Hilltop 

18 

M.S., 

Education 

2nd 

All  subjects 

19 

2 I-Macs,  printer 

Kathy 

Extensive 

Fairview 

6 

B.S., 

Elementary 

Education 

3rd 

All  subjects 

Began 

with 

18; 

ended 
with  19 

1 I-Mac,  1 Apple,  2- 
Emates,  printer 

Table  2 (Cont  W. ).  Demographics  for  Research  Participants  - Spring  200}  (Names  are  pseudonyms) 


Teacher 

Research 

School 

Yrs 

Teacher 

Grade 

Content 

Class 

Classroom  and  Lab 

Name 

Participation 

Level 

Tchg 

Degrees 

Obtained 

Level 

Taught 

Size 

Computer  Resources 

Eleanor 

Extensive 

Fairview 

10 

B.A., 

4th 

All  subjects 

26 

2 Apples,  2 Macs,  and 

Elementary  a printer. 

Education  Library:  2 I-Macs, 

approx.  10  E-Mates. 
Several  people  in 
building  to  ask  for 
help. 


BEST  COPY  AVAILABLE 


Jennifer 

Extensive 

Elm  Creek 

15 

E.S.,Math. 

Julia 

Extensive 

Elm  Creek 

5 

E.S.,  Home 
Economics 

Amelia 

Limited 

Elm  Creek 

25 

E.A.,  Social 
Studies  Ed., 
M.A.,U.S. 
History 

Anne 

Limited 

Elm  Creek 

Masters, 

English 

Education 

Ruth 

Limited 

Middleton 

47 

E.A., 

Elementary 
Ed.,M.A., 
Elem.  Admin. 

Martha 

Limited 

Middleton 

13 

Masters, 
Admin,  and 
Supervision 

Sarah 

Limited 

Middleton 

E.S., 

Elementary 
Ed.,  E.S.  in 
Special  Ed. 

1'^  and  g* 

Math  and 
Geometry 

21 

Computer,  scanner. 
Fax,  printer;  access  to 
computer  lab 

7*^  and  8*^ 

Family 

Living; 

Clothing 

Construction 

17-18 

(Est.) 

Access  to  computer 
lab 

9-1 1 

U.S.  History, 

World 

History 

Access  to  computer 
lab. 

11-12 

English 

Literature 

- 

Access  to  computer 
lab. 

4th 

All  subjects 

— 

2 I-Macs  in  class 
room;  28  Macs  in 
computer  lab 

Principal 

(18yrs) 

- 

NA 

I-Macs;  access  to 
computer  lab 

jU. 

All  subjects 

- 

2 I-Macs  in  classroom; 
access  to  computer 
lab. 

Research  Design 

The  study  was  primarily  qualitative,  using  a case  study  methodology.  Quantitative  and  qualitative  data  were  gathered 
specific  to  self-efficacy  about  classroom  technology  use.  The  qualitative  data  were  gathered  to  explore  and  describe  teacher 
beliefs  about  technology  integration,  changes  in  technology  goals  and  teaching  practices,  and  their  self-efficacy  beliefs  with 
regard  to  technology  use.  Quantitative  data  were  also  gathered  to  examine  teachers'  levels  of  self-efficacy  with  regard  to 
classroom  technology  use.  Data  will  continue  to  be  collected  during  the  Fall  2001  semester  to  allow  time  for  changes  in  beliefs, 
perceptions,  and  practices  to  emerge. 

Procedure 

The  Spring  2001  semester-long  professional  development  course  was  conducted  once  a week  in  three-hour  sessions.  Using 
electronic  and  peer  models  of  technology  integration,  the  course  was  set  up  to  facilitate  discussion  about  technology  integration 
issues  and  to  identify  different  strategies  that  might  be  used  in  the  teachers’  classrooms. 

The  course  components  included  (1)  presentation  of  information  on  problem-based  learning  (PEL),  (2)  a series  of  facilitated 
discussions,  (3)  presentation  of  the  electronic  models  with  related  discussions,  (4)  course  leadings,  and  (5)  the  collaborative 
development  of  a technology  -based  PEL  unit.  These  course  components  were  based  upon  ideas  and  strategies  that  the  teachers 
found  useful  from  their  experiences  and  knowledge  gained  during  the  course  discussions  and  activities.  The  teachers  also  were 
asked  to  submit  reflections  on  the  various  parts  of  the  technology  -based  PEL  unit. 

Instruments 

Primary  data  sources  included  initial  and  post-course  teacher  interviews,  observations  of  teacher  classrooms,  surveys,  course 
discussions,  and  course  assignments.  Course  assignments  included  teachers’  visions  of  themselves  as  technology -integrating 
teachers  (at  course  outset  and  course  end),  development  of  a technology  -based  PEL  unit  for  their  classrooms  and  accompanying 
reflections.  During  the  class  sessions,  group  discussions  occurred  specific  to  teacher  ideas  about  what  they  were  seeing  and  how 
that  tied  in  with  their  views  and  goals. 

Initial  and  post-course  interviews  were  conducted  to  learn  about  changing  teacher  visions,  beliefs  (i.e.  teacher  views  about 
classroom  organization  and  management,  assessment)  and  practices  specific  to  technology  integration,  as  well  as  changes  in  self- 
efficacy.  The  first  interview  was  conducted  in  February  2001,  approximately  one  month  after  the  outset  of  the  course;  the  second 
interview  was  conducted  in  June  of  the  same  year,  the  week  following  the  end  of  the  course.  Two  interviews  were  planned  for 
the  following  term  (Fall,  2001)  to  continue  to  explore  evolving  teacher  beliefs  and  visions,  classroom  practices,  and  changes  in 
self-efficacy. 

Observations  were  conducted  during  the  course  to  observe  class  activities  and  teacher  discussions.  In  addition,  classroom 
observations  of  the  key  participants  were  conducted  on  a weekly  basis  to  observe  teacher  classroom  practices  and  technology  use. 

Teachers  also  completed  a pre-  and  post-course  online  survey  relative  to  teacher  confidence  (self-efficacy).  This  survey  is 
currently  being  administered  mid-term  Fall  2001  to  revisit  changes  in  teacher  confidence. 
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This  survey  instrument,  specific  to  teacher  confidence  about  their  technology  integration  practices  (self-efficacy),  is  an 
adaptation  of  an  instrument  developed  and  tested  in  Fall  2000  (Ertmer,  Conklin,  Lewandowski,  Osika,  Selo,  & Wignall).  The 
constructed  instrument  has  three  categories  (planning  for  classroom  technology  use,  technology  classroom  implementation,  and 
assessment  of  classroom  technology  use  and  impact,  respectively),  with  10  items  each,  for  a total  of  30  items.  The  instrument, 
based  on  a five-point  scale,  asks  for  responses  ranging  from  “Very  confident”  to  “Not  at  all  confident.”  Item  examples  follow: 
Relative  to  planning  for  technology  use,  I am  confident  that  I can: 

1)  define  teacher/student  roles  in  a technology  -integrated  classroom. 

2)  plan  classroom  activities  that  facilitate  technology  integration. 

3)  plan  for  the  use  of  computers  with  large -group  instruction. 

Data  Analysis 

Introduction 

Initial  course  assignments  have  been  analyzed,  as  have  self-efficacy  data.  Specific  to  teacher  confidence  (self-efficacy),  the 
preliminary  analysis  has  been  conducted  on  initial  and  end  of  term  teacher  interviews  and  surveys. 

Quantitative  Data  Analysis 

Data  were  analyzed  for  the  12  course  participants,  based  on  their  pre-  and  post-course  survey  results.  Paired  t tests  were 
conducted  for  the  self-efficacy  survey  instrument;  reliability  was  also  measured,  using  Cronbach’s  alpha. 

Qualitative  Data  Analysis 

Interview  data  (pre-  and  post -course)  for  the  key  participants  (seven  teachers)  and  pre-  and  post -course  teacher  profiles  on 
teacher  technology  visions  (all  12  course  participants)  were  analyzed  and  coded  inductively  specific  to  teacher  beliefs,  practices, 
and  self-efficacy  (confidence),  using  cross-case  and  within-case  analysis.  Examples  of  emergent  codes  include  “Contributors  to 
Learning,”  “Technology  Practice,”  and  “Contributors  to  Confidence."  This  coding  was  done  using  Atlas.ti®,  a qualitative 
analysis  software  package  produced  by  Scientific  Software  Development. 

Results  and  Discussion 

Quantitative  Data  Results 

At  the  course  outset,  teachers’ self-efficacy  scores  averaged  from  1. 1 to  3.0,  with  a mean  score  of  2.2  (five-point  scale)  and  a 
standard  deviation  of  0.67.  At  the  end  of  the  course,  teachers’  scores  averaged  from  1.2  to  4.0,  with  a mean  score  of  3.4  and 
standard  deviation  of  1.0.  Using  the  paired  t-test  on  pre-  and  post-course  data,  a value  of  6.66  was  obtained,  significant  at 
0.0001.  Instrument  reliability  for  the  self-efficacy  instrument,  based  on  pre-course  data  was  0.98;  post-course  reliability  was 
0.99. 

Qualitative  Data  Results 

Preliminary  data  analysis  suggests  the  following  emergent  themes: 

• Contributors  to  learning  (including  components  of  the  staff  development  model)  viewed  as  useful  varied  with  individual 
teachers,  although,  the  more  “active”  types  of  learning — peer  models  and  collaboration,  hands-on  experiences  working 
on  the  PBL  technology  unit,  and  class  discussions  were  mentioned  most  often  as  contributors. 

• There  are  indications  of  some  teachers'  revising  their  beliefs  with  regard  to  technology  (the  role  of  the  student,  for 
example). 

• Qverall,  at  course  end,  teachers  reported  an  increase  in  confidence  with  regard  to  technology  use.  Contributors  to  this 
increased  confidence  included  knowledge  increase,  hands-on  experience,  peer  support,  and  feelings  of  accomplishment. 
Confidence  indicators  included  experimentation  (willingness  to  experiment  with  technology  in  the  classroom)  and 
increased  student  technology  use. 

• Teacher  technology  practices  included  increased  student  technology  use,  including  using  their  students  to  teach  them. 
Teachers  still  expressed  concern  about  assessment,  classroom  management  and  classroom  organization  with  regard  to 
technology  use. 

Contributors  to  Learning:  Consideration  of  the  Staff  Development  Model 

Course  components  valued  as  learning  contributors  varied  with  the  individual  teacher.  The  strongest  contributors  to  learning, 
based  on  post -course  interviews  with  those  teachers  who  participated  extensively  in  the  research,  appeared  to  be  peer 
collaboration  and  peer  models  (other  teachers),  hands-on  experiences,  and  class  discussions.  To  a somewhat  lesser  extent,  the 
course  reflections,  course  readings  and  the  electronic  peer  model  (CD-RQM)  were  also  mentioned  as  being  contributors. 

Clara,  Greta  and  Caroline,  all  teachers  at  Hilltop  Elementary  in  grades  1-3,  respectively,  collaborated  together  on  their 
technology -based  PBL  unit.  Qf  the  contributors  to  their  learning,  Clara  and  Greta  both  mentioned  peer  collaboration  as  being 
valuable.  According  to  Clara: 

I think  the  biggest  thing  was  the  peer  collaboration,  because  Caroline  was  so  wise,  in,  you  know,  in  her  knowledge.,  .where 
Greta  and  I were  lacking... even  the  confidence..  .So  that,  and  feeling  that  we  had  someone  that  did  know  part  of  what  was  going 
on;  and  it  was  neat  working  with  Caroline  because  we  found  out  she  didn't  know  everything.  And  some  of  the  things  we 
learned  together.  [Post-Course  Interview,  June  4,  6,  2001] 

Greta  also  appreciated  the  support  that  Caroline  gave  to  her.  Further,  it  appears  that  Caroline  (who  acted  as  an  informal 
technology  coordinator  for  the  school)  acted  as  a peer  model  for  Greta. 
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. . .1  think  Caroline  was  such  an  awesome. , .leader  for  us.  She  gave  us  a lot  of— but,  at  the  same  time,  a lot  of  the  things 

she  did — we  just  watched. . .And  so,  Tm  hoping  that  I can  take  what  I saw  her  doing  and  try  and  do  myself  [Post-Course 
Interview,  June  4,  6,  200 1 ] 

Greta  found  that  her  contributors  to  learning  included  the  electronic  peer  model  and  the  class  discussions,  as  well  as  the 
hands-on  experiences  provided. 

Hands  on-I  liked  the  opportunity  to  do  the  things, , .but  class  discussions,  seeing  what  the  other  people  were  thinking  and 
feeling  and-and  doing  in  class.  Like  we  would  watch  a video. . .where  we  would  go, . .and  see  some  of  the  teachers  in  action, 
[Post-Course  Interview,  June  4,  6,  2001] 

Caroline  found  that  her  contributors  to  learning  were  based  on  the  course  readings,  the  class  discussions  and  the  reflections. 
With  regard  to  the  reflections: 

And  the  thing  that's  least— we  griped  about  those  reflections,.,But,,.it  did  make  you  think,.,and— and  then  I would  re- 
examine something-even  though  I'd  said  it  two  reflections  before,  now  maybe  I'm  looking  at  it  in  a little  different  way,,,or, 
um,  I see  where  I can  improve  it  or  know  that  it's  not  even  possible, , , [Post  Course  Interview,  June  4,  6,  200 1 ] 

At  Fairview,  Kathy  (a  third  grade  teacher)  and  Eleanor  (a  fourth- grade  teacher)  also  collaborated  together  on  their  project, 
Kathy  found  working  with  Eleanor  helpful,  as  well  as  the  class  discussions. 

Well,  definitely  the  class  discussions,  especially  the  one  that  we  had  on  the  security  issues  and  privacy,,,!  liked  that  one  a 
lot,  Um,  and  working  with  Eleanor  helped  a lot.. .She  -she'd  always  have  so  many  great  ideas -I  love  working  with  her.  [Post- 
Course  Interview,  June  6,  200 1 ] 

Eleanor  found  the  hands-on  experiences  of  the  project  most  useful:  "...All  of  that  contributed.  And,  but  actually  doing 
it... .The  hands-on  was  the  most-yeah,  for  me...”  [Post-Course  Interview,  June  7,  2001] 

It  is  interesting  to  note  that  these  teachers  overall  selected  more  “active”  methods  of  learning— hands-on  experiences, 
working  with  others,  participating  in  class  discussions.  While  these  components  were  mentioned  frequently,  it  should  also  be 
noted  that  reflections  and  course  readings  were  also  mentioned  as  learning  contributors.  It  is  also  possible  that  the  electronic  peer 
model  (which  was  used  in  the  course  a couple  of  times)  might  have  been  a stronger  contributor  with  more  use. 

Changing  Teacher  Beliefs 

For  some  teachers,  existing  teacher  beliefs  about  technology  changed  or  were  enhanced  with  regard  to  technology  use,  Clara 
learned  early  in  the  term  that  technology  could  be  used  as  a tool,  rather  than  her  earlier  conceptions  of  how  she  should  use  it  in 
the  classroom. 

I've  learned,  mainly,  that  technology  is  a tool;  and  I think  before  I thought,  in  reading  these  things  and  really  thinking 
about  it,  I was  thinking  technology  more  as  a subject?,,, And  that-  that -I  teach  across  curriculums,,,!  was  looking  at  technology 
as  another  subject  area,  rather  than  seeing  is  a tool  that  enhances  subjects  areas  I already  have,  so  that's  where  my  whole 
thinking  has  changed,  and  I can  see  now  that  this  can  be  a really  great,  effective  way  to  teach  some  of  the-my  areas  to 
enhance  them...  [Initial  Interview,  February  8,  2001] 

When  asked  at  course  end  what  the  main  thing  that  she  had  learned  was,  Clara  reiterated,  “All  right,  the  biggest  thing  for  me 
is  that  I know  the  computer  is  a tool.  Where  before,,,!  had  the  idea  that  they  wanted  me  to  use  the  computer  for  almost 
everything.”  [Post-Course  Interview,  June  6,  2001] 

Although  her  beliefs  about  the  value  of  technology  had  not  changed,  Jennifer,  a grade  Math  teacher,  spoke  of  revising  or 
revisiting  her  beliefs  about  the  role  of  the  teacher,  specifically  teacher-directed  learning  in  the  classroom.  Class  discussions  and 
course  readings  reinforced  her  beliefs. 

To  a certain  degree  I would  have  to  say  that  my  ideas  about  technology  have  not  changed  that  much.  Prior  to  this  course  I 
felt  technology  was  an  important  tool  for  educators  to  use  to  help  better  educate  our  students,  I think  that  the  ideas  we  have 
discussed  and  the  material  that  I have  read  have  just  reinforced  this  belief  In  fact,  I am  more  motivated  and  convinced  that  I 
need  to  find  even  more  areas  to  implement  technology  into  the  curriculum. 

As  a result  of  developing  my  problem-based  unit,  I have  learned,  or  been  reminded,  that  technology  makes  learning  more 
student -centered,  I think  I would  be  very  naive,  even  wrong,  to  believe  that  all  students  learn  best  in  a teacher-centered 
classroom,  I have  found  myself  asking  "Do  the  students  really  need  me  to  show  them  how  to  solve  a linear  equation  (any  math 
concept  for  that  matter)  or  could  they  discover  this  on  their  own?"  [Technology  Vision  Revisited,  April  24,  2001] 

Julia  found  that  the  course  reflections  helped  her  reach  an  insight  about  her  students  and  active  teaching  methods,  with  regard 
to  the  student  role  in  the  classroom  and  the  benefits  to  them  long-term. 

When  Amelia  and  Jennifer  and  I were  working  on  the  PBL  Group  Presentation,  we  had  three  different  ideas  as  to  what  was 
expected  of  us,  three  different  propositions  on  how  to  approach  it  and  three  different  thoughts  on  how  to  organize  it,,,  I 

remember  commenting  that  I wanted  my  classwork  organized  for  me  and  presented  in  a neat  little  package  because  the  whole 
rest  of  my  life  consists  of  problem-based  learning.  As  the  semester  unfolded,  I became  aware  of  how  very  true  this  was  of  me 
and  the  adults  around  me.  What  better  way  to  teach  my  students  skills  for  life  than  to  engage  them  early  in  PBL!  Could  I really 
teach  them  anything  more  important?  [Technology  Vision  Revisited,  April  24,  2001] 

Teacher  beliefs  are  not  easily  changed,  and  traditional  roles  in  the  classroom  can  be  hard  to  overcome  (Ertmer  et  al,,  2000; 
Fullan,  1993;  Schrum,  1999;  Van  Haneghan  & Stofflett,  1995),  As  Schrum  stated,  "Teachers  need  compelling  reasons  to  change 
their  practice"  (1999,  p,  85),  In  the  above  examples,  it  appears  that  the  components  of  the  model  may  be  helping  some  teachers 
consider  and  possibly  revise  their  beliefs  with  regard  to  classroom  technology  integration. 


345 


354 


Increased  Confidence.  Confidence  Contributors  and  Indicators 

Post-course  interviews  and  post-course  teacher  profiles  (visions  of  technology -integrating  teachers)  indicated  increases  in 
confidence  for  five  out  of  seven  course  participants  with  regard  to  classroom  technology  use.  Contributors  to  this  increased 
confidence  included  knowledge  increase,  hands-on  experience,  support,  and  feelings  of  accomplishment,  as  well  as  the 
opportunity  (and  willingness)  to  experiment  with  technology  in  the  classroom. 

For  Kathy,  the  third  grade  teacher  at  Fairview,  an  increase  in  knowledge  also  related  to  her  confidence  in  classroom 
technology  use,  as  well  as  willingness  to  ask  for  support. 

I still  believe  that  technology  is  a wonderful  and  exciting  way  to  teach  and  motivate  students.  I feel  somewhat  less 
overwhelmed  by  the  amount  of  information  and  software  available.  I have  more  confidence  in  myself  and  my  abilities  to 
utilize  technology.  In  addition,  I no  longer  feel  embarrassed  to  ask  for  help  when  I need  it.  [Technology  Vision  Revisited, 
April  24,  2001] 

According  to  Ruth,  a fourth  grade  teacher  at  Middleton,  fear  and  lack  of  knowledge  has  been  replaced  by  less  fear,  more 
knowledge,  and  a sense  of  accomplishment,  as  well  as  experimenting  with  technology  in  her  classroom. 

I have  to  admit,  I came  into  this  class  fearful,  apprehensive  and  knowing  - not  much  — about  computers  or  their  use. 

Now  I have  to  admit  that  I have  enjoyed  this  work  and  I am  definitely  not  as  fearful  about  computers  as  I was  at  the  beginning. 
I even  try  things  and  often  am  successful  at  what  I try  e.g.  making  vocabulary  charts  —using  e-  mail  — using  the  internet.  I can 
even  talk  a little  intelligently  about  some  areas  in  the  use  of  the  computer.  [Technology  Vision  Revisited,  April  24,  2001] 

Ruth’s  successes  with  technology  use  appeared  to  result  in  increased  classroom  technology  use: 

Since  I’m  less  afraid  and  apprehensive  I've  changed  my  mind  about  computer  use  in  the  classroom  too.  I have  learned  to  use 
the  internet  productively.  Finding  work  that  fits  with  my  class  especially  for  science.  I have  encouraged  the  students  to  use 
the  internet  for  reports  for  class.  I have  used  the  computer  to  make  charts  for  classroom  use.  I am  planning  to  use  the 
computer  for  my  grading  program  this  fall.  [Technology  Vision  Revisited,  April  24,  2001] 

Peer  support  and  accomplishment  continued  to  contribute  to  confidence,  as  well  as  hands-on  experience.  Greta  found  that 
working  with  Clara  and  Caroline  helped  her  confidence  and  her  learning,  as  did  the  hands-on  experience.  When  asked  about 
confidence  contributors,  Greta  answered: 

Yes,  and  doing  it  with  Caroline,  you  know. . .and  working  on  our — just  getting  that  whole  page  together,  as  I just  said,  it 
turned  out  marvelously  because  of  Caroline,  I think... But —cause  having  her- in  the  group,  that  made  me  be  able  to  say,  "Yeah,  I 
can  do  that,"  you  know,  or  "How  did  you  do  it,  Caroline?". . .Trying  to  get  so  that  I would  learn  from  her.  [Post-Course 
Interview,  June  4,  2001] 

Kathy  believed  that  the  changes  in  her  classroom  could  be  attributed  to  her  increased  confidence  and  hands-on  experience, 
which  led  to  her  increased  technology  use  and  experimentation. 

I guess,  mainly  my  confidence  has  gone  up,  so  that  created  a change.  And  that  started  with  the  discussions  and. . .actually 
getting  in  and  using  the  computers,  and  finding  new  things  to  do  with  the  computers  myself  [Post-Course  Interview,  June  6, 
2001] 

As  their  technology  journey  continued,  teacher  confidence  with  regard  to  technology  use  increased  overall,  due  in  part  to 
hands-on  experiences,  support,  increased  knowledge,  and  feelings  of  accomplishment.  Indicators  of  such  confidence  were 
reflected  in  their  encouragement  of  and  confidence  in  increased  student  computer  use  and  willingness  to  experiment  with 
technology. 

Teacher  Technology  Practices 

Perhaps  in  part  due  to  their  overall  increased  confidence  with  using  technology  in  the  classroom,  many  teachers  are  trying  out 
new  ideas  with  regard  to  technology.  In  doing  so,  they  are  addressing  challenges  such  as  classroom  organization  and 
management,  as  well  as  time  constraints. 

Clara  described  a successful  use  of  technology  for  her,  one  that  she  had  only  tried  out  that  term  with  her  students: 

Each  one  wrote  a little,  um,  paragraph  about  themselves. . .and  then  they  had  to  find  some  Clip  Art  or  something  that 
looked... that  showed— reflected  them... and  put  it  on  their— their  paper... and  print  it  out.  And  we  did  get  that  done!  So  that 
was  successful...!  think  because  the  children  enjoyed  it  so  much.  They  were  able  to  follow  the  directions.  They  ended  up  with  a 
good  product. . .and  they  had  fun.  [Post-Course  Interview,  June  4,  6,  2001] 

Clara  also  had  her  students  working  on  researching  a dinosaur  project  on  the  Internet,  using  the  students  to  help  each  other. 
What  I tried  to  do  there  is  put  someone  that  was  more  computer-literate,  someone  that  could  read  well. . .with  someone  that 
was  a little  bit  lower... so  they  could  work.  So  sometimes,  I didn't  have  as  many  high  students,  so  sometimes  the  high  student 
would  work  with  three  or  four.,  .students  at  different  times.  [Post-Course  Interview,  June  4,  6,  2001] 

Eleanor  had  her  students  teach  her  how  to  use  KidPix,  preparatory  to  using  her  technology  unit,  which  she  had  developed 
with  Kathy,  the  following  term:  In  doing  so,  she  and  her  students  used  KidPix  to  run  ads  on  the  VCR  monitor  during  their  Mini- 
Economy  activity: 

Um,  I've  kind  of— we've  had  classes  before  on  technology,  so  I was  aware  of  a lot  of  this  stuff,  but  planning  our  unit,  we  were 
planning  on  using  KidPix  and  I had  never  done  that  before. . .and  so  I did  go  ahead  and,  um,  look  at  it  and  then  use  it  at  the  end 
of  this  year,  so  that  I'll,  um,  be  more  able  to  jump  into  the  unit  next  year. 

hat  was  a lot  of  fun.  And  here,  um,  a couple  of  my  student  experts  helped  me  cause. . .they  had  it  at  home  and  helped  me 
how  to  use  it  and,  um,  so  I had  no  idea  what  I was  going  to  do  with  the  kids,  but  we  ended  up  doing,  um,  our  ads  with  our 
Mini-Economy  on  there... with  the— the  music  and  the— the  slide  show  and  that— that  was  really  cool.  [Post-Course  Interview, 
June  7 2001] 
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Kathy  began  organizing  her  classroom  and  time  such  that  her  students  had  greater  access  to  and  time  to  work  on  the 
computer. 

Urn,  we  set  up  a time  during  the  day  when  people  would  have  to  go  back.  They  couldn’t  just... you  know — because 
sometimes  some  of  the  kids  are  intimidated  by  it,  and  they  would... kinda  say,  No,  I don’t  want  to  today.  I’ll  do  something 
else,”  but... we’ll  say,  “Yes.  You  go.”  [Post-Course  Interview,  June  6,  2001] 

As  might  be  expected,  teachers  are  stfll  evolving  in  various  aspects  of  technology  practice.  Caroline  expressed  concerns 
about  assessment: 

You  know,  I still-I  was -the  assessment  issue  is  still  something  that-I  feel  better  about  it.  I don't  know  that  it's 
something  I do  as  well  as— as  I could  or  should.  Sometimes  I fly  by  the  seat  of  my  pants.  [Post-Course  Interview,  June  4,  6, 
2001] 

Greta  discussed  her  challenges  in  dealing  with  classroom  management,  ensuring  that  all  students  had  the  opportunity  to  work 
with  the  computers. 

I think  that-yeah-the  challenge  is — is  still  how  to  give  everyone  the  opportunity.  How  to  utilize 
my  time. . .The  management  of  it.  Urn,  this  room  is  small. . .And  so,  my  stations  were  never  really— so  they  went  to  one 
station  and  did  this,  whereas  in  our  project,  t he  whole  thing  set  up.  So  there  would  be  four  different  groups  or  so  and  they 
would  be  doing  different  things,  so  that  I would  be  working  with  one  group. . .hopefully  at  the  computer,  or  moving  about  to 
help  some  of  the  others,  but  I was  hoping  to  make  the  groups  such  that  there  would  be  a leader  type  in  each  one,  who  could  keep 
the  group  kind  of  going.  [Post-Course  Interview,  June  4,  2001] 


Conclusions  and  Future  Implications 

Impact  of  the  Staff  Development  Model 

It  appears  that  the  course  components  were  (perhaps,  not  surprisingly)  of  value  to  different  teachers  in  different  ways.  The 
hands-on  experience,  as  part  of  the  PEL  technology  unit,  was  mentioned  often  as  being  useftil  in  terms  of  increased  knowledge 
and  confidence.  Peer  collaboration  was  particularly  useful  to  those  who  looked  to  “leaders”  to  help  them  in  skill  building  and 
developing  web  pages.  Reflections  were  useful  for  “rethinking”  where  they  were  and  where  they  are  going.  In  some  cases, 
reflections  may  have  contributed,  in  part,  to  changing  teacher  beliefs. 

Overall,  the  impact  of  the  staff  development  model  appeared  helpful  in  terms  of  changing  teacher  beliefs  and  practices. 
Participants  did  not  directly  say,  “This  component  helped  me  change  my  practice.”  However,  in  the  sense  that  the  various 
components  contributed  to  increased  knowledge  and  confidence,  it  may  be  termed  useftil.  Pre- course  profiles  indicated  that 
teachers  vary  from  traditional  teaching  approaches  toward  more  integrated  approaches  to  classroom  technology  use  (Grabe  & 
Grabe,  1996).  While  further  analysis  is  needed  to  confirm  these  initial  results,  this  may  lead  to  an  interesting  contrast  relative  to 
changes  that  may  occur  as  a result  of  the  professional  development  course,  given  the  diversity  of  approaches. 

Further  Data  Analysis  Needs 

Initial  interview  data  with  seven  teachers  appeared  to  indicate  that  their  school  cultures  encourage  collaboration  and 
reflection,  with  some  teachers  also  referring  to  other  teacher  "models."  Given  the  existing  culture,  it  will  also  be  interesting  to 
see  how  teacher  beliefs  and  technology  integration  practices  are  facilitated  by  such  cultures,  in  conjunction  with  (and  supported 
by)  the  professional  development  model,  which  also  encourages  collaboration,  reflection,  and  modeling.  Further  analysis  is 
needed  to  confirm  these  impressions. 

As  indicated  above,  further  analysis  remains  to  be  done  on  course  data  obtained,  as  well  as  teacher  interviews  and 
observations.  While  preliminary  data  analysis  has  focused  on  teacher  beliefs,  practices,  and  self-efficacy,  as  well  as  learning 
contributors,  results  are  yet  to  be  determined  in  greater  depth. 

With  regard  to  efficacy  data  analyzed  to  this  point,  results  are  consistent  with  those  of  Stein  and  Wang  (1988).  While  many 
studies  have  examined  teacher  efficacy  in  professional  development  settings,  often  self-efficacy  is  measured  at  one  point  in  time 
(Ross,  1995).  However,  research  has  shown  that  teacher  efficacy  may  be  curvilinear  across  time  (Stein  & Wang,  1988). 
According  to  Stein  and  Wang  (1988),  teacher  self-efficacy  may  increase  during  a professional  development  course,  then  decrease 
as  the  teacher  attempts  to  put  what  was  learned  into  practice,  then  later  increase  again  as  the  teacher  masters  what  was  learned.  It 
may  then  be  useful  to  measure  teacher  self-efficacy,  not  only  during  a professional  development  course,  but  also  as  a follow-up 
(as  in  the  following  term)  in  the  teachers'  classrooms. 

From  the  results  on  self-efficacy  collected  thus  far,  it  appears  to  support  Stein  & Wang’s  (1988)  conclusions.  The  survey 
instruments  are  currently  being  completed  by  the  teacher  participants  (mid-term  Fall  2001),  and  these  data  will  contribute  to  the 
knowledge  base  on  the  curvilinearity  of  the  results  specific  to  teacher  self-efficacy,  as  well  as  idea  generation.  Repeated 
measures  data  may  confirm  (or  disconfirm)  the  nature  of  teacher  efficacy  across  time. 
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LARGE  SCALE  INTERVENTIONS: 

AN  HISTORICAL  CASE  STUDY  OF  FLORIDA  SCHOOLYEAR  2000 
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Abstract 

Throughout  the  ages,  men  and  women  have  struggled  with  educating  their  young  in  the  most  effective  and  efficient  ways 
possible.  They  have  learned  the  value  of  education  is  not  a unique  interest  of  a family,  community,  state,  or  nation.  Education  is 
now  embraced  by  a global  society  and  serves  as  the  equalizer  for  the  betterment  of  all.  To  meet  this  responsibility,  it  becomes 
society's  challenge  and  opportunity  to  identify,  design  and  develop  interventions  that  provide  quality  education.  Therefore,  this 
research  study  focuses  an  historical  case  study  on  a large-scale  intervention  called  Florida  SchoolYear  (SY2000),  which  was  a 
visionary  example  of  how  the  state  of  Florida  tried  to  design  and  develop  a means  for  delivering  quality  education. 

By  studying  the  Florida  SY2000  Initiative,  this  research  study  examined  issue-oriented  questions  (e.g.,  impacts  and 
influences)  that  confronted  those  individuals  and  groups  who  sponsored  and  advocated  a large-scale  reform  effort  designed  to 
drive  educational  change  in  Florida  K-I2  schools.  Conveying  the  research  findings  through  an  historical  case  study  allowed  the 
story  to  unfold  as  each  informant  had  his/her  own  perspective  and  story  to  tell,  from  which,  collectively,  patterns, 
understandings,  and  recommendations  were  drawn. 

The  outcome  of  this  research  study  was  the  presentation  of  SY2000  sponsors ' and  advocates ' perceptions  of  what  happened 
during  SY2000,  what  they  believed  should  have  happened,  and  what  they  wished  would  happen  in  future  education  reform 
efforts.  These  individuals  were  selected  based  on  their  leadership  or  influential  roles  and  positions  held  during  SY2000. 

Data  that  emerged  via  surveys  and  interviews  addressed: 

• Factors  that  affected  the  implementation  of  SY2000  (e.g.,  differing  cultures,  leadership,  vision,  buy-in,  sustained 
commitment,  and  change  process); 

• Costs  of  implementation  factors  (e.g.,  research  and  development,  allocation  of  resources,  measurable  results,  time,  and 
technology  use  and  support);  and 

• ‘‘Lessons  learned"  (e.g.,  perceived  effects  of  SY2000,  obstacle  of  misaligned  cultures,  value  of  research  and 
development,  importance  of  deciding  whether  to  redesign  or  recreate,  criticality  of  having  shared  vision  and  mission, 
and  the  power  of  strong  leadership). 

The  researcher  presented  limitations  of  the  research  study  and  future  research  dealing  with  large-scale  interventions.  The 
researcher  concluded  this  study  by  challenging  the  reader  to  a new  way  of  thinking  in  terms  of  education  reform  (Russo- 
Converso,  200 J). 

Introduction 

This  historical  case  study  focused  on  a large-scale  intervention  called  Florida  SchoolYear  (SY2000),  which  was  a visionary 
example  of  how  the  state  of  Florida  tried  to  design  and  develop  a means  for  delivering  quality  education.  SY2000  was  a seven- 
year  (1989-1996)  collaborative  program  among  nine  Florida  districts,  the  Florida  Department  of  Education,  the  Florida 
Department  of  Health  and  Rehabilitative  Services,  and  the  State  University  System,  led  by  Florida  State  University  (FSU).  The 
purposes  of  SY2000  were  to  redesign  and  implement  a technology -supported  system  of  schooling.  Total  funding  for  the  program, 
including  investments  made  by  private  corporations  and  district  “in-kind”  contributions,  exceeded  $30  million. 

The  research  revealed  an  in-depth  look  at  how  the  experience  of  SY2000  influenced  SY2000  sponsors  and  advocates. 
Respondents’  stories  explained  what  happened  during  SY2000,  what  they  thought  should  have  happened,  and  what  they  wish 
would  happen  in  future  reform  efforts. 

Therefore,  the  focal  points  of  this  case  study  centered  on  SY2000  sponsorship  and  advocacy  and  the  “lessons  learned”  by 
those  individuals  having  had  those  roles.  For  the  sake  of  this  study,  definitions  of  sponsors  (initial  and  sustaining)  and  advocates 
were  based  on  the  work  of  change  management/leadership  expert  Daryl  Conner  and  findings  he  presented  in  Managing  at  the 
Speed  of  Change  (1992)  and  Leading  at  the  Edge  of  Chaos  (1998). 

A sponsor  is  defined  as  an  individual  or  group  who  has  the  authority  to  legitimize  and  power  to  enforce  the  intervention  (e.g., 
the  Florida  State  legislature).  Sponsorship  involves  far  more  than  ideas  and  rhetoric;  it  requires  the  ability  and  willingness  to 
apply  the  meaningful  rewards  and  pressure  that  produce  desired  results.  Major  change  will  not  occur  unless  appropriate  sponsors 
demonstrate  sufficient  commitment.  There  are  two  kinds  of  sponsors,  initial  and  sustaining.  An  initial  sponsor  is  defined  as  an 
individual  or  group  who  has  the  power  to  break  with  the  status  quo  and  sanction  a significant  change.  An  initial  sponsor  is 
usually  higher  in  the  hierarchy  than  those  who  must  perform  the  duties  of  sustaining  sponsors  (Conner,  1992).  A sustaining 
sponsor  is  defined  as  one  who  supports  and  follows  through  with  the  sponsor  commitment  and  allocation  of  resources  for  his/her 
arena  of  influence.  A sustaining  sponsor  has  enough  proximity  to  local  targets,  those  individuals  or  groups  who  must  actually 
change,  to  maintain  focus  and  motivation  on  the  change  goals  (e.g.,  Florida  State  Department  of  Education  or  SY2000 
operational  test  site  school  superintendents).  Sustaining  sponsors  minimize  logistic,  economic,  or  political  gaps  that  exist  between 


349 


358 


BEST  COPY  AVAILABLE 


layers  of  the  organization  (education  system)  and  produce  the  appropriate  structure  of  rewards  and  punishments  that  promote 
achievement  (Conner,  1992). 

An  advocate  is  defined  as  an  individual  or  group  who  wants  to  achieve  a change  but  lacks  the  power  to  sanction  it.  However, 
advocates  are  influential  and  valued  for  the  advice  and  recommendations  given  to  the  sponsor  and  others  (e.g..  Public  School 
Council  or  Policy  Advisory  Council).  Successful  advocates  spend  time  with  sponsors  as  they  engagp  in  remedy  selling  (e.g., 
problem  solving  and  persuasion/communication)  and  pain  management  (e.g.,  dealing  with  resistance,  barriers,  and  constraints). 
Advocates  help  the  sponsor  realize  the  importance  of  the  desired  change  (Conner,  1992). 

To  understand  the  roles  and  influence  of  sponsors  and  advocates,  the  following  assumption  must  be  carefully  analyzed:  There 
is  a tendency  for  those  involved  in  or  affected  by  a change  initiative  to  overestimate  the  short-term  effects  of  change  and 
underestimate  the  long-term  effects  (Conner,  1992;  Reigeluth  & Garfinkle,  1994).  The  short-term  effects  are  the  results  of 
incremental  inputs  and  processes.  The  conditions  for  successful  implementation  and  sustained  change  are  greatly  influenced 
when  short-term  effects  are  internalized  and  institutionalized.  Change  transforms  into  long-term  effects  when  people  are  results- 
driven  and  have  been  given  purpose  and  value  as  to  why  the  change  is  required.  Initial  and  sustained  sponsorship  are  key  to 
making  this  type  of  transformation/change  possible.  The  following  display  (see  Figure  1)  is  an  example  of  a systems  approach  to 
educational  reform.  This  operations  model  was  created  during  Florida’s  SY2000  educational  reform  initiative  and  presents  a 
proposed  education  system  and  related  subsystems.  Having  no  intent  to  show  linear  relationship,  the  subsystems  are  connected 
by  a circle.  The  circular  pattern  shows  connectivity  or  inter-relatedness  among  the  eight  subsystems  (i.e.,  mission,  curriculum, 
instruction,  student  and  family  services,  assessment  and  information  management,  human  resource  development,  management 
operations,  and  logistical  services). 


Figure  L SchoolYear  2000  Operations  ModeL 
Source:  Robert  K.  Branson  © 

Reprinted  with  permission 

A working  design  team  backed  each  SY2000  subsystem.  Each  design  team  was  composed  of  FSU/CET  design  team  leaders 
and  staff,  school  district  administrators,  teachers,  parents,  and  school  community  members.  The  design  teams  met  on  a regular 
basis  to  design  and  develop  SY2000  interventions  and  products. 

Depicted  in  each  of  the  four  comers  is  an  external  client  (stakeholder- one  who  influences  or  interferes  with  an  innovation) 
who  is  directly  or  indirectly  affected  by  any  change  within  or  to  the  system.  The  circular  shape  forms  the  framework  that 
signifies  an  iterative  process  of  continuous  improvement  that  is  driven  by  quality  and  electronic  systems. 

The  Florida  SY2000  Initiative  was  a comprehensive  attempt  to  change  how  education  was  being  delivered  and  applied  in  the 
state  of  Florida.  Based  on  his  “upper-limit  hypothesis”  (Branson,  1987,  1998),  Dr.  Robert  K.  Branson,  director  of  FSU’s  Center 
for  Educational  Technology  and  primary  leader  (principal  investigator)  of  SY2000,  offered  the  following  rationale  in  1987  for 
changing  how  education  was  currently  being  delivered.  “As  we  look  at  the  future,  we  in  education  have  two  paths  we  can 
consider.  We  could  continue  down  our  current  path  and  experiment  with  a series  of  new  educational  alternatives,  or  we  could 
realize  that  the  current  system  might  be  performing  at  its  peak  and  it  was  time  to  consider  a new  system.  SchoolYear  2000  was  a 
way  of  heading  down  that  second  path"  (Branson,  Retrieved  February  17,  2001  on  the  World  Wide  Web 
http  ://cpt.  fsu.edu/sy2000AVHYNEED.  HTML). 
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Implementing  SY2000  was  a major  change  that  required  knowledge,  skills,  and  resources  required  for  moving  from  where 
the  state  of  education  was  to  where  it  should  be.  It  was  a major  transition  and  transformation  for  people  and  processes.  The 
greatest  challenge  was  to  sustain  the  change  for  the  long  journey,  making  a difference  in  the  lives  of  tomorrow’s  child. 

Leading  change  and  diffusion  of  innovation  experts  highlight  the  critical  roles  of  leadership  and  sponsorship  for  the 
successful  implementation  and  maintenance  of  major  change  or  reform  efforts  (Conner,  1992,  1998;  Ely,  1990;  Rogers,  1995; 
and  Waterman,  1 990).  It  is  for  this  reason  that  sponsors  and  advocates  were  selected  as  the  target  audience  for  this  historical  case 
study  research.  The  purpose  of  this  study  was  not  to  judge  right  or  wrong,  true  or  false,  significant  or  insignificant,  but  rather  to 
offer  a deeper  and  richer  understanding  of  what  happened,  why  it  happened,  and  ultimately,  what  were  the  long-term  effects  of 
SY2000. 

Factors  Affecting  Implementation 

What  does  it  take  for  an  implementation  to  be  successful?  What  factors  cause  concern  if  they  are  not  implemented  correctly? 
Many  agreed  that  SY2000  was  not  strengthened  when  important  implementation  factors  were  not  effectively  executed.  In  this 
section,  eight  implementation  factors  are  discussed  that  emerged  from  the  SY2000  survey  and  interview  data.  Most  factors 
aligned  with  Ely’s  (1990)  conditions  for  successful  implementation  (i.e.,  leadership,  participation  - buy-in,  commitment).  Other 
factors  that  aligned  with  Ely’s  conditions  appear  in  the  next  section  as  implementation  costs  (e.g.,  resources,  knowledge  and 
skills,  training,  time).  To  summarize  the  factors  that  affected  the  implementation  of  SY2000,  one  must  consider  the  importance  of 
each. 

Differing  cultures 

There  were  three  distinctive  cultures  that  had  initially  rallied  to  form  a synergistic  partnership  (i.e.,  government  - State 
Legislature;  university  - FSU/  GET;  and  schools  - SY2000  CSD).  However,  over  time  communication  was  difficult  to  manage 
among  these  differing  cultures.  Each  culture  was  interpreting  events  and  taking  action  based  on  its  respective  set  of  assumptions, 
beliefs,  and  behaviors.  For  communication  to  be  effective,  individuals  and  groups  from  each  culture  should  have  had  the 
opportunity  to  learn  about  the  others’  cultures  in  order  to  reach  agreement  on  what  would  be  the  best  action  for  all. 

From  the  data  analyzed,  I was  skeptical  about  whether  SY2000  leaders  and  participants  understood  the  varying  cultures  from 
which  participants  or  stakeholders  had  emerged.  The  data  indicated  that  each  participant  seemed  to  have  his/her  respective  and 
unique  set  of  beliefs,  assumptions,  and  behaviors.  This  was  by  far  the  mostly  noted  factor  by  respondents.  It  was  the  general 
belief  that  SY2000  did  not  plan  for  the  management  and  education  of  those  representing  the  differing  cultures  (e.g.,  government, 
K-12  schools,  and  university). 

Leadership 

Many  change  experts  state  that  leadership  is  crucial  to  successful  implementation  (Ely,  1990;  Conner  1992;  Rogers,  1995; 
Lick  & Kaufman,  2000).  SY2000  was  dependent  on  the  capacity  (willingness  and  ability)  of  its  leaders  to  systematically  design 
and  deliver  a large-scale  reform  initiative.  SY2000  leaders  had  to  anticipate  the  implications  of  implementation.  By  doing  so, 
they  would  have  realized  the  impact  of  clearly  articulating  SY2000’s  vision  and  philosophy  to  others.  Many  respondents 
perceived  this  role  to  articulate  and  communicate  with  sponsors  in  government,  university,  and  schools  as  that  of  the  SY2000 
advocates.  This  factor  was  critical  for  identifying  the  assumptions,  beliefs,  and  behaviors  of  differing  cultures  and  finding  ways 
to  manage  and  educate  their  understanding  and  sustained  commitment  and  buy-in  for  successfully  implementing  SY2000. 

Vision 

As  mentioned  earlier,  articulation  and  communication  of  an  initiative’s  vision,  goals,  and  objectives  is  critical  for  participant 
and  stakeholder  buy-in.  The  SY2000  vision  was  for  each  student  to  acquire  the  foundational  skills  and  competencies  needed  to 
succeed  in  adult  life  in  an  Information  Age.  SY2000  was  designed  as  a learner-centered  approach  for  teaching  and  learning.  The 
goal  of  SY2000  was  improving  learner  achievement  and  performance. 

It  is  critical  for  participants  and  stakeholders  to  share  the  vision  (Conner,  1990;  Barker,  1993,  Lick  & Kaufman,  2000).  The 
vision  is  the  guiding  star  and  gives  purpose  to  the  innovation.  However,  respondents  often  mentioned  how  the  vision  was  not  well 
articulated  and  therefore  not  well  understood. 

This  factor  closely  related  to  the  factors  of  leadership  and  buy-in.  Respondents  agreed  on  the  criticality  of  a shared  vision,  a 
vision  that  all  stakeholders  understood  and  believed  to  be  the  goal  of  SY2000.  When  participants  lost  sight  of  the  goal,  strong 
leadership  would  again  clear  the  path  for  them  to  see  the  vision.  When  participants  changed  (e.g.,  sponsors  or  advocates),  astute 
leadership  should  have  educated  the  new  participants  of  the  SY2000  goals  and  objectives  and  attained  their  buy-in  and 
commitment.  Buy-in  and  commitment  should  have  been  based  on  developing  synergistic  partnerships,  getting  people  together 
who  are  willing  and  able  to  bring  about  change. 

Sustained  commitment 

Buy-in,  often  associated  with  commitment,  is  another  critical  factor  for  successful  implementation  efforts  (Conner,  1992; 
Lick  & Kaufman,  2000).  Buy-in  must  come  from  the  participants  and  stakeholders  of  the  innovation.  With  continuous  buy-in 
(sustained  commitment)  they  will  develop  the  required  capacity  (willingness  and  ability)  to  achieve  desired  results.  And  with 
buy-in,  support  for  the  effort  is  provided  necessary  resources  to  achieve  results  - time,  money,  and  people.  Without  buy-in, 
resistance  to  change  flourishes  and  becomes  a serious  obstacle  to  successful  implementation. 
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School  districts  began  to  question  where  SY2000  was  headed,  leading  to  the  breakdown  in  dedication  and  commitment  of 
school  districts  to  SY2000.  A CSD  District  Administrator  expressed  this  sentiment  when  she  said,  ‘‘SY2000  should  have 
designed  incremental  results  built  on  goals.  An  urban  superintendent’s  life  is  a little  over  two  years.  To  keep  a district  involved 
and  committed  to  the  goals  of  SY2000,  you  have  to  create  something  that  shows  progress  within  short  periods  of  time.”  This  was 
the  factor  that  means  getting  people  to  stay  together  through  completion  or  achievement  of  goals  and  objectives.  This  is  the 
factor  that  respondents  perceived  waned  over  time.  Many  believed  that  when  SY2000  champions  lost  interest  or  were  no  longer 
involved,  sponsorship  was  lost  along  with  allocation  of  resources. 

Communication  is  how  one  articulates  and  conveys  a message.  Communication  during  a change  initiative  is  critical  (Conner, 
1990;  Lick  & Kaufman,  2000).  Change  participants  are  dealing  with  ambiguity  and  uncertainty.  They  are  venturing  into  new 
frontiers,  and  with  that  journey  comes  a natural  resistance  to  change,  to  thinking  and  doing  different  things.  Along  with  strong 
leadership,  a structure  and  organization  has  to  be  in  place  for  the  flow  of  communication  to  permeate  all  levels  of  participants  and 
stakeholders.  It  is  often  said  that  knowledge  is  power.  In  the  case  of  SY2000,  knowledge  was  the  foundation  on  which  to  build  a 
powerful  system  for  education  reform. 

Change  process 

Within  the  structure  and  organization  of  a reform  effort,  leaders  must  prepare  the  organization  for  change.  For  SY2000  to  be 
a success,  participants  and  stakeholders  had  to  be  prepared  for  the  change  effort.  There  should  be  a change  a process  for  creating 
and  managing  change.  Without  an  understanding  on  the  part  of  the  participants  of  the  reform  effort,  over  time,  synergy  and 
communication  dissolves,  participation  wanes,  and  eventually  interest  and  commitment  erode.  This  lack  of  synergy  and 
communication  was  an  issue  raised  by  many  of  the  SY2000  participants  surveyed  and  interviewed. 

A CSD  District  Administrator  explained,  “The  entry  point  for  change  in  Florida  schools  is  the  district.  Change  happens 
school -by -school,  and  significant  educational  reform  is  possible  district-by -district.  It  would  be  very  difficult  to  create  a rubric  to 
measure  reform  outcomes  at  the  state  level,  but  it  can  be  done  at  the  district  and  school  level.  You  must  tie  the  goals  of  the 
intervention  to  the  agreed-upon  rubric.” 

This  factor  dealt  with  getting  people  to  shift  from  “what  is”  to  “what  can  be.”  Respondents  felt  sponsors  or  SY2000  leaders 
didn’t  recognize  the  “what  is”  to  find  appropriate  solutions  to  move  towards  “what  can  be.”  The  lack  of  having  enough  time  and 
resources  was  blamed  for  the  change  process  not  being  fully  implemented. 

From  the  people  I interviewed  (i.e.,  initial  sponsors,  sustaining  sponsors,  and  advocates),  I heard  many  times  that  they  were 
eager  and  willing  to  learn  and  that  they  wished  SY2000  leadership  had  anticipated  the  time  and  effort  required  for 
communicating  and  articulating  the  SY2000  vision  and  measurable  outcomes.  Realizing  the  value  of  implementation  factors 
mentioned  in  this  section,  one  must  also  understand  there  are  costs  associated  with  implementing  change.  The  following  section 
brings  to  light  the  costs  that  SY2000  had  to  anticipate,  consider,  and  manage. 

Costs  of  Implementing  Factors 

There  are  costs  associated  with  successful  implementation.  Survey  respondents  (69%)  agreed  or  strongly  agreed  that  the 
sponsors  of  SY2000  were  aware  that  personal,  political,  or  system  cost  might  be  required  to  implement  SY2000.  Even  though 
the  perception  was  that  sponsors  had  this  understanding,  the  survey  data  showed  no  consensus  as  to  whether  sponsors  displayed 
strong  commitment  to  SY2000  - 75%  of  the  respondents  were  split  evenly  among  agree  and  disagree.  Supporting  these  data  were 
the  survey  and  interview  comments,  which  reflected  a perception  that  sponsors  did  not  manage  the  costs  well.  To  summarize  the 
cost  factors  for  implementing  SY2000,  one  must  consider  the  importance  of  each. 

Research  and  development 

R&D  are  perceived  as  valuable  inputs  for  any  change  effort.  R&D  identify  theories,  models,  and  current  best  practices 
associated  with  the  design  and  development  of  a new  initiative.  R&D  are  major  components  of  an  iterative  process  and,  as  a 
result,  are  ongoing  throughout  the  systematic  design  phases.  Most  respondents  appeared  to  recognize  the  value  of  R&D  but 
perceived  that  it  was  not  valued  by  SY2000  sponsors  (i.e.,  legislature  and  school  districts)  and  should  have  been  developed  with  a 
broader  base  (e.g.,  greater  university  and  college  input  throughout  the  state  of  Florida).  R&D  were  not  highly  valued  by 
government.  Politicians  were  under  pressure  to  produce  tangible  results  in  the  eyes  of  their  constituents.  FSU/CET  was  placed 
in  a hurry-up  mode  to  produce  products  to  satisfy  this  pressure  which  distanced  them  from  their  expertise  of  R&D. 

Allocation  of  resources 

Funding  was  a major  issue  with  all  respondents.  They  realized  that  a large-scale  education  reform  like  SY2000  required  a 
large  sum  of  money  and  long-term  support.  This  effort  was  not  a tweaking  of  the  current  system,  but  the  creation  of  a new  way 
of  schooling.  The  cost  to  bear  was  the  long-term  commitment  of  sponsors  to  allocate  required  resources  - time,  money,  and 
people.  Over  time,  the  CSDs  and  FSU/CET  resented  the  legislative  year-to-year  funding  of  SY2000.  From  the  very  beginning  of 
SY2000  it  was  explained  that  it  would  take  ten  years  and  $100M  to  fully  implement  the  proposed  changes  in  Florida  schools. 
Pressure  mounted  each  year  to  produce  tangible  results  in  order  to  justify  continued  funding. 

Measurable  results 

As  stated  in  previous  sections  the  lack  of  measurable  results  and  outcomes  were  mentioned  time  and  time  again.  Promoting 
an  initiative’s  worth  and  value  depends  on  the  perception  of  its  success  and  goes  hand-in-hand  with  continued  allocation  of 
resources.  This  factor  was  perceived  by  many  to  be  the  demise  of  SY2000.  The  respondents’  felt  there  was  a lack  of  milestones 
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or  deliverables  to  justify  the  pressure  mounting  in  the  legislature,  or  to  answer  questions  raised  by  CSD  principals  and  teachers  as 
to  where  SY2000  was  headed. 

Time 

It  is  often  said  that  time  is  a variable,  not  a constant,  when  dealing  with  change  (Ely,  1990;  Conner,  1992).  Time  can  be 
shortened  or  lengthened  on  paper,  but  in  reality  it  can  not  be  tampered  with  without  the  buy-in  from  those  who  will  be 
implementing  the  change.  Many  respondents  felt  time  was  being  forced  by  the  legislators  by  the  pressure  to  produce  tangible 
results.  Others  felt  the  university  did  not  understand  where  school  districts  were  along  the  change  process  spectrum  and  felt  there 
was  no  time  to  catch  up  to  other  participants,  or  conversely,  wait  for  other  participants  to  catch  up.  Therefore,  it  was  the 
perception  of  many  respondents  that  time  to  implement  a change  initiative  should  not  be  dictated  or  mandated.  Most  respondents 
referred  to  time  as  a cost  dependent  on  the  demonstration  of  measurable  results.  Time  was  perceived  as  a valuable  resource,  but 
one  not  afforded  for  SY2000  to  be  successful. 

The  respondent  comments  demonstrate  that  there  were  many  lessons  to  learn  from  having  had  the  experience  of  SY2000. 
The  next  section  addresses  the  lessons  learned  as  described  by  SY2000  participants.  However,  the  real  impact  will  be  if  the 
lessons  learned  encourage  the  reader  to  reflect  on  these  past  experiences  to  plan  and  design  future  initiatives. 

Lessons  Learned 

One  of  the  greatest  accomplishments  in  any  reform  endeavor  is  the  learning  that  takes  place.  Through  reflection  one  has  an 
opportunity  to  gain  insight  personally  (and  collectively  with  others)  about  what  worked,  what  didn’t  work,  and  what  may  have 
worked.  It  is  from  these  insights  that  individuals  or  groups  can  create  brighter  visions,  better  plans,  more  effective  solutions, 
stronger  actions,  and  more  useful  evaluations  for  future  reform  efforts.  Following  are  my  interpretation  of  the  lessons  learned  as 
expressed  by  survey  and  interview  respondents.  My  interpretations  are  represented  via  the  following  categories: 

Perceived  effects  of  SY2000 

This  study  was  an  examination  of  how  the  experience  of  an  innovation  had  an  affect  on  its  participants.  Therefore,  survey 
participants  were  asked  to  state  the  greatest  personal  impact  that  they  could  directly  or  indirectly  attribute  to  having  had  the 
experience  of  SY2000.  It  was  surprising  that  many  of  the  personal  effects  related  to  system  issues  or  broad  views  (e.g., 
implementation  factors,  politics,  quality  systems,  and  strategic  planning).  My  interpretation  of  these  perceptions  is  that  many  of 
those  surveyed  and  interviewed  realized  they  were  individuals  working  together  within  an  organization  (e.g.,  governmental, 
academic,  corporate).  Perhaps  this  was  the  reason  why  personal  impacts  were  expressed  in  terms  of  wholes  - education  system, 
and  not  parts  - the  individual.  Another  reason  why  personal  impacts  were  viewed  globally  may  have  been  because  of  the 
background  and  experience  of  those  surveyed  and  interviewed.  I had  contacted  the  perceived  movers  and  shakers  of  SY2000  of 
which  many  continue  to  serve  in  leadership  roles.  These  individuals  tend  to  think  “big  picture”  and  see  the  spectrum  of  impacts 
of  change  on  their  respective  organizations  before,  during,  and  after  reform  efforts.  It  would  be  interesting  to  examine  the 
perceptions  of  those  that  were  and  remain  in  management  or  operational  roles  (e.g.,  principals,  teachers,  and  school  support 
personnel)  to  see  if  they  too  view  things  from  a broad  perspective. 

Differing  cultures 

It  is  my  thought  based  on  the  surveys  and  interviews  that  the  most  damaging  obstacles  for  SY2000  becoming  a successful 
large-scale  reform  effort  were  the  differing  cultures  (i.e.,  government,  university,  and  K-12  schools)  that  remained  intact  with 
their  respective  assumptions,  beliefs,  and  behaviors.  It  was  futurist  Alvin  Toffler  who  aptly  said,  “The  illiterate  of  the  21®^ 
century  will  not  be  the  one  who  can  not  read  and  write,  but  the  one  who  can  not  learn,  unlearn,  and  relearn.”  From  the  data,  I 
believe  learning  emerged  as  the  critical  factor  in  getting  people  from  differing  cultures  to  come  together,  stay  together,  and  work 
together. 

Research  and  Development 

SY2000  was  a research-based  change  reform  initiative.  Many  of  the  school-based  and  corporate  (e.g.,  university,  K-12,  and 
corporate)  respondents  agreed  on  the  importance  of  R&D  and  its  role  in  developing  SY2000  concepts  and  products.  However, 
these  same  respondents  made  it  clear  that  LSI/CET  spent  too  much  time  and  energy  in  product  development  when  their  expertise 
was  in  research  and  development  of  concepts,  theories,  and  models.  And  these  respondents  were  critical  of  the  state  legislature 
for  not  valuing  R&D  as  they  should  have  in  terms  of  allocating  required  time  and  money. 

Redesign  or  Recreate 

Where  do  we  go  from  here?  Do  we  redesign  or  recreate?  Whether  reformers  choose  to  redesign  education  by  doing  different 
things  in  the  current  framework  or  whether  they  choose  to  recreate  education  by  doing  things  differently  using  a new  framework, 
all  agree  something  must  be  done  to  improve  our  student  achievement  and  performance  in  our  preK-20  education  system.  The 
respondents  raised  many  questions  based  on  their  comments  and  reflections  regarding  wishes  for  education  reform  (see  Table  1). 
Answering  these  questions  is  the  challenge  we  educators  have  before  us;  although,  I believe  questions  present  an  opportunity  for 
us  to  design  a successful  large-scale  reform  initiative. 
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Table  L Respondents  * wishes  and  questions  raised  for  future  education  reform. 


Respondents’  Wishes 

Question  Raised 

Allocation  of  resources 

How  money,  time,  people,  and  training  will  affect  the  implementation  of  a large-scale  reform 
effort 

Creation  of  a systemic 
design 

Why  a systemic  design  should  be  valued  as  the  foundation  for  large-scale  education  reform 

Development  of  a change 
process 

How  a change  process  should  adapt  to  the  capacity  levels  (i.e.,  willingness  and  ability)  of 
sponsors,  advocates,  and  participating  school  districts  and  individuals 

Development  of  a systematic 
design  process 

How  a process  should  give  organization  and  structure  to  the  implementation  of  a large-scale 
reform  effort 

Establishment  of  leadership 
and  support 

Why  the  importance  of  establishing  communication  and  articulation  with  and  between 
sponsors  and  advocates 

Table  1.  continued. 


Respondents’  Wishes 

Question  Raised 

Formation  of  partnering 

Why  the  importance  of  establishing  communication  and  collaboration  among  co-development 
partners 

Decision  for  where  do  we  go 
from  here 

Why  the  question  of  whether  to  redesign  or  recreate  is  raised 

Many  of  those  surveyed  and  interviewed  gave  the  impression  that  merely  redesigning  what  we  are  currently  doing  is  not  the 
change  required  to  make  the  difference  for  today’s  and  tomorrow’s  child.  From  their  comments,  I believe  they  prefer  doing 
things  differently,  a primary  focus  was  on  outcomes  for  learning  and  the  learner,  but  a secondary  focus  was  on  a change  process, 
a process  that  includes  and  is  dependent  on  how  to  make  it  happen. 

Driving  force  for  Education  Reform 

Shared  Vision  and  Mission:  For  large-scale  education  reform  to  be  valued  and  deemed  imperative  by  sponsors,  advocates, 
change  agents,  and  change  targets  and  external  stakeholders  (i.e.,  business/industry,  families,  community,  state  and  local 
agencies)  it  is  important  for  each  individual/group  to  agree  on  the  reasons  for  doing  so.  It  is  critical  to  share  the  vision  and 
mission  if  change  is  to  be  successful. 

The  Goal  of  Education  Reform  Initiatives 

2000  was  designed  as  a learning- centered  system  for  schools  in  Florida.  This  study’s  surveys  and  interviews  with  national 
and  state  leaders  of  education  reform  confirmed  this  premise.  The  true  driving  force  of  a school  system  is  curriculum, 
instruction,  and  assessment  (CIA).  It  is  CIA  from  which  all  other  education  system  functions  serve  as  supports  and  resources. 

CIA  as  the  driving  force  of  a school  system  was  discussed  duringan  interview  with  a CSD  District  Administrator.  He  began 
to  draw  an  illustration  to  depict  the  driving  force  behind  a school  system.  He  drew  a straight  line  of  boxes  labeled  with  different 
operational  functions  within  a school  system  (e.g..  Management  Information  Services  (MIS),  Legal,  Finance,  Student  Services, 
Facilities,  Personnel,  and  Human  Resource  Development  (HRD).  The  administrator  explained  that  most  school,  district, 
community,  state,  and  public  attention  and  resources  are  spent  on  these  operational  or  functional  supports.  In  contrast,  in  his  30+ 
years  of  service  to  public  education,  he  believes  the  attention  and  resources  must  be  spent  on  the  true  driving  force  of  a school 
system,  the  curriculum,  instruction,  and  assessment  (CIA).  He  said  research  and  development  provide  inputs  to  CIA,  and  CIA 
serves  as  the  direct  path  to  the  teacher  and  student,  for  it  is  CIA  that  directly  impacts  student  learning,  achievement,  and 
performance.  I began  to  share  my  thoughts  and  ideas  and  together  we  developed  the  following  CIA  driven  school  system  model 
(see  Figure  2). 
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Figure  2,  Driving  force  behind  a school  system:  Curriculum,  instruction,  & assessment. 


Leadership 

Finally,  I will  close  this  section  on  lessons  learned  by  addressing  what  I feel  is  the  most  critical  component  of  any 
organization  undergoing  a change  effort,  leadership.  From  the  literature  review,  the  interviews  conducted  during  this  study,  and 
the  years  of  experience  in  public  school  education,  I have  learned  that  collaboration  among  participants  of  a change  effort  cannot 
be  derived,  but  rather  is  built  over  time.  One  can  bring  players  together,  but  it  is  ultimately  the  players  who  will  decide  if,  when, 
and  how  they  will  work  together.  I have  developed  a model  of  leadership  for  education  reform  (see  Figure  3), 

The  first  step  is  getting  participants  to  sit  at  the  table.  Implementing  change  takes  a unique  leader  with  leadership  capacity 
(i.e.,  willingness  and  ability)  to  orchestrate  a team  of  players  (e.g.,  thinkers,  visionaries,  planners,  designers,  developers, 
implementers,  and  evaluators).  For  this  type  of  leader  to  be  effective  he/she  must  actively  involve  the  participants  from  the  very 
beginning  and  sustain  their  involvement  throughout  the  diffusion  of  the  innovation  to  its  attainment  or  completion.  And  finally, 
education  reform  requires  resources  (e,g,,  time,  money,  and  political  finesse).  These  resources  are  developed  with  partnerships 
with  the  state  legislature,  schools,  universities,  and  business  and  industry  and  others.  These  individuals  or  groups  are  considered 
the  stakeholders  because  they  will  directly  or  indirectly  benefit  from  the  success  of  the  reform. 
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II  . 

Required  Resources:  Time,  Money,  & Political  Finesse 


Figure  3.  Leadership  mo3eland  required  resources  for  education  reform.  *«**‘<*****®’^^ 

So  what  can  be  concluded  about  SY2000?  Many  believed  SY2000  was  merely  ahead  of  its  time.  Following  are  comments 
reflecting  this  perception.  A former  State  of  Florida  Education  Officer  said,  ‘‘SY2000  left  an  impact.  SY2000  was  the  beginning 
of  The  Change  and  it  got  the  rhetoric  to  the  future  change.  It  was  a catalyst  for  Florida  to  bring  about  change.  SY2000  never 
died:  its  funding  was  terminated.  SY2000  was  a flag  for  'Here  is  Education  Reform'.  I believe  SY2000  was  the  kick  start  for 
education  reform;  however,  it  got  politicized,  and  funding  was  eventually  terminated.” 

Conclusion 

What  lies  ahead  for  education  reform?  Does  education  need  merely  reform  or  an  entire  re-creation?  Has  the  shift  from 
teaching  to  learning- centered  education  created  a purpose  for  change?  Has  technology  brought  new  ideas  and  opportunities  for 
teaching  and  learning,  as  we  have  never  seen  before?  These  changes  result  in  paradigms,  therefore  what  are  the  solutions  for 
these  new  paradigms?  And  what  change  is  required  to  align  education  with  these  new  mindsets  and  directions? 

Summary 

The  outcome  of  this  research  study  was  the  presentation  of  SY2000  sponsors’  and  advocates’  perceptions  of  what  happened 
during  SY2000,  what  they  thought  should  have  happened,  and  what  they  wished  would  happen  in  future  education  reform  efforts. 
These  individuals  were  selected  based  on  their  leadership  or  influential  roles  and  positions  held  during  SY2000. 

What  were  the  successes  of  SY2000?  I believe  SY2000  participants  understood  the  value  of  a systemic  approach  to  educational 
reform.  There  were  none  whom  I surveyed  or  interviewed  that  questioned  a system  design  for  education  reform.  On  many 
occasions  I heard  that  SY2000  was  brilliantly  conceptualized  and  designed.  From  all  that  I heard  and  read,  I believe  SY2000 
opened  many  minds  to  view  education  as  a system  comprised  of  subsystems  and  mere  “tweaking”  of  subsystems  would  not  make 
for  sustained  change  or  reform. 

Another  success  of  SY2000  was  the  introduction  to  change  management  concepts  and  principles.  Many  agreed  that  change 
was  a process,  not  an  event.  It  was  believed  that  moving  from  the  status  quo  to  the  ideal  vision  required  strategic  and  tactical 
operations.  Respondents  welcomed  what  they  were  learning  about  change  and  how  and  why  people  make  the  transition  from 
“what  is”  to  “what  should  be.” 

However,  with  success  of  SY2000  came  failures.  The  failures  of  SY2000  were  often  missed  opportunities  or  lack  of 
understanding  or  expertise  to  anticipate  possible  outcomes.  SY2000  leaders  underestimated  the  sign  ificance  of  vision,  buy-in  and 
sustained  commitment  for  SY2000  to  be  deemed  successful.  Over  time,  if  those  involved  were  not  continuously  educated  to 
where  they  were  in  the  change  process,  interest  and  commitment  faltered,  and  was  eventually  lost.  This  was  the  case  with 
sponsors,  advocates,  and  CSD  participants. 

Legislators  should  have  better  realized  the  value  of  time  and  research  and  development  as  resources  for  embarking  on  a 
massive  and  complex  reform  effort  such  as  SY2000.  One  can’t  expect  results  of  an  unfinished  research-based  intervention. 
However,  SY2000  should  have  built  in  incremental  successes  or  measurable  results  that  could  have  been  perceived  by  sponsors 
and  advocates  as  successes  to  gain  and  sustain  buy-in  and  commitment. 

Without  doubt,  SY2000  should  have  better  developed  the  required  synergistic  partnerships  among  the  differing  cultures. 
From  hearing  the  different  perspectives  of  government,  K-12,  and  university  respondents,  I find  this  to  be  the  main  demise  of 
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SY2000.  Education  of  these  individuals  and  groups  was  crucial  for  vision,  buy-in,  and  commitment.  I believe  all  three  shared 
the  vision  of  SY2000,  however  I question  whether  they  shared  an  agreement  on  how  it  was  to  be  achieved  and  measured. 

And  probably  the  greatest  lesson  learned  from  these  successes  and  failures  or  missed  opportunities  was  realizing  the 
importance  that  learning  must  precede  change.  This  learning  is  the  cornerstone  or  bedrock  for  getting  people  to  come  together, 
work  together,  and  stay  together.  Learning  depends  on  open  communication  and  collaboration.  Both  are  built  on  developing 
trust  to  solve  problems  together  and  having  respect  for  the  power  of  the  whole  being  greater  than  the  sum  of  its  parts. 

Limitations  of  this  Research  Study 

There  were  two  key  limitations  to  this  research  study:  time  and  resources.  There  was  not  enough  time  to  explore  the 
relationships  between  other  SY2000  change  participants  and  sponsors  and  advocates  (i.e.,  change  agents  - GET  design  team 
leaders  and  participants,  school  principals,  district  staff;  and  change  targets  - SY2000  teachers,  instructional  support  staff, 
students,  and  families). 

In  addition,  there  was  not  enough  time,  travel  funds,  or  political  clout  to  gain  access  to  (or  obtain  the  inputs  of  or  feedback 
from)  Florida  State  Senators  or  State  and  DOE  Commissioners  of  Education  who  were  in  office  during  the  SY2000  years.  The 
lack  of  time  also  limited  the  breadth  and  depth  of  this  study  to  obtain  a broader  and  sharper  image.  I would  have  liked  to  pursue 
making  contact  with  some  of  the  individuals  who  did  not  respond  to  mail,  email,  or  telephone  requests.  I believe  due  to  current 
sensitive  issues  regarding  politics  and  education  issues  (e.g.,  vouchers,  charter  schools,  accountability,  and  testing/assessment, 
university  and  college  restructuring,  and  skeptical  public  perception  of  education  in  general),  I would  have  had  to  secure  others’ 
support  to  gain  access  to  some  individuals. 

Future  Research  in  This  Field 

Future  research  in  the  field  of  large-scale  education  reform  should  investigate  the  role  of  change  management  and  change 
creation  in  the  diffusion  of  an  education  innovation  or  initiative.  Action  research  should  validate  systematically  planned, 
designed,  developed,  implemented,  and  evaluated  reform  efforts  that  are  based  on  the  change  frameworks  and  methodologies 
outlined  by  Conner  (1992),  Lick  and  Kaufman  (2000),  and  Ely  (1996).  In  addition,  future  research  should  investigate  the  role 
and  impact  of  leadership  during  diffusion  of  an  innovation  in  an  educational  setting  (e.g.,  knowing  whom  to  turn  to  for  support 
and  resources  - information,  time,  money,  people).  I suggest  further  study  to  find  ways  to  eliminate  communication  voids  when 
revolving  sponsorship  is  inevitable. 

I suggest  further  research  to  validate  Lick’s  Universal  Change  Principle^  learning  (both  as  a verb  and  noun)  must  precede 
change,  by  applying  it  to  the  diffusion  of  an  implementation  and  evaluating  it  in  terms  of  cost  and  benefit  value.  Such  a research 
study  could  examine  and  validate  ways  to  educate  sponsors,  advocates,  change  agents,  change  targets,  and  stakeholders  on  the 
strategic  and  tactical  approaches  to  change  creation  and  implementation  (Conner,  1992;  Lick  & Kaufman,  2000). 

Finally,  future  research  should  consider  an  in-depth  examination  of  the  impacts  from  a large-scale  education  reform  on  other 
key  change  roles,  the  change  agents  (e.g.,  innovation  design  team  leaders  and  participants,  school  principals,  district  staff)  and 
change  targets  (e.g.,  teachers,  instructional  support  staff,  students,  and  families). 
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Abstract 

Since  the  early  1990's,  laptop  computers  have  repeatedly  been  suggested  as  a tool  to  improve  instruction  and  student 
learning.  Meanwhile,  several  evaluations  have  shown  that  laptops  helped  to  increase  the  amount  of  independent  and 
collaborative  learning.  Instruction  with  laptops  was  also  found  to  be  more  student- centered  in  many  cases.  However,  most  of 
these  studies  are  based  on  subjective  data  (interviews  and  questionnaires)  exclusively.  Observational  data  to  confirm  these 
findings  is  missing.  The  present  study  examines  45  lessons  (24  with  and  21  without  laptop  use)  that  where  videotaped  over  the 
course  of  2 Vi  years  in  a laptop  program  of  a German  high  school.  It  is  found  that  when  laptops  are  used,  the  use  of  other 
instructional  media,  particularly  notebooks  and  the  blackboard,  decreases  significantly.  The  amount  of  independent  work  is 
found  to  increase  significantly,  while  for  other  forms  of  instruction,  such  as  teamwork,  pair  work,  lectures  and  teacher -guided 
discussions  no  differences  can  be  determined.  The  results  are  discussed  in  relation  to  interview  and  survey  data  gathered  in  the 
same  study. 

Introduction 

Many  studies  have  shown  that  computers  at  school  can  have  a beneficial  effect  not  only  on  student  achievement  but  also  on 
students’  learning  motivation,  on  classroom  atmosphere,  and  on  the  teachers’  willingness  to  experiment  with  new  and  innovative 
instructional  approaches  (Christmann,  Badgett  & Lucking,  1997;  Dwyer,  1994;  Kulik  & Kulik,  1991;  Liao,  1992;  Losak  & 
MacFarland,  1994;  Sivin-Kachala  & Bialo,  1996).  Yet,  it  is  often  lamented  that  schools  are  reluctant  and  slow  in  integrating 
computers  and  in  adapting  to  the  needs  of  the  information  society  (e.  g.  Peck  & Dorricott,  1994).  Studies  found  that  the  schools’ 
computer  equipment  is  often  used  inadequately  or  not  at  all  (Marcinkiewicz,  1993).  One  reason  for  this  might  be  that  most  school 
computers  are  desktop  machines  located  in  computer  labs,  which  cannot  be  used  flexibly  when  needed  and  cannot  be  used 
outside  school  (Fabry  & Higgs,  1997).  Obviously,  in  an  arrangement  like  this,  computers  cannot  become  the  natural  tool  for 
learning  that  is  so  often  called  for. 

The  use  of  mobile  computers  is  often  suggested  as  a possibility  to  solve  this  problem.  Only  if  every  student  and  every  teacher 
had  his/her  own  laptop  computer,  say  proponents  of  mobile  computers,  information  technology  in  education  could  be  used  to  its 
full  effect.  Therefore,  the  introduction  of  laptops  to  the  classroom  is  seen  as  a major  catalyst  for  a profound  change  in  learning 
and  instruction  in  K-12  as  well  as  in  higher  education  (Owen  & Lambert,  1996;  Robertson,  Calder,  Fung,  Jones  & O'Shea,  1997; 
Stager,  1995). 

Since  the  start  of  the  Microsoft  „Anytime,  Anywhere  Learning**  initiative  in  1996  (to  name  only  one  of  numerous  sponsoring 
programs),  the  use  of  mobile  computers  has  spread  worldwide  and  several  extensive  evaluations  have  been  carried  out  (e.  g. 
Bruck,  et  al.,  1998;  Fouts  & Stuen,  1997;  Hill  & Reeves,  1999;  Rockman  et  al.,  1998;  Stevenson,  1999).  With  regard  to 
instructional  strategies  and  classroom  practice,  some  of  these  studies  found  that  phases  of  independent  work  as  well  as  project 
work  increased  (Fouts  & Stuen,  1997;  Rockman  et  al.  1998)  or  that  teachers  felt  that  laptops  were  particularly  suited  to  this  kind 
of  work  (Bruck  et  al.,  1998).  Studies  also  repeatedly  showed  that  there  was  more  extensive  collaborative  learning  in  the  laptqj 
classrooms  (Bruck  et  al.,  1998;  Fouts  & Stuen,  1997;  Rockman  et  al.  1998)  and  that  students  liked  to  help  each  other  as  well  as 
their  teachers  with  laptop  problems  (Bruck  et  al.,  1998;  Rockmann  et  al.,  1998).  On  the  other  hand,  it  was  also  found  that 
technical  problems  often  hampered  the  successful  use  of  laptops  in  the  classroom  (Bruck  et  al.,  1998;  Robertson  et  al.,  1996, 
Rockman  et  al.,  1997).  In  addition,  Bruck  et  al.  (1998)  reported  that  some  teachers  deplored  a high  level  of  student  distraction,  as 
well  as  the  loss  of  authority  and  control  in  the  laptop  classroom. 

Overall,  it  seems  that  many  of  the  positive  expectations  that  are  associated  with  the  use  of  mobile  computers  in  school  could 
be  confirmed  in  recent  studies.  However,  the  studies  referred  to  above  are  tainted  with  various  problems:  First,  the  results  are 
mainly  based  on  questionnaire  and  interview  data  exclusively,  which  is  known  to  be  biased  in  several  ways  (Bortz  & Doring, 
1995).  Behavioral  and  observational  data  to  confirm  the  self  reported  data  is  missing  in  most  studies.  Also,  many  studies  did  not 
investigate  control  groups  so  that  the  effect  of  the  laptop  computers  is  not  clear.  Finally,  results  are  sometimes  based  on  rather 
short  observation  periods  (ranging  from  a couple  of  weeks  to  2 years  at  most)  and  small  student  samples. 

Background  of  the  study 

In  March  1999,  one  of  the  first  laptop  programs  in  Germany  started  at  the  Evangelisch  Stiftisches  Gymnasium  Guetersloh. 
More  than  300  students  of  this  high  school  and  their  teachers  were  gradually  furnished  with  networked  laptop  computers. 
Students  of  four  cohorts  entered  the  program  in  grade  7 and  are  using  laptops  regularly  at  school  and  at  home  until  the  end  of 
grade  10. 
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The  schoors  administrators,  participating  teachers  and  parents  have  developed  an  elaborate  pedagogical  concept  for  the 
integration  of  laptops,  which  is  based  on  the  school’s  three  general  assumptions  about  the  benefits  of  incorporating  media  in 
instruction:  Media  are  essential  to  foster  students’  media  literacy,  they  can  serve  as  a tool  to  connect  the  classroom  to  the 
authentic  world  outside  school  and  they  are  a valuable  means  to  improve  instruction  (Engelen,  2000;  2001a,  b).  With  regard  to 
the  improvement  of  instruction,  the  concept  draws  from  a reform-pedagogical  framework  that  states  that  media  should  be  used  to 

• foster  the  concreteness  and  vividness  of  instruction, 

• facilitate  individualized  learning  and  intensify  phases  of  student  work 

• strengthen  teamwork  and  communication  in  the  classroom 

• increase  the  amount  of  independent  learning  and  problem  solving 

• further  students’  responsibility  and  improve  their  attitude  toward  co-operation  and  sense  of  belonging  to  the  school 
community 

This  framework  provides  the  foundation  for  the  school’s  concept  of  laptop  learning.  Teachers  jointly  develop  instructional 
units  based  on  these  reform-pedagogical  ideas,  making  sure  that  the  computers  are  used  to  serve  a sound  didactic  purpose. 
Important  cornerstones  of  the  concept  are  also  that  the  computer  is  integrated  into  the  regular  curriculum  from  the  very  beginning 
and  that  the  frequency  of  use  of  the  laptops  is  increased  step-by-step,  starting  with  only  one  subject  and  gradually  expanding  to 
the  whole  curriculum  within  the  first  project  year. 

An  extensive  evaluation  that  is  carried  out  by  the  Center  for  Media  Research,  Freie  Universitaet  Berlin  accompanies  the  pilot 
project.  The  evaluation  focuses  on  changes  in  instructional  strategies  and  classroom  practice,  student  achievement  in  selected 
subject  areas  and  the  acquisition  of  cross-curricular  competencies  (an  extensive  report  will  be  published  in  2002  by  the 
Bertelsmann  Foundation,  one  of  the  main  sponsors  of  the  project).  This  paper  presents  preliminary  results  regarding  the  first 
aspect,  instructional  strategies  and  classroom  practice. 

Method 

To  overcome  the  shortcomings  of  other  studies  pointed  out  above,  observational  data  was  gathered  to  investigate  changes  of 
teaching  strategies  and  classroom  practice.  A randomly  selected  sample  of  lessons  of  the  laptop  classes  was  videotaped  over  the 
past  V/i  years.  The  same  classes  and  the  same  teachers  were  recorded  repeatedly.  As  the  laptop  classes  are  not  using  their 
computers  every  day,  lessons  with  and  without  laptop  use  could  be  recorded  in  the  same  classes  with  the  same  teachers. 
Confounding  effects  that  result  from  comparing  different  classes  or  teachers  with  different  teacher  styles,  could  thus  be  reduced. 
Subjects  covered  are  mathematics,  German,  and  English. 

A body  of  45  lessons  was  videotaped.  24  of  these  were  laptop  lessons,  21  were  lessons  without  laptop  use.  Table  1 provides 
an  overview  of  the  distribution  of  lessons. 

Table  I:  Distribution  of  lessons  with  and  mthout  laptops 


Laptop 

Grade 

Total 

Grade  7 

Grade  8 

Grade  9 

With  Laptop  Subject  English 

4 

3 

1 

8 

German 

5 

4 

1 

10 

Maths 

3 

2 

1 

6 

Total 

12 

9 

3 

24 

Without  Laptop  Subject  English 

4 

1 

- 

5 

German 

5 

3 

1 

9 

Maths 

2 

3 

2 

7 

Total 

11 

7 

3 

21 

The  lessons  with  and  without  laptops  have  so  far  been  compared  for  use  of  instructional  media  and  form  of  instruction.  Every 
lesson  was  divided  into  a maximum  of  nine  intervals  of  5 minutes  length.  For  each  interval,  two  trained  observers  recorded  the 
dominant  media  use  and  form  of  instruction.  The  inter-rater  agreement,  as  determined  with  the  intra-class  coefficient,  was  .93  for 
the  use  of  instructional  media  and  .79  for  the  form  of  instruction. 

The  results  of  the  video  study  are  discussed  in  the  light  of  data  gathered  from  student  surveys  and  qualitative  interviews  with 
teachers  and  students  in  the  last  section  of  this  paper. 

Results 

Use  of  Instructional  Media 

On  a descriptive  basis,  it  was  found  that  the  use  of  the  laptop  had  a decreasing  effect  on  the  frequency  of  use  of  “traditional” 
instructional  media,  such  as  textbook  and  worksheets,  blackboard  and  notebooks  (see  Fig.  1). 


Fig.  1:  Media  use  (number  of  5-minute  intervals)  in  lessons  with  and  without  laptops 


The  difference  between  laptop  and  non-laptop  lessons  was  tested  with  a multivariate  two-factorial  analysis  of  variance  with 
laptop  use  and  school  subject  as  fixed  factors.  The  dependent  variables  were  the  frequencies  of  instructional  media  use,  A 
significant  main  effect  was  found  for  the  factor  laptop  use  (Wilks’s  Lambda  = .10;  F(7,32)  = 40.04;  p < ,01),  An  analysis  at  the 
univariate  level  showed  that  the  frequency  of  use  of  the  blackboard  (f(  1 ,38)  = 9.62;  p = .04),  notebook  (F(  1 ,38)  = 52,37;  p<  .01 ) 
and  of  course  laptop  (?^(1,38)  = 214,48;  p < .01)  was  significantly  different  between  laptop  and  non-laptop  lessons.  The 
frequency  of  use  of  worksheets  tended  to  be  significant  (F(  1 ,38)  = 3,72;  p = .06). 

The  interaction  of  laptop  use  and  school  subject  tended  to  be  significant  at  the  multivariate  level  (Wilks’s  Lambda  = ,53; 
f*(  14,64)  = 1.69;  p = .08).  A subsequent  univariate  test  indicated  that  there  was  a significant  interaction  for  the  use  of  the 
blackboard  (F(2,38)  = 3.32;  p = .05)  and  notebook  (F(2,38)  = 6.60;  p - ,03).  The  interaction  plot  is  depicted  in  Fig,  2.  It  can  be 
seen  that  there  is  a particularly  sharp  decrease  in  the  use  of  the  blackboard  and  of  notebooks  in  mathematics,  while  the  difference 
is  much  less  distinct  in  the  two  other  subjects. 


Fig.  2:  Interaction  plots  for  blackboard  and  notebook 

Blackboard  Notebook 
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Form  of  Instruction 

For  the  form  of  instruction,  the  descriptive  analysis  showed  that  guided  discussions  are  the  most  prominent  form  of 
instructional  activity  both  in  laptop  as  well  as  in  non- laptop  lessons  (see  Fig  3).  Laptop  lessons  differed  most  strongly  from  non- 
laptop lessons  in  that  students  work  individually  more  frequently.  Qualitatively,  it  was  interesting  to  note  that  phases  of 
individual  work  were  often  longer  when  the  laptop  was  used  than  in  non-laptop  lessons.  Group  activities  were  also  observed 
slightly  more  often  in  laptop  lessons  than  in  non-laptop  lessons  and  tended  to  be  more  project -based,  while  group  activities  in 
traditional  lessons  were  shorter  exercises  that  could  be  solved  in  a limited  amount  of  time,  typically  within  the  respective  lesson. 
The  frequency  of  lecture  and  pair  work  is  almost  identical  in  laptop  and  non-laptop  lessons. 


Fig.  3:  Form  of  instruction  (number  of  5-minute  intervals)  in  lessons  with  and  without  laptops 
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The  difference  between  lessons  with  and  without  laptop  use  was  again  tested  with  a two-factorial  analysis  of  variance  with 
laptop  use  and  school  subject  as  fixed  factors  and  the  five  forms  of  instruction  as  dependent  variables.  A significant  main  effect 
was  found  for  the  factor  laptop  use  (Wilks’s  Lambda  = .71;  F(5,34)  = 2.82;  p = .03).  At  the  univariate  level,  only  the  frequency 
of  individual  work  was  found  to  be  significant  (F(l,38)  = 10.82;  p < .01)  while  the  other  differences  could  not  be  statistically 
confirmed.  The  interaction  of  laptop  use  and  school  subject  was  not  significant  (Wilks’s  Lambda  = .73;  F(  10,68)  = 1.16;p  = .33). 

Discussion 

The  differences  in  the  use  of  instructional  media  in  laptop  and  non-laptop  lessons  show  that  the  computer  primarily  takes 
over  the  function  of  the  notebook.  This  finding  corresponds  with  the  statement  of  students  and  teachers  in  questionnaires  and 
interviews  that  the  laptops  are  most  frequently  used  as  a writing  tool.  However,  this  does  not  mean  that  the  laptops  are  simply 
used  as  an  “electronic  notebook”.  In  fact,  especially  in  the  teacher  interviews  it  is  emphasized  that  using  an  electronic  writing  tool 
enhances  the  use  of  the  traditional  notebooks  in  several  ways:  Most  importantly,  the  possibility  to  easily  change  and  edit  text  is 
seen  as  a tremendous  advantage  of  the  laptops  for  introducing  students  to  the  process  of  writing.  It  helps  to  encourage  them  to 
critically  analyze  their  own  and  their  classmates’  texts  and  improve  them  by  revising  and  re-writing  them  multiple  times.  Also, 
some  teachers  say  that  writing  electronically  motivates  reluctant  writers  and  many  students  stated  that  writing  with  the  laptops 
was  fun  and  one  of  their  favorite  activities.  In  addition,  the  laptops  are  used  to  create  archives  for  certain  subject  matter  (e.  g.  an 
English  grammar  archive  or  a Mathematics  archive)  with  entries  that  are  electronically  linked.  These  archives  are  continually 
expanded  throughout  the  years.  Both,  teachers  and  students  said  that  the  archives  have  many  advantages  in  comparison  to 
traditional  notebooks,  as  they  have  a more  transparent  structure,  new  entries  can  easily  be  connected  to  existing  entries,  they  do 
not  get  lost  and  some  students  found  that  they  are  better  suited  for  revision  and  learning.  Finally,  teachers  and  students  stated  that 
co-operative  writing  tasks  could  be  carried  out  much  easier  with  computers. 

Another  remarkable  difference  in  the  use  of  instructional  media  is  that  the  blackboard  was  used  less  often  in  laptop  than  in 
non-laptop  lessons.  This  may  indicate  that  instruction  in  the  laptop  lessons  is  less  teacher-centered  as  the  blackboard  is  typically 
used  in  teacher- guided  phases.  The  teacher  interviews  confirm  this  interpretation  as  teachers  said  that  when  they  teach  with 
laptops  they  often  do  not  teach  the  whole  class  but  interact  with  individual  students  while  the  class  is  working  on  a problem 
individually  or  in  groups.  However,  the  analysis  of  the  frequency  of  lectures  and  teacher- led  discussions  in  laptop  and  non-laptop 
lessons  weakens  this  assertion  (see  below). 

Regarding  the  school  subjects,  the  clearest  difference  in  the  use  of  instructional  media  was  found  in  mathematics. 
Mathematics  was  found  to  be  the  subject  that  makes  most  extensive  use  of  the  traditional  media  notebook  and  blackboard  in  non- 
laptop lessons.  Thus,  in  terms  of  instructional  media  use,  it  is  impacted  most  profoundly.  However,  subjectively,  the  change  is 
felt  similarly  in  all  subjects,  as  the  observations  from  the  interviews  referred  to  above  were  comparable  across  subjects. 

Regarding  the  form  of  instruction,  the  differences  between  lessons  with  and  without  laptop  use  are  less  clear.  Phases  of 
individual  work  increased.  This  corresponds  to  the  pedagogical  concept  of  the  school  on  the  one  hand,  as  the  laptops  were  to  be 
used  to  facilitate  individualized  learning  and  intensify  phases  of  student  work  and  to  increase  the  amount  of  independent  learning 
and  problem  solving.  It  also  confirms  the  statement  of  the  teachers  in  teacher  interviews  that  they  use  the  laptops  to  activate  the 
students  and  often  employ  them  for  individual  work  to  make  sure  that  every  student  acquires  computer  literacy,  which  is  one  of 
the  main  goals  of  the  project. 

The  frequency  of  teacher-led  discussion  decreased  in  laptop  lessons.  However,  the  difference  in  comparison  to  non-laptop 
lessons  was  not  significant.  This  difference  was  felt  to  be  much  stronger  in  the  teacher  interviews,  where  a majority  of  teachers 
stated  that  instruction  is  becoming  considerably  less  teacher-centered  when  laptops  are  used.  The  video  observation  shows  that 
just  like  in  traditional  lessons,  guided  class  discussions  are  still  the  most  prominent  form  of  instruction  in  laptop  lessons.  Also, 
the  amount  of  lectures  was  almost  identical  in  laptop  and  non-laptop  lessons. 

Moreover,  the  frequency  of  group  work  did  not  differ  strongly  between  laptop  and  non-laptop  lessons.  This  result  is  partly 
confirmed  by  student  surveys  as  two  of  the  three  cohorts  reported  only  a slight  increase  of  group  work  or  even  a decrease  of 
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group  work.  Teachers  held  varied  opinions  on  whether  they  had  students  do  more  group  work  when  laptops  were  used. 
Especially  some  teachers  of  the  first  cohort  experimented  extensively  with  teamwork  and  carried  out  several  student  projects. 
Experiences  were  mixed,  as  students  were  partly  overwhelmed  when  they  worked  on  projects  in  multiple  subjects  at  the  same 
time.  In  the  following  years,  project  work  was  therefore  reduced.  Some  teachers  of  the  second  and  third  cohort  reported  that  they 
do  not  use  group  work  more  often  than  in  regular  instruction.  Yet,  students  as  well  as  teachers  unanimously  reported  that  they  felt 
that  laptops  are  a valuable  tool  for  teamwork  and  that  working  collaboratively  is  easier  with  laptops  than  with  traditional  media. 


Conclusion 

The  results  show  that  the  major  change  in  the  laptop  classroom  is  an  increase  in  individual  work.  Students  work  more  often 
independently,  which  according  to  many  teachers,  results  in  a higher  degree  of  activation  than  in  traditional  lessons.  In  contrast  to 
other  evaluations  mentioned  earlier,  this  study  could  not  uiequivocally  confirm  that  using  laptops  led  to  more  collaborative 
classroom  activities.  Even  though  strengthening  teamwork  and  communication  in  the  classroom  is  one  of  the  declared 
pedagogical  goals  of  the  project,  there  was  no  clear  general  increase  in  group  work.  It  seems  that  while  extensive  group  projects 
have  been  carried  out  with  the  laptops,  this  was  rather  the  exception  than  the  rule.  If  only  subjective  data  had  been  gathered,  these 
projects  could  have  easily  been  overemphasized.  The  same  is  true  for  the  change  from  teacher-  to  student-centered  instruction. 
While  there  are  clear  indications  that  the  laptop  classroom  is  becoming  more  student-centered,  the  video  analysis  also  shows  that 
the  change  is  less  profound  than  only  looking  at  the  subjective  data  might  have  suggested. 

The  use  of  video  proves  to  be  a helpful  tool  to  corroborate  findings  that  are  based  on  subjective  data.  Results,  which  were 
derived  from  questionnaires  and  interviews,  could  be  confirmed  through  observation  in  many  cases,  thus  raising  their  credibility. 
While  it  should  not  be  concluded  that  one  source  of  data  has  more  value  than  the  other,  this  study  shows  that  combining 
observational  and  survey/interview  data  is  worthwhile  to  enhance  the  validity  of  a study’s  results.  It  reduces  the  danger  of 
misinterpreting  findings  and  helps  to  detect  possible  biases  in  subjective  reports. 
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Abstract 

An  instructional  approach  for  conceptualizing  the  five  domains  of  the  instructional  technology  knowledge  base  defined  by 
AECT  Initial  and  Advanced  Program  Guidelines  —design,  development,  utilization,  management  and  assessment~that  were 
required  to  be  integrated  into  professional  electronic  portfolios  developed  by  educational  technology  degree  candidates  is 
examined.  An  in  depth  look  of  the  process  is  provided  through  quantitative  and  qualitative  data  about  the  students'  perceptions  of 
the  utility  of  the  AECT  domains  for  communicating  their  professionalism  in  their  portfolios.  The  process  is  further  examined 
through  the  lenses  provided  by  five  departmental  faculty  members'  qualitative  assessments  of  the  students'  portfolios  and  the 
students'  presentations  of  these  artifacts.  Results  of  this  preliminary  study  indicate  that  although  the  five  domains  tend  to  be 
valued  for  their  usefulness  in  designing  to  express  emerging  professionalism,  unless  required,  most  would  not  employ  them. 
Once  applied,  however,  most  would  retain  this  framework.  Additional  suggestions  for  modifying  the  portfolio  design  aid 
development  process  from  initiation  in  a web-based  course  towards  the  beginning  of  the  students'  course  of  studies  through  to  the 
final  presentation  at  the  conclusion  of  their  program  are  offered.  The  increased  emphasis  on  portfolio  development  throughout 
education,  the  growth  of  web-based  design,  and  the  acceptance  of  the  AECT  Guidelines  for  candidates  in  both  Basic  Media 
Technology  in  Teacher  Education  and  Advanced  Programs  in  Educational  Communications  and  Technology  by  the  Council  for 
the  Accreditation  of  Teacher  Education  (NCATE)  suggest  continued  study.  Such  study  is  especially  warranted  in  terms  of  the 
potential  for  providing  learner  guidance  relative  to  this  area  for  students  enrolled  in  educational  technology  programs. 

Introduction 

The  Accreditation  Committee  of  Association  for  Educational  Communications  and  Technology  (AECT)  provides  Folio 
Review  Guidelines  related  to  educational  technology.  The  guidelines  include  Basic  Guidelines  for  Media  Technology  in  Teacher 
Education,  as  well  as  for  Advanced  Programs  in  Educational  Communications  and  Technology  (Association  for  Educational 
Communications  and  Technology,  2000)  that  are  accepted  by  the  National  Council  for  the  Accreditation  of  Teacher  Education 
(NCATE).  These  guidelines  provide  a systematic  means  for  internal  and  external  program  reviews  at  colleges  and  universities 
throughout  the  United  States. 

Standards  in  Education 

In  this  twenty-first  century  such  an  emphasis  on  standards  for  education  is  not  uncommon.  Examples  can  be  found  relative  to 
overall  professionalism  for  teachers  (National  Board  for  Professional  Teaching  Standards,  2001),  specific  Standards  for 
Technology  in  Teacher  Preparation  (International  Society  for  Technology  in  Education,  2001),  and  specific  content  areas,  such  as 
standards  for  mathematics  (National  Council  for  the  Teachers  of  Mathematics,  n.d.). 

Standards  in  Educational  Technology 

In  the  field  of  educational  technology,  candidates  in  both  entry  level  and  advanced  programs  are  expected  to  acquire 
knowledge  and  skill  within  five  main  areas  of  the  instructional  technology  knowledge  base  in  domains  of:  design,  development, 
utilization,  management  and  assessment.  Within  advanced  programs,  candidates  should  demonstrate  minimal  competencies 
within  each  area  along  with  advanced  capability  in  aspects  specifically  emphasized  by  the  program  at  their  institutions. 

Web-Based  Delivery  of  Instruction  and  Professional  Portfolio  Development. 

Additionally,  two  other  areas  that  impact  educational  technology  majors  are  web-based  delivery  of  instruction  and 
professional  portfolio  development.  These  students  may  acquire  some  of  their  capabilities  through  web-based  coursework. 
Educational  technology  majors,  as  current  and  future  teachers  or  trainers,  may  be  expected  to  teach  by  means  of  the  World  Wide 
Web  (WWW).  Experience  with  this  mode  of  learning  is  essential  for  th^  future  distance  learners  and  for  their  colleagues  when 
they  function  as  consultants.  Furthermore,  these  majors  are  increasingly  expected  to  demonstrate  their  knowledge  and  skills  at  the 
completion  of  their  program  of  studies  in  electronic  portfolios. 

Professional  Electronic  Portfolios 

Professional  electronic  portfolios  would  appear  to  be  an  integral  part  of  the  core  concepts  of  post-modernism  that  Solomon 
(2000)  relates  to  instructional  technology.  Professional  portfolio  development  supports:  1)  respect  for  differences  and  multiple 
explanations  through  inherent  nonlinear  design;  2)  construction  of  personal  views  by  prospective  employers  and  others  through 
text,  audio,  and  visual  information  about  the  candidate;  3)  critical  evaluation  as  candidates  assess  themselves  at  the  design  phase; 
and  4)  expression,  to  some  extent,  of  the  complexity  of  the  system  of  educational  technology  as  hypermedia  links  to  design,’ 
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development,  utilization,  management,  and  evaluation  aspects  and  to  candidates’  professionalism  and  theoretical  constructs  of  the 
field.  Additionally,  development  of  Web-based  electronic  portfolios  is  growing  at  all  levels  of  education  (Sanders,  2000). 

Web-Based  Course^  Program  of  Studies,  and  Web-Based  Production 

At  a department  of  educational  technology  at  one  Pacific-based  university,  graduate  students  participate  in  a totally  Web- 
based  course  on  newer  technologies.  Typically,  they  enroll  in  this  course  in  their  first  or  second  semester  in  the  program.  The 
Web-based  delivery  mode  of  this  course  with  its  multilevel,  hypertext  structure  and  design  for  interactivity  with  the  learners  by 
synchronous  and  asynchronous  communications,  text,  audio,  and  graphic  display  (Chu  2000)  offers  a supportive  context  for  core 
concepts  of  postmodernism. 

Along  with  earning  39  graduate  level  credits  in  required  and  elective  courses,  completing  6 credits  in  prerequisite  courses, 
engaging  in  both  a collaborative  and  individual  practicum,  and  carrying  out  a comprehensive  master’s  project,  these  students  were 
recently  required  to  also  produce  a professional  electronic  portfolio  during  their  program  of  studies.  Introduction  to  this  portfolio 
process  was  incorporated  in  the  required  Web-based  course  that  addresses  basic  concepts  of  instructional  design  and  multimedia 
authoring.  Typically,  students  enroll  in  this  course  during  their  first  or  second  semester  of  study.  In  following  this  newer 
departmental  process  for  developing  this  Web-based  multimedia  perspective  about  themselves,  they  continue  the  process 
throughout  their  program.  As  currently  designed,  after  the  initial  course,  consultation  with  peers  and  advisors  about  their 
continued  development  of  their  portfolios  occurs  primarily  when  initiated  by  the  student.  During  a culminating  course,  they  do 
have  the  opportunity  to  critique  each  other’s  work  prior  to  formally  presenting  their  completed  portfolios  to  the  faculty. 

When  this  project  was  first  instituted,  students  studied  multimedia  design  principles  (Park  and  Hannafin,  1993)  and  addressed 
general  guidelines  for  content  broadly  described  to  encourage  individuality  of  expression.  At  the  conclusion  of  the  course,  initial 
portfolios  were  assessed  in  terms  of  effective  design  and  plan  for  content  coverage.  An  initial  investigation  of  the  students’  work 
indicated  that,  although  they  demonstrated  competencies  in  all  five  domains  that  define  an  educational  technologist,  they  did  not 
tend  to  overtly  conceptualize  their  expertise  within  this  framework.  Many  communicated  their  capabilities  in  terms  of  courses, 
which  may  or  may  not  be  meaningful  to  future  employers  (Sherry,  2000). 

Problem  Statement 

Although  faculty  in  educational  technology  programs,  particularly  those  members  engaged  in  the  NCATE  process,  may 
become  adept  at  describing  their  programs  and  their  candidates’  capabilities  in  terms  of  these  domains,  how  might  their  students 
begin  to  conceptualize  their  leaning  within  this  framework?  Presenting  their  competencies  in  this  manner  could  offer  potential 
employers  and  their  graduate  faculty  a consistent,  professional  view  of  their  expertise  in  these  critical  areas  in  relation  to 
standards  set  by  a professional  association. 

As  these  findings  emerged,  departmental  faculty  members  decided  to  disseminate  information  about  the  domains  and  the 
framework  for  the  portfolios  as  the  students  were  developing  their  initial  portfolios  in  the  Web-based  course.  They  followed  up 
with  a print-based  reminder  as  the  students  advanced  toward  completion  of  their  portfolios  prior  to  graduation.  These  students 
earn  the  required  minimum  of  39  credits  in  courses  in  areas  specific  to  these  domains  and  in  specific  related  aspects  of 
educational  technology.  As  students  who  are  used  to  developing  and  analyzing  instructional  designs  in  a structured  manner,  it 
became  essential  to  evaluate  their  perceptions  of  the  utility  of,  and  type  of  integration  of,  this  structured  framework  for 
expressing  their  nascent  professionalism. 

Methodology 

An  investigation  was,  thus,  undertaken  as  a preliminary  evaluation  study. 

Ethical  Practice  and  Participants 

This  examination,  construed  as  standard  educational  practice,  received  the  anticipated  exempt  classification  after  review  by 
the  Institutional  Review  Board  Committee  for  the  Protection  of  Human  Subjects  at  the  university  where  the  study  was  conducted. 
In  compliance  with  that  process,  all  participants  were  fully  apprised  in  writing  of  the  nature  of  the  study,  their  option  for 
voluntary  participation,  and  their  option  to  participate  in  having  examples  of  their  work  displayed  as  results  of  the  study  were 
disseminated.  Of  the  eight  students  who  completed  portfolios  prior  to  graduation,  seven  agreed  to  participate  in  the  study.  The 
resulting  88%  return  rate,  represents  five  of  the  six  females  and  both  of  the  males;  an  acceptable  rate  to  provide  an  initial 
perspective  of  the  perceived  usefulness  of  the  five  domains  for  communicating  professionalism  as  educational  technologists 
through  the  medium  of  electronic  portfolios. 

Data  Gathering 

Data  are  based  on  quantitative  and  qualitative  responses  from,  and  the  actual  portfolios  created  by,  participants  and  on 
qualitative  data  from  their  faculty  members. 

Survey  Instrument. 

As  no  published  survey  was  identified  that  addressed  the  questions  being  studied,  a written  survey  was  developed.  Questions 
designed  to  capture  demographic  data  about  gender,  employment,  planned  portfolio  usage,  experience  producing  portfolios  prior 
to  creating  these  professional  portfolios,  and  awareness  of  the  five  domains  of  an  educational  technologist  were  included  in  the 
survey.  The  students’  views  of  the  usefulness  of  the  five  domains  were  operationalized  in  terms  of  five  constructs.  The  five  are 
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the  utility  of  employing  the  domains  in  their  portfolios  for  communicating:  1)  the  students'  knowledge,  2)  the  students'  skills,  3) 
the  range  of  the  students'  professionalism,  4)  the  contribution  the  domains  made  to  the  students'  descriptions,  and  5)  the  students' 
intent  to  retain  the  domains  in  their  portfolios  after  graduation.  Their  reactions  to  these  five  aspects  were  captured  on  a ten  item 
survey  that  has  a negatively  and  positively  worded  item  for  each  aspect. 

The  Loyd  and  Gressard  (1986)  Computer  Attitude  Survey  consisting  of  40  positively  and  negatively  worded  items  about 
attitudes  toward  computers  served  to  guide  the  construction  of  these  types  of  questions.  For  example,  the  two  items  about 
perceptions  of  skills  appear  as,  "The  5 domains  are  useful  in  my  electronic  portfolio  in  describing  my  skills  in  the  field  of 
educational  technology"  and  "My  skills  can  be  described  better  if  descriptors  other  than  the  five  domains  are  used."  A graduate 
student  in  the  program  also  offered  one-on-one  feedback  about  the  survey  during  its  construction. 

Students  indicated  their  degree  of  agreement  on  a four  point  Likert-like  scale  that  ranged  from  "strongly  agree"  to  "strongly 
disagree".  In  addition,  five  open-ended  questions  were  developed  to  obtain  input  about  the  type  of  examples  that  students' 
selected  for  their  portfolios  when  expressing  their  performances  in  each  of  the  five  domains  of  an  educational  technologist.  A 
final  broad,  open-ended  question  sought  additional  comments. 

Artifacts. 

The  students'  electronic  portfolios  also  served  as  a means  of  verifying  application  of  the  domains. 

Faculty  Responses. 

The  five  full-time  departmental  faculty  members  also  provided  oral  input  after  viewing  the  students'  formal  presentations  of 
their  portfolios.  The  relatively  small  size  of  the  department  (5  full-time  faculty  members  and  c.  50  master's  degree  candidates)  did 
not  permit  the  separation  of  the  investigator  as  a participant. 

Data  Analysis 

During  analysis,  an  emphasis  on  the  more  objective  data — the  quantitative  responses  from  the  students — and  validation  of  the 
faculty  members'  judgements  expressed  as  emergent  themes  about  the  students'  portfolios  and  presentations  were  confirmed  by 
all  the  faculty  members.  This  approach  was  taken  to  mitigate  to  some  degree  any  potential  for  bias  that  might  occur  from  the 
close  association  between  the  primary  investigator  and  resulting  interpretations  of  the  data.  Despite  the  small  numbers  of 
participants,  the  multiple  sources  of  data  collected  in  this  study  do  have  potential  to  offer  an  initial  view  of  a process  that  others 
may  wish  to  employ  to  some  degree  as  they  engage  in  similar  efforts. 

Quantitative  Data 

Quantitative  data  were  analyzed  in  terms  of  frequency  distributions.  From  those  analyses  the  picture  that  appears  shows  that 
among  the  seven  respondents  who  provided  usable  data,  two  were  unemployed  prior  to  graduation,  four  were  working  as 
graduate  assistants  in  a technology  support  capacity,  and  one  was  employed  in  a network  security  role.  Five  had  had  no 
experience  producing  any  type  of  portfolio  prior  to  initiating  work  on  their  professional  electronic  portfolios  for  their  master's 
program.  All  of  the  participants  indicated  that  they  only  become  aware  of  the  five  domains  of  an  educational  technologist  as  they 
completed  their  electronic  portfolios. 

To  more  readily  express  the  sentiments  of  the  limited  number  of  participants,  the  "strongly"  and  "slightly"  agree  categories 
for  both  agreement  and  disagreement  were  collapsed  into  two  categories  of  agreement  and  disagreement. 

As  indicated  in  Figure  1 , the  five  domains  are  viewed  ® contributing  to  the  communication  of  knowledge  and  skills  as  an 
educational  technologist  by  six  of  the  seven  respondents  for  the  former  aspect  and  all  of  the  students  for  the  latter  one. 
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as  Other  Means 

Knowledge  Description  Skills  Description 


Fi2ure  1.  Perceptions  of  the  usefulness  of  the  five  domains  of  an  educational  technologist  when  conceptualizing  professionalism 
as  an  educational  technologist  in  a professional  electronic  portfolio. 

Despite  the  positive  regard  for  applying  the  domains  to  describe  skills,  negativity  predominates  (n=4)  in  regard  to  the  domains 
being  used  for  skill  description  rather  than  some  other  means. 

Figure  2 reflects  students'  responses  to  the  picture  of  the  domains,  and  the  perceived  contribution  to  professionalism,  as  well 
as,  to  confusion  during  use.  For  expressing  overall  professionalism,  all  but  one  of  the  respondents  view  the  domains  as  being 
helpful  and  not  hindering  such  a depiction.  Most  (n=5)  view  the  domains  as  contributing  to  depicting  their  range  of  capabilities. 
The  same  number,  however,  do  agree  that  during  production  the  domains  are  confusing  to  use. 
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Fimire  2.  Perceptions  of  the  usefulness  of  the  five  domains  of  an  educational  technologist  when  conceptualizing  one's  range  of 
professionalism  and  as  an  educational  technologist  and  of  the  contribution  that  the  domains  made  to  this  picture  in  a 
professional  electronic  portfolio. 

That  same  number  of  students  state  they  would  not  employ  the  domains  if  they  were  not  required  to  do  so,  although  five  of 
them  do  plan  to  retain  usage  of  the  domains  (see  Figure  3). 
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Fi^re  3.  Perceptions  of plans  to  retain  the  five  domains  of  an  educational  technologist  to  communicate  professionalism  as  an 
educational  in  a professional  electronic  portfolio. 

Students*  Qualitative  Data 

Six  of  the  seven  respondents  completed  the  open-ended  section  of  the  survey  that  related  to  the  manner  in  which  they 
conceptualized  the  examples  they  selected  to  represent  their  professionalism  for  each  of  the  five  domains.  Half  briefly  referenced 
their  selections.  Each  domain,  for  example,  is  expressed  in  the  following  manner:  for  design,  "Grant  from  ETEC  600;"  for 
development,  "ETEC  603  Designing  a paper-based  module;"  for  management,  "ETEC  650  assignments;"  for  utilization,  "ETEC 
662  Planning  a computer  lab  (networking)";  and  for  evaluation,  "final  project"  [master's  study].  Of  those  respondents  who 
expressed  their  choices  in  this  manner,  they  referred  specifically  to  nine  courses,  five  of  which  are  required  courses  in  the 
master's  degree  program. 

The  other  three  respondents  offered  brief  narratives  about  their  knowledge  and  skills,  rather  than  referencing  a course  alpha 
and  number.  For  example,  in  relation  to  the  domain  of  design,  one  student  wrote,  "Print -based  instructional  module — surfing 
lessons"  This  instructional  module  was  designed  following  the  Dick  & Carey  instructional  design  model."  A similar  example,  in 
relation  to  the  domain  of  management  is  expressed  as, 

A visit  to  UHM  video  (on  Products  Page)  based  on  Mintzberg's  (1971)  principles  of  mgmt.  I was  the  director  of  the  video.  I 
served  as  the  liaison  between  the  various  actors  groups  for  resources  & communication.  I was  resource  allocator  & the  negotiator 
in  that  I made  sure  the  project  was  recorded,  edited,  & turned  in  on  time. 

Analysis  of  Portfolios 

A review  of  the  portfolios  themselves  by  all  the  departmental  faculty  members  indicates  that  all  the  students  completed  their 
portfolios,  addressing  design  and  content  requirements  to  a degree  deemed  acceptable  for  this  newer  departmental  requirement. 

Comparison  of  Faculty  Members*  and  Students*  Qualitative  Data 

Qualitative  data,  the  students'  responses  to  the  open-ended  questions,  their  actual  portfolios,  and  the  faculty  members'  oral 
comments  about  the  portfolios  were  analyzed  in  terms  of  emergent  themes. 

Emergent  themes  during  the  analysis  of  the  open-ended  comments  offered  by  both  the  students  and  the  faculty  members 
indicate  that,  although  there  is  some  convergence  in  regard  to  the  issues,  students'  comments  relate  primarily  to  learner  guidance 
about  the  domains.  Specifically,  they  mention  the  need  to  provide  information  about  the  domains  at  the  beginning  of  the  process 
with  details  about  the  domains  as  they  develop  their  portfolios.  There  are  suggestions  related  to  the  format  that  the  guidance 
could  assume,  such  as,  a brochure  about  the  domains  and  examples  of  portfolios  that  integrate  explicit  domain-based  information. 

To  some  extent  the  feedback  from  the  faculty  members  reflects  the  students'  ideas.  The  faculty  members'  note  the  need  for 
the  candidates  to  be  clear  and  overt  in  explanations  about  the  domains  in  their  electronic  portfolios.  To  achieve  such  ends  and 
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overall  professionalism  in  portfolios,  the  faculty  members  cite  a need  for  a more  structured  ongoing  peer  and  faculty  feedback 
process  during  the  two  plus  years  of  design  and  development. 

In  terms  of  professionalism,  the  faculty  members  state  that  such  posture  needs  to  be  emphasized  in  the  design  and  selection 
of  content.  They  explicitly  note  this  situation  in  terms  of  the  degree  of  emphasis  given  to  work-related  information  over  personal 
data;  selection  of  concise  exemplars;  use  of  sophisticated  design  elements,  including  decisions  about  selecting  appropriate 
graphics;  and  increased  focus  on  the  candidates’  future  worth.  Formatting  is  also  mentioned  with  recommendations  for  the 
inclusion  of  alt  tags  and  pdf  files  to  ensure  that  awareness  of  ADA  and  printing  concerns  are  conveyed. 

Of  particular  note  is  a reminder  to  address  privacy  issues  in  terms  of  self  and  others,  such  as  considering  the  amount  of 
personal  information  to  make  readily  available  on  the  WWW  and  obscuring  references  to  specific  individuals,  or  educational 
sites,  that  might  occur  in  samples  included  in  the  portfolios.  They  also  call  for  developing  a way  for  the  students  to  communicate 
the  rich  content  of  these  portfolios  in  a reasonable  timeframe,  while  ensuring  professionalism  during  presentation. 

Discussion 

Despite  introducing  the  concept  of  analyzing  professional  knowledge  and  ^ills  and  overall  range  of  professionalism  in 
relation  to  the  five  domains  of  their  field  toward  the  completion  of  the  portfolio  process,  it  is  encouraging  to  note  the  students’ 
overall  positive  reaction  to  the  utility  of  that  framework.  The  process  of  sy  nthesis  required  of  them,  rather  than  analysis,  can  be  a 
more  challenging  intellectual  skill.  This  cognitive  demand  may  be  reflected  to  some  extent  in  the  students’  mixed  feelings  about 
the  contribution  that  the  domains  make  in  conveying  the  range  of  the  students'  capabilities. 

Perceptions  of  Domains 

Although  they  primarily  agreed  (n=5)  that  the  domains  did  contribute  to  this  picture,  the  same  number  also  noted  that  during 
production  the  domains  were  confusing  to  use.  Their  comments  about  the  need  for  additional  guidance  in  regard  to  the  domains, 
particularly  their  request  for  brochures  and  examples  from  the  portfolios  of  others,  offer  support  for  providing  additional 
information  for  future  learners  both  about  the  domains  themselves  and  about  applying  them  to  interpret  and  communicate  their 
capabilities. 

At  first  it  might  appear  that  the  students  do  not  value  the  domains  as  descriptors,  as  the  majority  of  the  students  (n=5)  indicate 
that  they  would  not  employ  them  unless  required  to  do.  Their  acknowledgement  (n=5),  however,  that  they  plan  to  retain  the 
domains  as  descriptors  during  revisions  appears  to  reflect  the  worth  which  they  apparently  are  beginning  to  place  on  the  domains. 
A review  of  the  final  versions  of  the  portfolios  at  graduation,  when  minor  revisions  were  made  after  the  formal  presentations  also 
supports  this  contention. 

Expanded  descriptions  provided  by  three  of  the  students  in  regard  to  addressing  each  of  the  domains  in  their  portfolios,  as 
opposed  to  the  other  students’  succinct  references  to  course  alpha  and  numbers,  suggest  a deeper  integration  of  the  concepts  by 
the  former.  The  reason,  however,  is  not  evident  from  data  gathered  in  this  study. 

Process  Issues 

Instructional  design  issues,  thus,  are  the  predominant  concerns  identified  by  responses  to  the  survey,  in  the  actual  portfolios, 
and  in  the  faculty  members'  evaluations.  Contextual  issues  related  to  resources — hardware  and  software  accessibility — did  not 
appear  in  any  comments  other  than  a reminder  from  the  faculty  members  about  aisuring  that  the  required  CD-ROM  copy  of  the 
portfolios  continue  to  be  made  to  ensure  that  the  students'  work  not  be  lost  in  '’cyberspace.'’  The  absence  of  concerns  about 
hardware  and  software  where  not  surprising.  As  majors  in  an  educational  technology  program,  personal  ownership  of  a computer 
is  expected.  Additionally,  generous,  successful  grant  funding  has  resulted  in  a relatively  high-end  technology  laboratory  being 
available  not  only  for  these  majors,  but  for  all  in  the  college  of  education. 


Implications 

The  students  do  offer  perspectives  that  indicate  their  awareness  of,  as  well  as  the  usefulness  of,  incorporating  the  five 
domains  into  their  portfolios,  as  well  as  a variation  in  the  level  of  conceptualizing  domain  usage.  As  professionals  in  the  field  of 
educational  technology,  planning  for  effective  transfer  of  critical  learning  experiences  is  an  essential  part  of  their  learning.  The 
results  of  this  preliminary  investigation  about  graduate  level  educational  technology  students’  perspectives  of  engaging  in  the 
electronic  portfolio  process  and  their  depictions  of  themselves  as  designers,  developers,  users,  managers,  and  evaluators  in  terms 
of  these  five  domains  suggest  future  directions  for  faculty  members  in  similar  situations. 

Incidental  information  related  to  candidates’  portfolio  development  offers  a glimpse  of  the  potential  connection  that  might 
result  from  the  students’  efforts  in  designing  and  developing  their  portfolios.  Based  on  a recent  summary  of  Kirkpatrick’s  classic 
Four-Level  Evaluation  Model  from  the  late  1950’s  (2001),  at  Level  One,  Reaction,  the  participating  students  hold  predominantly 
positive  attitudes  toward  the  utility  the  domain-centered  framework  provides.  At  Level  Two,  Learning,  they  appear  capable  of 
communicating  their  knowledgp  and  skills  in  relation  to  the  domains.  A slight  suggestion  of  how  Level  Three,  Behavior  on  the 
Job,  is  offered  in  terms  of  most  students  indicating  that  they  plan  to  retain  the  domains  as  descriptors  in  future  revisions  of  their 
portfolios.  It  may  be  that  they  are  planning  ongoing  usage  in  future  work  situations.  It  is  nearly  impossible  to  determine  in  any 
substantive  manner  insights  about  such  portfolio  usage  in  terms  of  Kirkpatrick’s  evaluation  Level  Four,  Results,  as  it  refers  to 
impact  on  the  job.  Preliminary,  incidental  data  do,  however,  offer  a very  faint  preliminary  sketch.  One  student  nearing 
graduation,  who  produced  a portfolio  before  the  requirement  to  overtly  address  the  domains  existed,  received  two  university- level 
job  offers  within  three  days  of  posting  his  portfolio  on  the  WWW.  This  outcome  resulted  despite  his  avoidance  of  posting  his 
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work  on  a job  recruitment  site.  A recent  student,  who  incorporated  the  domains  in  her  portfolio  and  obtained  a position  as  a 
lecturer  at  a two-year  college,  was  told  after  being  selected  that  her  portfolio  contributed  to  the  decision  to  hire  her. 

The  mere  suggestion  of  such  impact  makes  continuing  efforts  in  refining  the  professional  electronic  portfolio  process, 
particularly  for  educational  technology  majors  who  are  expected  to  communicate  their  professionalism  using  newer  technologies, 
essential.  It  is  particularly  critical  as  the  portfolio  process,  in  varying  formats,  gains  increasing  attention  for  pre-  and  in-service 
educators  in  all  fields  (Campbell,  Cignetti,  Melenyzer,  Nettles,  & Wyman,  Jr.,  2001;  Sanders,  2000). 

Additionally,  scaffolding  candidates'  professional  development  is  an  important  theme  for  virtual  universities  (Collis,  1997).  It 
may  well  be  that  an  electronic  portfolio  development  process  wit  hin  the  WWW  environment,  one  that  is  continually  scrutinized 
in  terms  of  the  process  and  output  can  provide  such  support  for  educational  technology  candidates.  Future  studies  conducted  by 
investigators  at  institutions  of  higher  education  in  the  United  States,  as  well  as  ones  overseas,  that  offer  graduate  programs  in 
educational  technology  would  seem  warranted  to  determine  the  ultimate  value  of  this  intended  exemplar  of  professionalism. 
Given  the  time  and  effort  that  both  faculty  and  students  expend  on  the  p rocess  it  is  critical  that  such  studies  occur. 
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Abstract 

This  paper  details  the  theoretical  underpinnings  of  one  university's  approach  to  technology  integration  in  its  pre-service 
teacher  preparation  programs,  and  the  results  of  a continuous,  feedback -driven  project  to  evaluate  for  technology  integration 
through  a student  portfolio  development  process.  Portfolios  are  assessed  for  multiple  education  and  technology  standards. 
Results  of  the  project  to  date  as  well  as  future  plans  are  shared  and  discussed, 

Georgia  State  University  received  a Preparing  Tomorrow’s  Teachers  to  Use  Technology  Grant  at  the  Implementation  level 
during  the  first  year  of  grant  awards.  In  the  first  year  of  the  grant,  the  College  of  Education,  as  a whole,  worked  to  put  standards 
and  assessment  procedures  into  place  to  meet  state  and  national  technology  standards.  Due  to  a reorganization  of  the  project 
management  team  prior  to  the  second  year  of  funding,  a different  approach  was  deemed  necessary  to  meet  the  requirements  of 
the  grant  funding.  Mini-grants  were  awarded  to  faculty  teams  to  support  grass-roots  level  technology  integration  projects.  This 
paper  reports  on  the  development  efforts  of  one  such  mini- grant.  This  PT3  mini- grant  had  two  foci:  to  continue  the  development 
of  an  on-line  support  system  to  assist  preservice  teachers  in  meeting  the  NETS-T  standards,  and  the  extension  of  a portfolio 
development  and  assessment  process  to  include  assessment  for  NETS-T  standards. 


Teacher  Technology  Integration  Skills 

Federal,  state,  and  local  agencies  are  investing  billions  of  dollars  to  equip  schools  with  the  technology  that  may  well  be  the 
key  to  improving  the  learning  experience  of  our  nation’s  youth.  Despite  these  gargantuan  investments,  only  20  percent  of  the  2,5 
million  teachers  currently  working  in  public  schools  feel  comfortable  using  these  technologies  in  the  classroom  (U,S,  Department 
of  Education,  1999;  Technology  Counts,  1997),  As  reeducating  the  existing  teaching  force  to  take  full  advantage  of  these 
technology  tools  will  require  expensive  professional  development  over  many  years,  the  preparedness  of  new  teachers  entering  the 
field  becomes  critical  to  the  success  of  these  investments  to  improve  education.  As  the  federal  government  predicts  that  2,2 
million  new  teachers  will  be  needed  in  the  next  decade,  the  time  to  address  these  issues  is  now  (Milken  Exchange  on  Education 
Technology,  1999). 

Computer  technology  has  been  available  for  use  in  educational  settings  for  several  decades.  According  to  a survey  of  U,S, 
state-level  technology  officials  (Trotter,  1999),  42  states  require  teacher  preparation  programs  to  include  technology.  One  might 
think  that  by  this  time  colleges  of  education  (COEs)  are  successfully  preparing  teachers  to  integrate  technology  into  instructional 
practices.  However,  this  has  not  necessarily  been  the  case.  In  1995,  the  U,S,  Office  of  Technology  Assessment  (OTA)  published 
a report  on  the  status  of  teachers  and  technology.  According  to  the  OTA,  teachers  were  not  and  did  not  feel  adequately  prepared 
to  integrate  technology  into  their  teaching  practices.  One  of  the  contributing  factors  cited  was  the  lack  of  technology  training 
available  in  teacher  preparation  programs  at  colleges  of  education  (COE),  When  technology  instruction  was  provided,  it  involved 
teaching  about  technology  not  teaching  with  technology.  In  most  instances,  COE  faculty  did  not  model  technology  integration 
with  their  preservice  students. 

According  to  a recent  survey  of  416  teacher  preparation  institutions  commissioned  by  the  Milken  Exchange  of  Education 
Technology,  most  faculty  members  did  not  model  the  use  of  instructional  technology  skills  in  their  teaching  (Moursund  & 
Bielefeldt,  1999),  In  several  studies  it  appears  that  faculty  who  are  not  modeling  are  also  not  requiring  students  to  use  technology 
in  their  lessons  or  assignments  (Lewallen,  1998;  U,S,  Congress,  1995;  Wetzel,  1993), 

However,  a report  produced  by  the  U,S,  Department  of  Education  (2000  revealed  refreshing  news:  less  experienced  teachers 
were  more  likely  than  experienced  colleagues  to  indicate  that  college  course  work  prepared  them  to  use  computers  in  their 
classrooms,  “84  percent  of  teachers  with  3 or  fewer  years  and  76  percent  of  teachers  with  4 to  9 years  of  teaching  experience 
reported  that  college/graduate  work  prepared  them  to  use  these  technologies  to  any  extent,  compared  with  44  percent  of  teachers 
with  10  to  19  years  and  31  percent  of  teachers  with  20  or  more  years  of  teaching  experience”  (p,  78),  While  teacher  education 
programs  still  face  obstacles  as  they  prepare  preservice  teachers,  it  is  evident  they  are  making  in-roads. 

Teacher  education  programs  across  the  country  struggle  with  how  to  increase  the  technology  integration  skills  of  the  students 
they  educate.  Not  only  are  standards  such  as  the  National  Educational  Technology  Standards  for  Teachers  (NETS-T) 
(International  Society  for  Technology  in  Education  National  Educational  Technology  Standards  Project,  2000)  being  adopted, 
states  are  now  requiring  institutions  to  guarantee  the  technology  proficiency  of  their  graduates  (University  System  of  Georgia 
Board  of  Regents,  1998),  At  the  rate  of  technological  innovation,  will  it  make  any  difference  if  pre-service  teachers  are  taught 
simple  productivity  skills  for  both  themselves  and  their  students  once  the  technology  changes?  What  exactly  should  we  be 
teaching  pre-service  educators  about  using  technology  in  their  future  classrooms?  The  answer  to  these  questions  might  help 
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teacher  educators  to  focus  not  only  on  what  content  and  skills  need  to  be  taught,  but  also  on  what  instructional  strategies  might  be 
used  in  teaching  about  technology  integration. 

Factors  Hindering  Technology  Use  in  the  Classroom 

Several  factors  have  been  cited  as  hindering  new  teacher  use  of  technology.  These  include  inadequate  training  in  proper 
technology  skills  and  methods,  lack  of  technology  modeling  on  the  part  of  their  university  faculty,  lack  of  positive  technology 
experience  in  school  settings,  and  university  faculty  out-of-touch  with  the  technology  explosion  in  schools  and  how  it  is  effecting 
teaching  practice  (Kent  & McNergnery,  1999;  National  Council  for  Accreditation  of  Teacher  Education,  1997;  Persichitte,  Tharp, 
& Cafarella,  1997;  Office  of  Technology  Assessment,  1995;  Byrum  & Cashman,  1993).  The  re-design  of  entire  teacher  education 
programs  is  called  for. 

To  this  end,  the  Instructional  Technology  unit  at  Georgia  State  University  determined  to  undertake  a two-fold  approach  to 
increasing  the  technology  integration  skills  of  preservice  teachers.  First,  in  order  to  facilitate  preservice  training  in  proper 
technology  skills,  and  methods,  an  on-line  support  system  was  developed  not  only  for  course  use,  but  also  to  support  the 
preservice  teacher  throughout  student  teaching  and  on  throughout  the  critical  induction  experience.  Secondly,  to  address  teacher 
education  faculty  awareness  and  knowledge  of  teacher  technology  integration  skills,  and  to  increase  modeling  to  technology 
integration  skills  throughout  the  preservice  teacher  experience,  a portfolio  development  process  was  instituted  with  the  Middle 
Childhood  Education  unit.  Portfolios  are  assessed  throughout  the  preservice  program  for  meeting  first  technology  standards,  and 
then  for  meeting  new  teacher  standards. 

Early  in  1999,  Georgia  Governor  Roy  Barnes  formed  an  Education  Reform  Study  Commission  to  look  at  ways  to  improve 
Georgia’s  schools.  Governor  Barnes  used  the  results  of  the  commission’s  study  to  produce  the  A Plus  Education  Reform  Act  of 
2000  (2002).  Out  of  the  act  came  technology -related  initiatives  that  impact  teachers  and  teacher  preparation  programs.  Primarily 
the  act  holds  teacher  preparation  programs  at  universities  and  colleges  responsible  for  their  graduates’  technology  competencies. 
Universities  and  colleges 

shall  require  students  in  such  programs  to  be  proficient  in  computer  and  other  instructional  technology 
applications  and  skills  including  understanding  desktop  computers,  their  applications,  integration  with 
teaching  and  curriculum,  and  their  utilization  for  individualized  instruction  and  classroom  management. 

There  shall  be  a test  to  assess  the  proficiency  of  students  oirolled  in  teacher  preparation  programs  in 
computer  and  other  instructional  technology  applications  and  skills,  (p.  68) 

Prior  to  the  A Plus  Education  Reform  Act  of  2000  (2000),  the  Middle,  Secondary  and  Instructional  Technology  Department 
(MSIT)  recognized  the  need  to  prepare  pre-service  teachers  in  the  area  of  technology  integration.  The  Instructional  Technology 
unit  has  been  working  closely  with  the  Middle  Childhood  Education  unit  for  the  past  four  years  to  develop  a WWW-supported 
resource  based  learning  environment  (RBLE)/methods  course  called  Teachers  and  Technology^  IT  3210.  This  course  focuses  on 
and  models  technology  integration.  It  is  a required  course  for  Middle  Childhood  (MCE),  Secondary  Education  (SEC)  students 
and  Foreign  Language,  Art,  and  Music  (FLAM)  students  seeking  teacher  certification,  and  an  elective  course  for  Early  Childhood 
Education  students  (ECE),  as  well  as  Kinesiology/Health  (K/H)  students  seeking  certification.  The  pre-service  students  generate 
a portfolio  documenting  the  design  of  technology -supported  instructional  environment  that  facilitates  student  learning  through  the 
design  and  development  of  student -centered  learning  activities.  The  IT3210  portfolio  serves  as  a starting  point  for  middle  grades 
students  who  continue  with  the  portfolio  assessment  process  throughout  their  studies  at  GSU  demonstrating  they  have  met 
FNTASC  (Interstate  New  Teacher  Assessment  and  Support  Consortium)  Standards  (Council  of  Chief  State  School  Officers, 
1999). 

IT3210  and  its  web-based  resources  address  the  National  Educational  Technology  Performance  Profiles  for  Teachers  as  well 
as  supports  all  six  of  the  National  Educational  Technology  Standards  for  Teachers  (NETS-T)  and  contributes  to  student 
understanding  of  the  TNTASC  Standards.  The  middle  grades  portfolio  assessment  addresses  only  the  FNTASC  Standards. 

Program  Issues  - Stand  Alone  Course  or  Cross  Curriculum  Integration? 

Many  teacher  education  programs  focus  on  either  a stand-alone  course,  or  on  a model  of  technology  infused  throughout  all 
teacher  preparation  courses.  Some  schools,  including  GSU,  have  opted  to  do  both.  Kovalchick  (1997)  offers,  “An  approach  that  I 
have  found  useful  is  to  blend  elements  from  both  a competency  based  models  and  integrative  models  into  a reflexive  approach  in 
which  students  use  technology  as  both  learner  and  teacher.  In  this  way,  preservice  teacher  education  students  are  challenged 
through  direct  experience  to  generate  personally  relevant  conceptions  of  technology”  (p.  31).  Smaldino  and  Muffoletto  (1997) 
also  promote  a combination  approach.  “Our  model  attempts  to  blend  the  contents  of  the  existing  single  course  with  the  need  to 
nurture  technology  applications  within  methods  and  other  courses.  Thus,  students  first  gain  an  understanding  of  the  applications 
of  technology  in  education  in  the  broad  sense,  with  an  in-depth  examination  of  how  technology  supports  learning  in  specific 
content  areas”  (p.37). 

Technology  Integration  Support  for  Preservice  Teachers 

Course  Design  History 

Prior  to  1997,  the  technology  course  at  GSU  was  a stand-alone,  skills-based  course  that  focused  on  the  use  of  technology  as  a 
teacher  tool.  Content  included  such  technology  usage  as  word  processing,  mail  merging  a letter  home  to  parents,  and  using  a 
spreadsheet  program  to  calculate  grades.  Little  to  no  learning  theory  or  instructional  methods  were  included  in  the  lab-based 
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course.  In  addition,  the  technologies  covered  were  basic  in  nature  - telecommunications  coverage  consisted  of  e-mail,  and  in  later 
years,  the  Internet  as  a database  of  lesson  plans.  As  pedagogy  played  virtually  no  role  in  the  course,  students  were  allowed  to 
substitute  a passing  grade  on  a pencil  and  paper  competency  test. 

In  1997,  at  the  request  of  the  Middle  Childhood  Committee,  the  standard  skills -based  preservice  technology  curse 
underwent  a major  redesign.  In  the  first  year,  the  course  refocused  from  teacher-resource-based,  skills-based  to  a technology - 
integration-into-the-curriculum  approach.  This  refocus  was  done  in  part  to  address  a potential  cause  of  low  technology  adoption 
in  preservice  teachers:  deficiencies  in  technology  -integration  methods  (Leggett  & Persichitte,  1998). 

In  fall  semester  1998,  the  IT  unit  worked  with  the  MCC  to  redesign  the  course  to  further  situate  the  course  content  in 
teaching  methods.  While  maintaining  a lecture/lab  approach,  a WWW-based,  resource-based  learning  environment  (RBLE)  was 
introduced  as  part  of  the 

course  (Hill,  1999;  Shoffner,  1999).  The  course,  and  its  related  resource  laden  WWW  site,  incorporates  a problem-centered, 
activity-based  approach  where  the  computer  applications  are  anchored  in  authentic  and  familiar  contexts  in  which  teaching  and 
learning  occurs  (Cognition  and  Technology  Group  at  Vanderbilt,  1991;  Vygotsky,  1978).  This  approach  is  based  on  the  view  of 
an  open  learning  environment  in  which  learners  have  direct  input  on  the  direction  of  the  course  based  on  their  needs  (Hannafin, 
1999;  Hannafin,  Hall,  Land,  & Hill,  1994).  In  navigating  through  the  environment  and  tackling  challenges,  it  is  proposed  that 
students  will  also  develop  self-directed  learning  skills,  which  will  serve  them  well  as  they  enter  the  teaching  profession.  Along 
with  confidence  in  using  the  technology,  self-directed  learning  skills  have  been  identified  as  a characteristic  of  successful 
technology -using  teachers  (Shoffner,  1996).  The  RBLE  can  be  accessed  at  http://msit.gsu.edu/IT/3210/index.html , The  site  map 
for  the  course  appears  below  in  Figure  1 . 

At  the  same  time,  the  course  serves  as  an  introductory  teaching  methods  course,  introducing  preservice  students  to  such 
concepts  as  instructional  objectives,  lesson  planning,  evaluation,  and  assessment.  The  course  offers  more  than  teaching  the  basic 
ADDIE  instructional  design  model  as  a way  to  develop  lesson  plans  while  teaching  about  technology  integration  skills.  In  the 
Technology  for  Teachers  course  at  GSU,  the  technology  is  immersed  in  learning  about  what  being  a teacher  entails  - briefly, 
planning,  learning  theory,  instructional  strategies,  classroom  management,  and  assessment.  Our  hope  is  that  by  introducing  the 
technology  and  the  methods  together,  early  in  the  program,  that  a)  students  will  forever  forward  view  technology  as  natural  to  the 
learning  process  as  the  textbook  and  the  pencil;  and  b)  both  the  technology  and  the  methods  will  be  reinforced  throughout  their 
other  courses  at  GSU.  One  way  in  which  continuity  and  reinforcement  occurs  is  in  the  use  of  portfolios  for  assessment.  In  the 
Technology  for  Teachers  course,  preservice  students  generate  a portfolio  documenting  the  design  of  technology -supported 
instructional  environment  that  facilitates  student  learning  through  the  design  and  development  of  student -centered  learning 
activities.  The  use  of  portfolio  development  and  assessment  continues  throughout  the  remainder  of  Middle  Childhood  Education 
program  of  study. 
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IT  3210:  Teachers  and  Technology 
Course  Site  Map 


Available  at:  http://msit.gsu.edu/IT/32lO 
Course  Coordinotor:  Dr.  M.B.  Shoff  ner 


IT  3210  Technology  for 
Teachers  Course  Home  Page 


Discussion  1: 

What  Mokes  o 6ood  Teacher? 


Instructors 
(background  and 
contact  information) 


Discussion  2: 

Information  Literacy/ Electomic 
Communications 


Discussion  3; 
Visud  Literacy 


Discussion  4: 
Informotion  Copyright/ 
Technology  Inequities 


Discussion  5: 

Support  Roles  ond  Reosueoes  in 
0 School 


Course  We  blinks 
(links  to  on-line 
resources) 


Course  On-line 
Discussion  Bulletin 
Board  (link  to  WebCT 
gateway) 


Unit  Work  Samples 

Guidelines 

(Weekly  Assignments) 

Learning  Environment  Portfolio  \ ^ 

of  an  instructional  unit)  1/  \ 

Assessment  Rubrics 

Ref  lection  Papers 


On-line  Software  A 
Wethods  Tutorials 


Figure  1.  Course  Site  Map  of  IT  3210:  Teachers  and  Technology 


From  Course  to  Ou-Line  Support  System  for  NETS-T 

One  focus  of  this  PT3  mini-grant  was  to  expand  the  IT  3210  Teachers  and  Technology  website  to  provide  not  only  a 
backdrop  for  the  course  but  also  to  serve  as  a resource  for  pre-service  teachers  during  their  student  teaching  experience  as  well  as 
their  first  year  induction  period  into  the  teaching  profession.  To  facilitate  this  continued  use,  we  redesigned  the  interface  of  the  IT 
3210  Teachers  and  Technology  resource  based  learning  environment  (RBLE).  Using  the  PT3  website  development  done  by  the 
project  director  in  summer  2000,  materials  were  merged  into  one  master  RBLE.  The  interface  was  changed  from  a homepage - 
linked  pages  interface  (http://msit.gsu.edu/IT/3210/index.htmn  (Figure  2,  below)  to  a frame-set  interface 
(httD://msit.gsu.edu/PT3/Shoffner/index.htmn  (Figure  3,  below).  The  grant  team  felt  this  change  was  necessary  to  facilitate 
navigation  through  what  is  becoming  a very  large  resource  site.  When  complete,  the  site  will  serve  not  only  the  IT  3210  audience, 
but  also  as  a mentoring  resource  to  new  teachers  in  the  field.  Furthermore,  the  new  frameset  navigation  will  allow  users  from 
other  universities  to  more  easily  access  portions  of  the  site  for  their  students  (we  continue  to  receive  two  to  three  requests  for 
such  use  each  semester).  We  had  hoped  the  interface  frameset  would  be  in  place  for  the  entire  site  (with  placeholder  pages  for 
approximately  on  half  ^e  content)  by  the  mid  March  2001.  However,  the  development  of  the  frameset  interface  required  more 
time  than  initially  anticipated.  Now  that  the  frameset  interface  is  in  place,  work  will  resume  this  fall  semester  on  the  authoring 
and  placement  of  content,  as  well  as  the  identification  of  exemplar  lesson  plans  for  including  in  the  site. 


BEST  COPY  AVAILABLE 


375 


384 


(dt  fiawixTitattf  .'tjib  - c ' 

■f ' ■■  a'V  ra-  A ' & oc  ' a if 

. R*to»C  Mtw  . 8#a«c*>  FW  Soeuty  ' Shop 

>' ’ Jit-  U&d:>(K[HRtr>awi  paj  «duaT/3?l{WdeB  M 

r ^ WfW*l  ^ C«ltnd#  'jOp  ft«*D  ^ Paept*  ^ §1  Opwrbsd  (Ij  OKrawte  / ; •' t 

.■..-.A,  . ■ 

1 Mnl«wr*Ftv 

\ bcporriMnt  «f  Mtddb/&«eon<iarv  Educptlan  & Iurtructiaul 

f 

r 

1 

IT  3210  M 

Teachers  & Technology 

I 

i- 

® Svllabm  ^ Co  ijtk  ftniectf 

• Course  TimcLine  ® Instructorf 

MooilAV  cla»fri  • Rebtod  Web  Link; 

Tuttdav  cU?rci  • WcbCT 

W^cinerdav 

claftes 

■l 

Ovattioiis?  Corjact  die  Ccutjc  Web  Coordinsrer 

'i 

.1 

Last  updaiod:  17  Aug/dSJ  2001 

'1 

Figure  2.  Screen  Snap  of  Current  Interface  of  IT  3210:  Teachers  and  Technology 


Figure  3.  Screen  Snap  of  Frameset  Interface  of  IT  3210:  Teachers  and  Technology  Under  Development 

To  assess  continued  use  of  the  IT  321 0 RBLE  by  preservice  teachers  throughout  their  program  of  study,  we  planned  to  mail  a 
follow-up  survey  to  fall  2000  IT  3210  students  during  the  Spring  2001  semester.  Informal  interviews  with  fall  2000  IT  3210 
students,  however,  indicated  that  the  time  elapsed  between  the  close  of  the  fall  semester  and  the  planned  mid-March  mailing  was 
not  sufficient.  Students  indicated  they  would  be  more  likely  to  make  use  of  the  RBLE  for  final  semester  projects  and  for  field 
experiences.  Due  to  this  feedback,  the  mailing  was  postponed  to  Fall  2001  semester.  In  spring  2002,  fall  2000  IT  3210  cohort 
students  will  be  surveyed  to  ascertain  their  continued  use  of  the  RBLE  in  their  GSU  coursework.  This  survey  will  seek  responses 
to  the  following  questions: 
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• Do  they  access  the  site  for  support  during  their  GSU  class  experiences? 

• Do  they  access  the  site  for  field-based  experiences? 

• Do  they  feel  prepared  to  use  computers  and  the  Internet  for  classroom  instruction? 

• What  are  their  recommendations  for  the  improvement  of  the  support  site? 

Performance  Profiles  and  Assessment  Instruments 

The  second  portion  of  the  PT3  mini-grant  project  began  more  slowly.  Assessment  instruments  were  developed  for  two  of  the 
four  NETS-T  performance  profiles:  the  general  preparation  performance  profile  and  the  professional  preparation  performance 
profile.  Beginning  Spring  semester,  2001,  all  IT  3210  students  completed  a Likert-type  self-report  on  the  general  preparation 
performance  profile  at  the  beginning  of  the  semester.  As  IT  3210  is  a part  of  the  professional  preparation  of  middle  and 
secondary  education  programs,  students  should  meet  the  general  preparation  performance  profile  prior  to  beginning  the  course.  In 
the  event  that  a student  did  not  meet  the  general  preparation  performance  profile  at  the  beginning  of  professional  studies,  a 
second  administration  occurred  upon  completion  of  the  IT  3210  course.  In  addition,  all  students  complete  a 24-item  self-report  on 
the  professional  preparation  profile  at  the  close  of  IT  3210.  While  self-report  instruments  lack  rigor,  students  who  successfully 
complete  the  IT  3210  course  have  been  more  rigorously  assessed  for  not  only  the  general  performance  profile,  but  also  20  of  the 
24  indicators  on  the  professional  preparation  performance  profile.  In  time,  student  self-report  results  will  be  correlated  to  class 
assignment  grades.  Sample  items  on  the  general  preparation  and  professional  preparation  self-report  instruments  appear  below  in 
figures  4 and  5,  respectively. 


Technology  Standards  for  Pre-Service  Teachers  - General  Preparation 

Please  circle  the  number  corresponding  to  how  well  each  statement  describes  you  at  this  time. 

1 = not  at  all 
2 - a little 
3 = somewhat 
4 - a lot 

5 = very  much  so! 


1 . 1 am  comfortable  with  the  keyboard,  the  mouse,  and  the  basic  operation  | 
of  the  computer. 

2 

3 

4 

5 

2. 1 use  the  computer  for  basic  tasks  on  a regular  basis.  1 

2 

3 

4 

5 

3. 1 use  technology  tools  specific  for  a given  task  (for  example,  graphing  | 

calculators,  simulation  software,  and  so  on). 

2 

3 

4 

5 

4. 1 use  software  tools  to  organize  and  manipulate  data  for  clearer  j 

organization,  problem  solving,  and  decision-making. 

2 

3 

4 

5 

Figure  4:  Sample  Items  from  the  General  Preparation  Performance  Profile  Self-Report  Instrument 

To  date,  few  assessment  instruments  for  NETS-T  developed  by  other  PT3  initiatives  have  been  identified.  Conference 
presentations  and  discussions  with  colleagues  inform  us  that  a comparable  assessment  instrument  for  NETS-T  is  currently  being 
validated  at  another  university.  As  no  other  evaluation  instruments  were  reported  on,  we  anticipate  using  this  validated  rubric 
when  it  becomes  available  to  in  turn  validate  the  GSU  rubrics. 
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Technology  Standards  for  Pre-Service  Teachers  - Professional  Preparation 

Please  circle  the  number  corresponding  to  how  well  each  statement  describes  you  at  this  time. 

1 = not  at  all 

2 = a little 

3 = somewhat 

4 = a lot 

5 = very  much  so! 


1 . 1 can  identify  the  benefits  of  technology  to  maximize  student  learning 
and  facilitate  higher  order  thinking  skills. 

1 

2 

3 

4 

5 

2.  I differentiate  between  appropriate  and  inappropriate  uses  of 
technology  for  teaching  and  learning  while  using  electronic  resources  to 
design  and  implement  learning  activities. 

1 

2 

3 

4 

5 

3.  I can  identify  technology  resources  available  in  schools  and  analyze 
how  accessibility  to  those  resources  affects  planning  for  instruction. 

1 

2 

3 

4 

5 

4.  I can  identify,  select,  and  use  hardware  and  software  technology 
resources  specially  designed  for  use  by  PK-12  students  to  meet  specific 
teaching  and  learning  objectives. 

1 

2 

3 

4 

5 

Figure  5:  Sample  Items  from  the  Professional  Preparation  Performance  Profile  Self-Report  Instrument 

Summary  data  collected  using  the  NETS-T  general  preparation  profile  and  profession  preparation  profile  self-report 
instruments  completed  by  Spring  20001  IT  3210  students  is  included  in  Table  1,  below.  At  the  beginning  of  the  IT  3210  class, 
students  showed  an  average  composite  score  on  the  general  preparation  profile  of  42  points  of  a possible  70  points.  A score  of  42 
is  the  equivalent  of  students  feeling  they  can  “somewhat”  meet  all  14  indicators.  In  comparison,  at  the  end  of  the  class,  average 
student  composite  scores  on  the  general  performance  profile  instrument  for  those  same  students  increased  to  54  points,  indicating 
students  felt  more  adept  and  comfortable  with  the  technology.  A larger  net  gain  in  average  composite  score  would  be  expected; 
these  are  skills  the  student  should  have  prior  to  entering  their  professional  preparation.  One  possible  explanation  for  this  result  is 
that  students  overrate  their  abilities  at  the  beginning  of  a new  course,  particularly  in  the  area  of  technology.  Simply  put,  they 
don’t  know  what  it  is  they  don’t  know.  This  theory  is  supported  by  the  reduction  of  scores  reported  on  the  posttest  by  some 
students,  indicating  they  didn’t  feel  as  confident  in  their  skills  and  usage  after  completed  the  intensive  course. 


General  Preparation 
Performance  Profile 
Self-Report 

Pretest 

General  Preparation 
Performance  Profile 
Self-Report 

Post  test 

Professional  Preparation 
Performance  Profile 
Self-Report 

Post  Test  Only 

Mean- 

Composite  Score 

42.1 

54.0 

91.2 

St.  Dev.- 
Composite  Score 

13.6 

9.8 

18.6 

# items 

14 

14 

29 

Composite  Score 
Possible  Range 

0-70 

0-70 

0-  145 

Composite  Score 
Actual  Range 

18-67 

33-70 

39-120 

Table  1:  NETS-T  Self-Report  Instrument  Summary  Data  - Matched-Pre-Test  Post  test  Data  Only 

MCE  Portfolio  Review 

The  Middle  Childhood  Education  Unit  and  the  Instructional  Technology  Unit  together  formed  a program  oversight 
committee,  the  Middle  Childhood  Committee  (MCC).  At  GSU,  The  MCC  established  a continuous  process  of  portfolio 
development  and  assessment  for  all  students  (Shoffner,  Dias,  & Thomas,  2001Shoffner,  Thomas,  & Dias,  2001).  This  was  a key 
process  for  integrating  technology  across  the  program  and  into  the  content  of  every  course.  In  response  to  the  Board  of  Regents 
guarantee  principle,  increasing  accountability  in  teacher  preparation  programs,  and  the  Middle  Childhood  Committee’s 
recommendation  to  strengthen  the  preservice  teachers’  overall  professional  development,  the  committee  recommended  that  the 
program  include  an  exit  assessment  that  examined  the  student’s  ability  to  apply  what  they  learned  in  all  their  courses  in  some 
cohesive  manner.  After  examining  several  assessment  models,  both  traditional  and  alternative,  a portfolio  development  process 
with  benchmarks  throughout  the  program  and  final  submission  as  an  exit  requirement  was  adopted. 
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Although  most  skills  and  concepts  are  developed  in  individual  courses,  it  is  important  that  preservice  teachers  have  command 
of  these  concepts  and  skills  with  knowledge  of  how  to  integrate  these  concepts  and  skills  into  all  aspects  of  teaching.  Therefore,  a 
major  goal  of  portfolio  requirement  was  to  develop  the  preservice  students’  ability  to  integrate  several  components  of  the 
program  across  all  courses  and  to  develop  knowledge  and  skills  in  applying  these  components  in  all  aspects  of  teaching.  Among 
key  skills  and  concepts  under  discussion  were:  integrating  technology  into  learning,  developing  and  implementing  lesson  plans 
and  assessment  strategies,  developing  and  implementing  a classroom  management  plan,  working  with  diverse  learners, 
developing  as  reflective  practitioners,  and  so  on.  After  a review  of  the  principles  of  the  Interstate  New  Teacher  Assessment  and 
Support  Consortium  (INTASC),  the  committee  agreed  that  the  principles  of  INTASC  encompassed  and  addressed  all  major 
components  of  the  middle  childhood  program  and  could  be  used  to  facilitate  the  development  of  the  preservice  teachers.  Thus  the 
committee  established  portfolio  guidelines  that  focused  on  the  ten  principles  of  INTASC. 

Through  the  continuous  collaboration  of  the  MCC,  guidelines  for  portfolio  development  were  documented,  benchmarks  were 
established,  implementation  procedures  were  outlined,  and  an  assessment  instrument  and  procedures  were  designed.  The 
committee  reviewed  course  syllabi  for  all  MCE  undergraduate  education  courses  to  determine  which  INTASC  principles  were 
met  in  each  course.  The  principles  were  aligned  with  the  program’s  schedule  of  course  sequence  and  experiences  to  establish 
which  principles  the  preservice  students  would  be  able  to  address  at  established  intervals.  These  intervals  serve  as  benchmarks  to 
assess  the  students’  portfolios.  Portfolio  development  is  introduced  in  the  Teachers  and  Technology  course  in  the  form  of  the 
learning  environment  portfolio.  Subsequent  submissions  are  based  on  the  INTASC  principles,  and  occur  at  the  end  of  the  first 
year  of  professional  studies,  prior  to  student  teaching,  and  at  the  close  of  student  teaching. 

Another  aspect  of  our  PT3  mini-grant  study  was  to  determine  if  subsequent  MCE  student  portfolios  could  be  examined  to 
determine  preservice  students’  ability  to  meet  NETS-T  standards  as  they  continue  through  their  professional  preparation  program. 
A rubric  was  developed  for  the  24  indicators  of  the  professional  preparation  performance  profile  and  applied  to  Middle 
Childhood  Education  student  portfolios  submitted  prior  to  entrance  to  the  professional  sequence  of  coursework.  (Note:  the  rubric 
is  too  long  and  detailed  to  be  included  within  this  paper,  but  can  be  obtained  from  the  authors  upon  request.)  The  objective  of 
these  portfolios  was  for  students  to  demonstrate  six  of  ten  INTASC  principles.  We  were  hopeful  that  data  analysis  would  indicate 
that  students  continue  to  apply  the  technology  integration  skills  mastered  in  IT  3210  to  other  courses  in  their  program  of  study. 

However,  that  was  not  the  case.  Review  of  six  MCE  INTASC  portfolios  submitted  in  May  2001  showed  little  student 
documentation  of  technology  competencies.  It  was  surmised  that  as  students  were  not  specifically  directed  to  demonstrate 
technology  competencies,  they  simply  did  not  do  so.  A second  factor  that  might  have  contributed  to  the  lack  of  technology 
demonstrated  in  these  portfolios  was  timing.  Several  MCE  students  were  compiling  their  IT  3210  portfolios  at  the  same  time  as 
they  were  compiling  the  IT  3210  portfolios.  It  is  conceivable  that  technology  artifacts  went  in  one  portfolio,  and  not  the  other. 

Consultation  with  MCE  faculty  indicated  that  asking  MCE  students  to  specifically  include  technology  in  their  INTASC  might 
dilute  the  attention  they  paid  to  addressing  the  INTASC  principles.  Furthermore,  it  was  decided  that  review  of  all  MCE  courses 
should  be  conducted  to  determine  when  and  where  students  were  addressing  technology  integration  skills. 

MCE  Program  Review 

In  order  to  determine  what  NETS-T  performance  indicators  were  addressed  and  reinforced  in  each  of  the  courses  that  make 
up  the  MCE  program,  a matrix  was  developed  for  MCE  faculty  in  May  2001.  The  matrix,  included  in  appendix  I,  maps  the 
NETS-T  performance  indicators  for  the  general  preparation  profile,  the  professional  preparation  profile,  and  the  student 
teaching/intemship  profile  against  all  courses  in  the  MCE  program.  Faculty  were  asked  mark  what  performance  indicators  were 
addressed  in  their  courses  (regardless  of  indicator  level)  prior  to  the  October  2001  meeting.  To  date,  data  from  seven  courses  in 
four  content  areas  have  been  received.  The  data  from  the  remaining  nineteen  courses  are  expected  when  the  MCE  faculty  meets 
in  early  October.  Once  all  courses  have  been  reviewed,  decisions  will  be  made  to  ensure  students  meet  all  NETS-T  performance 
indicators  in  their  coursework  at  GSU.  In  addition,  once  coverage  of  all  NETS-T  standards  has  been  assured,  the  MCE  INTASC 
portfolio  will  be  accepted  in  electronic  format  only. 


Conclusions 

Accountability  directives  for  new  teacher  preparedness  are  not  likely  to  go  away  any  time  soon.  Indeed,  in  his  first  month  in 
office.  United  States  President  Bush  proposed  education  initiatives  to  increase  teacher  accountability  similar  to  those  in  place  in 
the  state  of  Georgia  be  implemented  nationwide.  Instructional  technology  preparation  will  likely  continue  to  be  a critical  issue  in 
teacher  education  for  many  years  to  come.  Instructional  technology  units  can  no  longer  teach  only  to  their  corporate  training 
design  and  development  roots.  For  colleges  of  education  to  successfully  prepare  teachers  for  the  2P*  century,  instructional 
technology  will  need  to  be  more  cohesively  included  in  teacher  preparation  programs.  It  is  imperative  that  more  cooperative 
partnerships  be  established  between  instructional  technology  units  and  initial  preparation  programs.  The  authors  encourage  IT 
units  to  initiate  and  nurture  these  partnerships,  making  possible  more  innovative  approaches  to  this  important  field  of  study. 
While  many  PT3  initiatives  are  on  too  large  of  a scale  to  be  adopted  without  a large  commitment  from  teacher  education 
programs,  the  strategies  presented  in  this  paper  may  be  attempted  at  any  institution  with  little  risk. 
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Abstract 

The  purpose  of  this  study  was  to  develop  a comprehensive  model  for  local  school  technology  professional  development  that 
facilitated  technology  integration  into  the  curriculum.  The  local  school  technology  committee  conducted  an  informal 
performance  analysis  to  determine  a plan  of  action.  As  a needs  assessment,  stakeholders  were  surveyed  to  determine  requisite 
technology  professional  development  for  the  local  school  staff.  From  this  data,  a long-range  professional  development  plan  was 
created,  including  goals  and  objectives  for  each  year,  an  in-depth  yearly  plan  with  detailed  implementation  procedures  for  each 
objective,  and  individual  teacher  plans.  The  goal  of  the  professional  development  plan  was  to  increase  staff  technology  skills  by 
5%  as  measured  by  the  Technology  Skills  Assessment.  A comparison  of  pre-  and  post-assessments  indicated  that  participants 
increased  technology  skills  by  29.59%.  During  the  pretest,  50%  of  the  participants  scored  in  the  beginner  skill  level.  A posttest 
comparison  revealed  that  none  of  the  participants  scored  in  the  beginner  level;  all  participants  scored  in  the  intermediate  or 
advanced  category. 

As  we  move  forward  in  the  twenty-first  century,  our  world  is  undergoing  a technological  revolution  much  like  the  industrial 
revolution  of  the  nineteenth  century.  Technology  is  permeating  and  transforming  every  part  of  our  society.  In  the  face  of  these 
rapid  changes  in  business  and  at  home,  technology  literacy  is  more  important  than  ever.  In  order  for  our  graduates  to  be  prepared 
to  meet  the  challenges  of  the  business  world  in  the  twenty-first  century,  they  must  be  proficient  at  accessing,  evaluating,  and 
communicating  information.  As  businesses  learn  to  compete  in  a global  environment,  students  must  also  become  globally  aware 
and  able  to  use  technology  resources  that  exist  beyond  the  walls  of  the  classroom.  Students  preparing  for  this  global  workplace 
must  be  able  to  use  technology  or  “face  a lifetime  of  menial  work”  (U.  S.  Department  of  Labor,  1991,  p.  15). 

The  National  Center  for  Educational  Statistics  (2000)  reported  that  the  average  number  of  computers  in  classrooms  in  the 
United  States  has  grown  significantly  in  the  last  twenty  years.  This  number  has  grown  from  one  computer  for  every  125  students 
in  1983  to  one  computer  for  every  six  students  in  1998.  In  addition,  Internet  availability  in  public  schools  has  increased  from  35 
to  95  percent  between  1 994  and  1998.  Unfortunately,  availability  has  not  ensured  use. 

Policymakers  and  the  public  are  beginning  to  question  the  effectiveness  of  these  substantial  investments  in  equipment  and 
infrastructure  and  demand  evidence  that  they  have  been  worthwhile  (Trotter,  1 998).  Some  suggest  that  if  all  the  computers  were 
taken  from  schools  tomorrow,  it  would  not  make  a difference.  In  contrast,  businesses  in  this  country  would  be  immobilized 
without  technology  (Peck  & Dorricott,  1994).  Why  is  it  that  “businesses  have  been  building  electronic  highways  while  education 
has  been  creating  an  electronic  dirt  road?”  (DTgnazio,  1993,  p.l  1). 

Substantial  investments  in  technology  hardware,  software,  and  infrastructure  will  not  impact  teaching  and  learning  until 
teachers  have  the  technology  skills  and  technology  integration  strategies  needed  for  today’s  modem  classrooms.  McKenzie 
(1999)  suggests,  “The  best  way  to  encourage  teachers  to  embrace  these  technologies  is  to  give  them  personal  learning 
experiences  which  win  them  over  to  the  worthwhile  classroom  activities  which  are  now  possible”  (p.6).  Local  schools  and 
districts  must  concentrate  on  professional  development  to  reap  a significant  return  on  these  investments  (Rettig-Seitam,  2000). 


Purpose  of  the  Study 

The  purpose  of  this  study  was  to  develop  and  implement  a comprehensive  model  for  local  school  technology  professional 
development  that  facilitated  technology  integration  into  the  curriculum.  Although  states  such  as  Georgia  (State  Data  and 
Research  Center  of  the  Georgia  Institute  of  Technology,  2001),  Florida  (Swain,  2000),  Michigan  (Hoffman  & Thompson,  2000) 
and  North  Carolina  (Walbert,  2000)  have  begun  to  fill  the  gap  in  providing  much  needed  training  for  teachers,  effective  local 
school  models  are  still  needed.  Many  teachers  have  received  little  or  no  technology  integration  training,  because  the  diffusion 
process  is  very  slow.  Well-designed  local  school  plans  can  augment  state  initiatives  and  provide  the  professional  development 
teachers  need  to  successfully  integrate  technology  into  instmction. 

Staff  development  models  that  focus  on  school  improvement  initiatives  are  often  initiated  to  address  a problem.  The 
successful  completion  of  the  initiative  may  call  for  teachers  to  gain  specific  knowledge  or  skills.  This  information  may  be 
obtained  through  reading,  discussion,  observation,  training,  or  trial  and  error.  In  other  cases,  experiential  learning  may  occur  in 
the  development  and  improvement  process.  The  model  developed  and  implemented  in  this  study  is  a combination  of  the 
knowledge  that  results  from  teacher  involvement  in  a development  and  improvement  process  (Sparks  & Loucks-Horsley,  1989). 

The  development  and  implementation  of  the  model  was  a multi-step  process.  The  first  step  was  to  identify  the  theoretical 
underpinnings  that  formed  the  background  for  the  study  and  to  detail  the  assumptions  on  which  the  model  was  based.  The 
development  of  the  model  and  assessment  plan  included  an  informal  performance  analysis,  development  of  needs  assessment 
instruments  for  all  stakeholders,  creation  of  the  implementation  plan,  and  development  of  the  pre-  and  post -assessments. 


381 


390 


Assumptions  of  the  model 

The  first  assumption  on  which  this  model  is  based  is  that  adults  learn  most  effectively  when  they  have  a need  to  know  or 
solve  a real-life  problem  (Knowles,  1980).  Knowles’  theory  of  adult  learning  has  become  well  known  in  recent  years  and 
proposes  the  following  assumptions  of  adult  learning: 

1 . Adults  have  a need  to  be  self-directing. 

2.  Adults  bring  a wealth  of  experience  that  should  be  used  in  a learning  situation. 

3.  Adults’  readiness  to  learn  is  based  on  a need  to  solve  a problem. 

4.  Adults  want  to  make  immediate  application  of  knowledge. 

Teachers  serving  on  a school  improvement  team  may  need  to  review  research  on  effective  teaching,  learn  new  group  and 
interpersonal  skills,  or  acquire  content  knowledge  for  new  curriculum  initiatives.  In  each  of  the  d)ove  examples,  teachers’ 
learning  is  motivated  by  the  need  to  solve  a problem. 

Secondly,  this  model  assumes  that  those  working  closest  to  the  problem  best  understand  what  is  needed  to  improve  their 
performance.  Teachers  rely  on  their  teaching  experiences  to  guide  them  as  they  identify  problems  and  develop  solutions.  Given 
the  opportunity,  the  unique  perspective  of  teachers  can  add  to  the  school  improvement  process  (Sparks  & Loucks-Horsley,  1989). 

Finally,  this  model  assumes  that  teachers  obtain  important  knowledge  through  their  participation  in  the  school  improvement 
process.  Teachers’  involvement  in  the  process  may  foster  alteration  in  attitudes  or  the  development  of  problem-solving  skills  as 
individuals  or  groups  work  toward  the  solution  of  a problem.  For  example,  teachers  may  begin  to  appreciate  individual 
differences,  improve  group  leadership  skills,  or  just  become  more  aware  of  the  perspective  of  other  staff  members.  Although  this 
type  of  learning  may  be  difficult  to  predict,  it  is  significant  to  the  teachers  (Sparks  & Loucks-Horsley,  1989) 

Description  of  the  Population 

The  population  involved  in  this  study  was  the  staff  of  an  elementary  school  in  a metropolitan  area  of  a large  city  in  the 
southeastern  United  States.  The  staff  had  69  certified  teachers,  17  teacher  assistants,  2 school  counselors,  2 speech  and  language 
pathologists,  1 technology  support  technician,  5 clerical  staff,  1 clinic  worker,  8 cafeteria  staff,  4 custodians  and  3 administrators. 
Among  all  certified  teachers,  32  held  a Masters  degree  or  higher,  6 had  gifted  certification,  3 had  ESOL  certification  and  7 had 
special  education  certification.  New  teachers  to  the  school  were  provided  with  a mentor  who  was  available  for  planning  and 
helping  with  basic  information  and  procedures.  The  professional  learning  community  worked  together  in  a collaborative  manner 
to  promote  teaching  and  learning  and  achieve  annual  school  goals. 

The  school  was  organized  using  a leadership  model  known  as  Shared  Governance.  Each  grade  level,  including  teacher 
assistants  and  clerical  personnel,  had  a chairperson  who  met  with  the  administrators  every  month  to  coordinate  and  discuss 
curriculum  matters,  school  organization,  areas  of  concern  and  monitor  the  Local  School  Plan  for  Improvement  (LSPI).  Dialogue 
flowed  to  and  from  the  individual  grade  level  groups  through  the  communication  vehicle  of  the  Shared  Leadership  Team  (SLT). 
SLT  made  decisions  after  careful  consideration  of  colleague  feedback  and  available  data. 

The  school  had  been  honored  as  a School  of  Excellence  and  was  committed  to  the  continuous  improvement  process.  The 
Local  School  Plan  for  Improvement  (LSPI)  was  written  each  year  using  pertinent  data  and  input  from  the  staff  LSPI  goals 
identified  specific  improvement  efforts.  Each  staff  member  s^  individual  goals  based  on  the  local  school  LSPI  goals.  When  staff 
members  successfully  completed  their  individual  goals  the  payoff  was  seen  in  improved  school-wide  LSPI  results.  Certainly,  one 
of  the  major  benefits  of  the  LSPI  was  the  opportunity  for  staff  development  in  the  identified  areas  of  improvement.  Teachers  had 
the  opportunity  to  improve  teaching  strategies  and  learn  cutting  edge  research  at  the  local  school. 

Development  of  the  Professional  Development  Model  And  Assessment  Plan 

With  the  1998  completion  of  the  school  network,  every  classroom  received  a multimedia  computer  with  Internet,  email,  and  a 
laser  printer,  and  the  school  had  three  computer  labs  with  Internet  connections.  As  technology  resources  increased,  teachers 
requested  more  on-site  technology  professional  development.  To  meet  the  growing  demands,  the  technology  coordinator  worked 
the  technology  committee  to  develop  a comprehensive  plan  for  technology  professional  development.  The  development  of  this 
plan  was  based  on  the  principles  of  instructional  system  design  described  by  Reigeluth,  Benathy,  and  Olsen  (1993)  and  Mager 
and  Pipe  (1997),  which  included  a performance  analysis,  a needs  assessment,  the  development  and  implementation  of  a 
professional  development  plan  and  the  assessment  of  the  impact  of  the  professional  development  plan. 

Performance  Analysis 

Although  the  school  had  received  a large  infusion  of  technology  in  the  last  several  years,  classroom  computers  were  not 
being  fully  utilized  and  computer  labs  were  mainly  being  used  for  drill  and  practice  activities.  The  school  leadership  team  and 
technology  committee  recognized  this  problem  and  began  to  investigate  solutions  that  would  foster  technology  integration  into 
the  curriculum.  This  was  important  because  the  district  had  invested  about  a million  dollars  in  technology  hardware,  software, 
and  infrastructure  in  each  elementary  school  in  the  district.  With  the  district’s  focus  on  accountability  for  results  and  continuous 
improvement,  the  local  schools  were  being  asked  to  measure  the  impact  of  technology  on  teaching  and  learning.  The  technology 
committee  and  school  leadership  team  realized  that  the  majority  of  the  teachers  did  not  feel  they  were  adequately  prepared  to  use 
the  new  technology  tools  in  instruction. 
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Needs  Assessment 

With  the  completion  of  the  preliminary  performance  analysis,  the  technology  committee  developed  several  surveys  to  ensure 
that  the  interests,  values,  and  perspectives  of  all  stakeholders  would  be  represented  in  any  instructional  design  or  organizational 
change.  The  stakeholders-  -all  the  groups  who  had  a stake  in  the  instructional  system  being  designed-  - in  this  project  included 
teachers,  assistant  teachers,  students,  and  parents  (Reigeluth,  1996).  The  survey  of  veteran  teachers  included  questions  on  staff 
development  preferences  such  as  class  organization,  session  length,  optimal  time  of  day  for  instruction,  teacher  interests  and 
needs,  and  obstacles  to  success.  The  new  teacher  survey  was  designed  to  identify  baseline  skills  for  new  staff  members.  The 
purpose  was  to  identify  any  significant  training  gaps  between  existing  staff  and  new  staff.  The  survey  of  the  assistant  teachers 
sought  to  identify  essentials  skills  that  support  staff  needed  to  effectively  aid  teachers  with  instruction.  Student  surveys  were 
designed  to  identify  technology  skills  that  fourth  and  fifth  grade  students  could  complete  independently.  The  survey  included 
skills  identified  in  the  instructional  technology  competencies  developed  by  the  local  school  district.  The  parent  survey  was 
designed  to  gain  the  parents’  perspectives  and  knowledge  of  local  school  technology  initiatives. 

Intervention  Selection  and  Development 

Prior  to  the  development  of  the  implementation  plan,  a careful  analysis  of  data  from  all  the  stakeholders’  surveys  was 
completed  to  determine  needs  and  interests  of  the  local  school  staff.  The  first  step  in  the  process  was  the  development  of  a long- 
range  plan.  This  overview  included  the  goal  for  the  professional  development  plan  and  objectives  for  each  year.  Second,  an  in- 
depth  yearly  plan  was  developed  with  detailed  implementation  procedures  for  each  objective.  Each  objective  listed  baseline  data, 
indicators  of  success,  measurement  tools,  an  implementation  plan  of  activities,  responsible  parties,  and  completion  dates.  Next, 
an  individual  teacher  professional  development  plan  was  created.  This  plan  was  directly  correlated  to  the  yearly  implementation 
plan.  The  individual  teacher  plan  also  included  a professional  development  log  and  a rubric  for  administrators  to  use  in  evaluating 
individual  teacher  professional  development  plans.  Based  on  data  from  all  the  stakeholder  surveys,  a menu  of  technology 
professional  development  classes  was  developed  to  support  the  implementation  plan.  Due  to  the  wide  range  in  teacher 
knowledge  and  skills,  provisions  were  also  made  for  an  independent  study  option. 

Implementation  of  Professional  Development  Plan 

In  the  early  fall,  the  technology  committee  completed  surveys  of  all  stakeholders.  The  data  was  compiled  and  analyzed  for 
strengths  and  weaknesses.  The  results  were  used  to  plan  technology  professional  development  for  the  school  year.  A menu  of  29 
classes  was  created  by  the  end  of  September.  Based  on  the  yearly  professional  development  plan,  each  teacher  developed  a 
personal  plan  for  improvement.  In  January,  the  administrator  on  the  technology  committee  and  the  technology  specialist  met 
with  each  grade  level  to  informally  discuss  the  progress  made  on  the  school  goals.  In  addition,  each  certified  staff  member  met 
with  the  t echnology  administrator  in  January  and  February  for  an  interim  review  of  progress  on  individual  teacher  goals. 

Implementation  Assessment 

A pretest -posttest  design  was  used  to  assess  the  implementation  plan.  Pretests  were  administered  in  August  1999  and 
posttests  were  completed  in  April  2000.  Teachers  were  assigned  code  numbers  to  maintain  the  confidentiality  of  the  pretest  and 
posttest  data.  Data  analysis  involved  the  comparison  of  pretest  and  posttest  data  for  each  objective  of  the  yearly  implementation 
plan.  Analysis  of  the  data  was  completed  for  each  grade  level  and  for  the  entire  staff.  The  analysis  gave  a percentile  score  and  a 
beginner,  intermediate,  or  advanced  skill  level  for  each  teacher  and  an  average  for  the  entire  staff 

Summary 

The  development  of  this  plan  took  several  months  and  implementation  spanned  an  entire  school  year.  Throughout  the 
process,  the  staff  at  the  local  school  was  given  opportunities  to  evaluate  the  progress  toward  the  professional  development  goals. 
This  type  of  feedback  spiral  was  designed  to  foster  continuous  improvement  through  individual  growth,  as  well  as  growth  of  the 
local  school  organization  (Costa  & Kallick,  1995). 

Methods 

In  order  to  develop  the  plan  and  to  assess  its  effectiveness,  data  was  collected  at  three  stages,  performance  analysis,  needs 
assessment,  and  assessment  of  the  professional  development  plan.  Data  collected  at  each  of  the  first  two  stages  informed  the 
development  of  the  next  stage. 

Performance  Analysis 

The  technology  committee  conducted  an  hformal  performance  analysis.  The  committee  was  composed  of  an  assistant 
principal,  the  technology  specialist,  the  media  specialist,  and  one  representative  from  each  grade  level.  The  data  examined  during 
this  process  included  the  amount  of  money  spent  on  technology  resources  in  the  past  few  years,  the  usage  of  classroom 
computers  and  labs,  and  teacher  preparedness  to  use  technology  tools  to  enhance  instruction. 

Needs  Assessment 

Prior  to  the  development  and  implementation  of  the  professional  development  model,  survey  data  was  collected  from  all 
stakeholders  to  ensure  that  the  interests  and  needs  of  each  group  would  be  addressed  by  the  plan.  Veteran  teachers,  new  teachers. 
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and  teacher  assistants  were  surveyed  about  technology  professional  development  needs  and  interests.  Sixty- five  veteran  teachers, 
twelve  new  teachers,  and  twelve  teacher  assistants  at  the  local  school  participated  in  the  needs  assessment.  In  addition,  parents 
and  students  were  surveyed.  All  surveys  were  in  paper  and  pencil  format  and  were  distributed  and  collected  by  members  of  the 
technology  committee. 

Assessment  of  the  Implementation  Plan 

A self-assessment  instrument  was  developed  to  gage  teacher  progress  in  acquiring  technology  integration  skills.  The  paper 
and  pencil  assessment  contained  twenty  questions  to  monitor  teacher  progress.  For  each  question,  teachers  were  to  mark  their 
proficiency  level  from  one  to  five,  with  level  one  indicating  a minimum  skill  level  and  level  five  indicating  a maximum  skill 
level. 


Results 

Veteran  teachers,  new  teachers,  and  teacher  assistants  were  surveyed  about  technology  professional  development  needs  and 
interests.  Sixty-five  surveys  (100%)  were  received  from  veteran  teachers,  12  surveys  (100%)  were  received  from  new  teachers, 
and  12  surveys  (100%)  were  received  from  teacher  assistants  at  the  local  school.  In  addition,  303  parents  and  students  were 
surveyed.  The  fourth  and  fifth  grade  students  returned  266  surveys  (88%).  Parents  returned  99  surveys  (33%).  Due  to  the  low 
return  rate,  parent  data  was  not  used  in  the  project  development. 

Teacher  Surveys 

Veteran  teachers  were  asked  to  respond  to  six  questions  concerning  technology  professional  development  needs,  interests, 
and  preferences.  Table  1 shows  the  survey  results.  Percentages  were  rounded  to  the  nearest  whole  number. 

An  analysis  of  survey  data  indicated  the  following  staff  development  preferences:  small  group  classes  (72%)  and  class 
sessions  of  30  minutes  to  2 hours  (96%).  Major  areas  of  interest  for  technology  use  included  technology  to  assist  in  the 
organization  and  access  of  student  information  (34%),  technology  used  as  an  integral  part  of  lesson  plans  (35%),  and  technology 
to  improve  students’  writing  skills  (31%).  Teachers  indicated  that  the  biggest  obstacle  to  technology  integration  was  lack  of  time 
to  plan  technology  -connected  units  (75%). 


Table  1.  Teacher  Professional  Development  Preferences 
N=65 

Teacher  Professional  Development  Preferences 
Technology  staff  development  can  best  meet  my  needs  when  it  is 
T aught  in  small  grade  level  groups 
Taught  one  on  one 
Taught  in  a large  group  lab  setting 

Technology  staff  development  can  best  meet  my  needs  when  taught  in 
Mini  class  sessions  (30  minutes) 

In-depth  sessions  (1-2  hours) 

Very  in  -depth  sessions  (1/2  day  or  1 day) 

Technology  staff  development  can  best  meet  my  needs  when  offered 
During  the  contract  day  - before  school 
During  the  contract  day  - after  school 
During  the  contract  day  - during  a planning  period 
As  an  SDU  class  before  school  - not  on  contract  time 
As  an  SDU  class  after  school  - not  on  contract  time 
Technology  staff  development  can  best  meet  my  needs  when  classes  are 
Organized  by  beginning,  intermediate  or  advanced  skill  levels 
Organized  by  topics  of  interest  with  mixed  technology  skill  levels 
Are  organized  according  to  grade  level  needs  and  interests 
1 am  most  interested  in  learning  to  use  technology 

That  can  average  my  grades  and  print  reports  for  parents 
That  will  help  me  to  organize  and  access  student  information 
To  improve  communication  with  parents  and  students 
As  an  integral  part  of  my  regular  lesson  plans 


Frequency  Percentage 


47 

72 

6 

9 

12 

18 

30 

46 

31 

48 

4 

6 

29 

45 

6 

9 

6 

9 

13 

20 

19 

29 

31 

48 

22 

34 

14 

22 

14 

22 

22 

34 

14 

22 

23 

35 
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To  assist  my  students  in  learning  the  research  process 

15 

23 

To  assist  my  students  in  improving  keyboarding  skills 

8 

12 

To  assist  my  students  in  improving  writing  skills 

20 

31 

The  biggest  obstacle  I face  in  integrating  technology  into  the  curriculum  is  * 
Lack  of  time  to  plan  technology- connected  lesson/units 

49 

75 

Insufficient  access  to  computer  hardware/equipment 

8 

12 

Insufficient  access  to  instructional  software 

10 

15 

Insufficient  training  in  the  use  of  the  technology  tools 

12 

18 

Not  comfortable  using  technology  with  students 

1 

2 

New  Teacher  Survey 

The  survey  of  new  teachers  included  first  year  teachers  and  teachers  who  were  new  to  the  local  school  involved  in  this 
project.  The  survey  was  designed  to  identify  gaps  in  teacher  technology  skills.  Table  2 indicates  new  teacher  survey  results. 
Percentages  were  rounded  to  the  nearest  whole  number. 

Analysis  of  the  results  indicated  staff  development  needs  in  several  areas.  In  basic  network  navigation,  only  25%  of  teacher 
could  log  on  or  off  the  network  and  17%  could  save  file  to  the  school  server.  Second,  17%  of  the  teachers  reported  that  they 
could  detach  e-mail  attachments  or  create  e-mail  groups.  Other  needs  identified  included  using  satellite  or  distance  learning 
resources,  using  Accelerated  Reader  to  support  reading,  creating  a basic  web  page,  and  basic  computer  troubleshooting  skills. 


Table  2,  New  Teacher  Technology  Professional  Development  Needs 
N=12 


New  Teacher  Technology  Skills 

Frequency 

Percentage 

Knows  correct  way  to  start  up  and  shut  down  a computer  with  Windows  95 

12 

100 

Knows  how  to  log  on  and  log  off  of  Novell  Netware 

3 

25 

Knows  Windows  95  basics  such  as  minimize,  maximize,  open,  close,  & quit 

12 

100 

Knows  how  to  use  Windows  95  Explorer  to  organize  personal  computer  files 

2 

17 

Knows  how  to  save  files  to  the  courseware  server 

2 

17 

Knows  how  to  back  up  network  files  to  floppy  disk 

4 

33 

Knows  word  processing  basics  such  as  changing  fonts,  size,  and  style 

10 

83 

Can  save  and  print  word  processing  files 

11 

92 

Can  copy  and  paste  graphics  and  text  in  word  processing 

5 

42 

Can  create  tables  in  word  processing 

1 

8 

Can  create  newsletters  in  word  processing 

6 

50 

Can  reply  and  forward  email 

10 

83 

Can  send  email  attachments 

4 

33 

Can  detach  email  attachments  and  save  to  the  courseware  server 

2 

17 

Can  create  email  groups 

2 

17 

Can  type  in  an  internet  web  address  and  bookmark  a web  site 

7 

58 

Knows  how  to  use  an  internet  search  engine  such  a Yahoo 

8 

67 

Knows  how  to  make  a basic  web  page 

2 

17 

Knows  how  to  plan  and  manage  taking  a class  to  computer  lab 

8 

67 

Can  create  and  teach  a technology -connected  lesson  plan 

6 

50 

Knows  how  to  us  satellite  resources  such  as  the  Peach  Star  Pipeline  to  support  instruction 

2 

17 

Knows  how  to  use  the  Accelerated  Reader  program  to  support  reading  instruction 

2 

17 

Knows  how  to  use  computer  tutorials  or  the  help  menu  in  computer  programs 

2 

17 

Can  troubleshoot  a frozen  computer  or  a jammed  printer 

2 

17 
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Teacher  Assistant  Survey  Results 

The  teacher  assistant  survey  was  designed  to  identify  gaps  in  skills  needed  to  support  instruction.  Table  3 shows  data  from 
teacher  assistant  surveys.  Percentages  were  rounded  to  the  nearest  whole  number.  An  analysis  of  survey  data  indicated  greatest 
needs  for  additional  training  in  the  areas  of  e-mail  attachments,  using  the  computer  for  student  drill  and  practice,  and  basic 
computer  troubleshooting. 


Table  3.  Teacher  Assistant  Survey  Results 
N=12 


Technology  Skills 

Can  do  task 
independently 

Percentage 

Can  do  task  with 
assistance 

Percentage 

Cannot  do  this 
task 

Percentage 

I can  start  up,  log  on,  log  off  and  shut  down  an  IBM  computer. 

100 

0 

0 

I know  how  to  search  for  a book  or  video  on  the  media  center  online  card 

100 

0 

0 

catalog. 

I can  save  to  the  server  and  print  in  word  processing 

43 

17 

25 

I can  access  my  e-mail  account 

75 

0 

25 

1 can  reply  and  forward  e-mail 

58 

17 

25 

I can  troubleshoot  a frozen  computer. 

17 

17 

50 

1 can  use  the  classroom  computer  for  student  drill  & practice,  and/or 

8 

8 

50 

educational  games. 

I know  Windows  95  basics  such  as  minimize,  maximize,  close  and  quit 

100 

0 

0 

I know  word  processing  basics  such  as  changing  fonts,  size  and  style. 

42 

25 

25 

I can  send  e-mail  attachments. 

8 

25 

50 

Student  Technology  Survey  Results 

Student  surveys  were  designed  to  identify  technology  skills  that  fourth  and  fifth  grade  students  could  complete 
independently.  The  survey  included  skills  identified  in  the  elementary  instructional  technology  competencies  developed  by  the 
local  school  district.  Table  4 shows  data  from  the  student  surveys. 


Table  4.  Student  Technology  Survey  Results 
N = 266 


Technology  Skills 

Can  do  task 
independently 
Frequency 

Can  do  task  with 
assistance 
Frequency 

Cannot  do 
this  task 

Frequency 

I can  use  the  correct  finder  positions  for  keyboarding. 

137 

48 

81 

I can  edit  my  own  writing  using  spell  checker. 

153 

53 

' 

26 

I can  change  fonts  and  sizes  in  word  processing. 

189 

27 

11 

I can  save  my  work. 

191 

23 

9 

I can  print  my  work. 

182 

28 

10 

I know  how  to  go  to  or  look  up  web  sites  on  the  internet. 

129 

60 

35 

I know  how  to  search  for  information  on  the  internet  or  CD  ROM  to 

110 

77 

40 

complete  classroom  assignments. 

I know  how  to  start,  minimize,  close,  quit  and  shut  down  the 

158 

49 

15 

computer. 

I know  how  to  properly  load  and  eject  a CD  ROM  from  a computer. 

190 

26 

9 

I know  how  to  make  a Kid  Pix  slide  show. 

94 

88 

42 

Assessment  of  Implementation  Plan 

A pretest-posttest  design  was  used  to  assess  the  effect  the  yearly  implementation  plan.  Teachers  were  assigned  code  numbers 
to  maintain  the  confidentiality  of  the  pretest  and  posttest  data.  Analysis  of  the  data  was  completed  for  each  grade  level  and  for 
the  entire  staff.  The  analysis  provided  a percentage,  a skill  level  for  each  teacher,  and  a mean  score  for  the  entire  staff. 
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A pretest  and  posttest  was  administered  to  all  certified  teachers.  Scores  were  reported  in  percentages,  A score  between  0 and 
39%  indicated  a beginner  skill  level,  a score  between  40  and  79%  indicated  an  intermediate  skill  level,  and  a score  between  80 
and  100%  indicated  an  advanced  skill  level.  Table  5 shows  the  summary  data  for  the  implementation  plan. 

Table  5.  Pretest  and  posttest  summary  data 


N 

M 

SD 

SE 

Pre 

61 

43.1 

14,7 

1.9 

Post 

61 

IIJ 

13,9 

1.8 

Tables  6 and  7 compare  pretest  and  posttest  skill  levels  by  grade  level.  On  the  pretest,  only  two  grade  levels  h 

advanced  scores.  In  comparison. 

the  posttest  scores 

revealed  that  all  grade  levels  had  gained 

in  the  advanced  skill  level 

for  fourth  grade.  In  addition,  all  pretest  beginners  had  progressed  to  intermediate  or  advanced  skill  levels. 

Table  6,  Pretest  Staff  Skill  Levels  By  Grade  Level 

Grade 

Beginner 

Intermediate 

Advanced 

Kindergarten 

6 

1 

0 

1 “ Grade 

4 

4 

0 

2"^  Grade 

3 

5 

0 

3"'  Grade 

3 

4 

1 

4"'  Grade 

4 

3 

0 

S""  Grade 

5 

2 

0 

Teacher  Specialists 

2 

5 

0 

Special  Education 

5 

5 

1 

Totals 

32 

29 

2 

Table  7.  Posttest  Staff  Skill  Levels  by  Grade  Level 

Grade 

Beginner 

Intermediate 

Advanced 

Kindergarten 

0 

6 

1 

1“  Grade 

0 

6 

2 

2"“  Grade 

0 

3 

5 

3"“  Grade 

0 

7 

1 

4"'  Grade 

0 

7 

0 

S"‘  Grade 

0 

3 

4 

Teacher  Specialists 

0 

2 

5 

Special  Education 

0 

5 

6 

Totals 

0 

39 

24 

Discussion 

This  project  attempted  to  develop  and  implement  a model  of  local  school  professional  development  that  facilitated 
technology  integration  into  the  curriculum.  Prior  to  the  development  of  the  model,  surveys  were  administered  to  the 
stakeholders.  The  data  was  compiled  and  analyzed  to  identify  the  needs  and  concerns  that  should  be  addressed  by  any  new 
professional  development  initiative. 

Through  the  development  and  implementation  of  the  model,  the  participants  at  the  local  school  recognized  that  all  parties 
were  change  agents  in  the  technology  integration  process.  Successful  integration  of  technology  in  the  curriculum  required  the 
collaboration,  shared  responsibilities,  cooperation,  and  investments  of  time  and  effort  from  all  involved  stakeholders. 

To  facilitate  the  integration  of  technology  into  the  curriculum,  all  staff  members  at  the  local  school  were  involved 
comprehensive  on-site  technology  staff  development.  Professional  development  was  designed  to  support  the  local  school 
improvement  plan  and  district  continuous  improvement  initiatives.  Staff  members  responded  positively  to  the  convenience  of  on- 
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site  training,  but  sometimes  training  was  hampered  by  local  school  distractions.  The  inclusion  of  an  independent  study  option 
provided  increased  flexibility  for  staff  members  to  pursue  some  of  their  own  interests  for  professional  growth. 


Conclusions 

The  goal  of  the  professional  development  plan  was  to  increase  staff  technology  skills  by  5%  as  measured  by  the  Technology 
Skills  Assessment.  The  mean  difference  between  the  pretest  and  posttest  scores  was  29.59,  which  was  statistically  significant, 
to5=20.506.  In  addition,  all  participants  made  significant  advances  in  skill  level  (see  Figure  1).  During  the  pretest,  50%  of  the 
participants  scored  in  the  beginner  skill  level.  A posttest  comparison  revealed  that  none  of  the  participants  scored  in  the  beginner 
level.  All  participants  scored  in  the  intermediate  or  advanced  category. 

Figure  1.  A Comparison  of  Pretest  and  Posttest  Skill  Levels 


A Comparsion  of  Pretest  and  Posttest  Skill  Levels 
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The  main  beneficiaries  of  the  extensive  focus  of  technology  training  were  the  students.  As  a result  of  improved  teacher 
technology  skills,  students  had  the  opportunity  to  use  technology  create  multimedia  slide  shows  and  web  pages,  produce  live 
action  videos,  and  use  internet  and  CD  ROM  resources  for  research.  In  addition,  several  students  won  county  and  state  media 
festival  awards  for  multimedia  and  video  projects  that  were  related  to  the  curriculum. 


Recominendations 

After  examining  the  data  from  the  professional  development  initiative  and  reviewing  research  on  effective  professional 
development,  the  following  improvements  to  the  model  are  recommended  for  future  years  of  implementation: 

1.  Offer  an  on-site  menu  that  includes  more  choices  and  less  mandated  sessions  for  staff  members. 

2.  Offer  options  for  large  group  training,  small  group  classes,  and  one-on-one  mentoring  to  better  meet  the  various 
learning  styles  of  the  staff. 

3.  Improve  and  increase  follow-up  training  to  ensure  successful  implementation  of  the  professional  development 
initiative. 
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VIRTUAL  QUESTS  AS  LEARNING 
ENVIRONMENTS  FOR  K-12  STUDENTS 


Linda  Spudic 
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Abstract 

Virtual  quests,  such  as  the  MayaQuest  expedition  produced  by  Classroom  Connect,  are  excellent  examples  of  classroom 
experiences  that  provide  students  with  authentic  opportunities  to  solve  real  life  problems.  This  applied  qualitative  study  looked  at 
the  value  of  virtual  quests  as  classroom  learning  environments,  at  the  instructional  characteristics  of  teachers  who  chose  to 
participate,  reviewed  related  literature,  and  discussed,  from  the  teacher's  perspective,  their  application  in  a classroom  setting. 

Introduction 

Use  of  the  Internet  in  today’s  classrooms  can  offer  opportunities  that  enrich  and  expand  existing  curriculum,  support 
educational  standards,  and  rejuvenate  the  teaching  and  learning  process.  Access  to  original  documents,  photos,  video,  sound, 
content  experts,  and  live  data,  expands  the  resources  available  to  the  learner  in  a way  that  can  make  learning  more  engaging, 
interactive  and  authentic. 

Perhaps  some  of  the  most  engaging,  comprehensive  and  unique,  web-based  activities  are  virtual  quests  that  take  student 
participants  along  on  real  expeditions,  following  a team  in  the  field  as  they  explore  new  territory  or  do  research  on  authentic 
scientific  problems.  This  study  looked  closely  at  these  comprehensive  virtual  quests,  defined  their  common  characteristics, 
reviewed  related  literature  on  their  efficacy  in  the  K-12  setting,  and  discussed,  from  the  teacher’s  perspective,  their  application  in 
a classroom  setting. 

Virtual  Quests 

Although  the  term  virtual  field  trip  is  often  used  in  reference  to  these  large  quests,  it  can  be  misleading,  as  the  term  “virtual 
field  trip”  also  refers  to  any  number  of  non-interactive  sites  that  can  be  found  on-line.  In  contrast,  virtual  quests  have  the 
following  unique  characteristics: 

• Problem-based:  Focus  is  on  an  authentic  mystery  to  be  solved, 

• Curricular  focus:  National  standards  support  the  content, 

• Multidisciplinary:  Resources  and  activities  reach  across  the  disciplines. 

• Real  time  interactivity:  A team  in  the  field  interacts  with  participants  via  e-mail,  live  chats,  and  video, 

• Participants  as  stake-holders:  Students  are  asked  to  make  decisions  about  the  quest  that  the  field  team  acts  on,  i.e, 
should  the  team  give  money  to  beggars,  travel  through  the  jungle  or  into  the  cave? 

• Website  as  communication  central : The  site  contains  activities  and  resources  for  teachers  and  students.  It  is  highly 
visual  and  interactive,  and  is  the  center  for  daily  updates  and  shared  knowledge  creation. 

• Scheduled:  The  real-time  component  usually  last  six  to  eight  weeks,  although  some  last  as  long  as  a year. 

Classroom  Connect  sponsors  two  expeditions  a year  that  follow  a team  of  explorers  and  scientists  on  a quest  to  solve  a great 

mystery.  The  live  experience  of  the  Classroom  Connect  quests  last  for  four  weeks,  although  the  website  becomes  live  one  week 
before  the  field  team  starts  posting.  The  broad  range  of  resources  available  on  the  site  makes  it  possible  for  teachers  to  choose 
areas  of  focus  most  pertinent  to  their  curricular  needs. 

Comprehensive  virtual  quests  are  designed  to  engage  students  in  learning  that  includes  authentic  problem  solving,  real-time 
interaction  with  experts  on  the  field,  and  an  opportunity  to  share  their  research  data,  writing  and  artwork  with  an  international 
team  of  peers.  Over  the  past  years  I have  offered  teachers  the  opportunity  to  participate  in  virtual  quests  with  their  students,  but 
had  received  a limited  response.  Even  those  who  did  commit  did  not  take  full  advantage  of  the  quest  activities  by  following  the 
quest  consistently  and  engaging  students. 

The  purpose  of  this  study  was  to  review  associated  literature  that  might  support,  or  not,  the  validity  of  virtual  quests  in 
classroom,  to  identify  the  aspects  of  student  outcome  that  would  justify  their  use  and  to  better  understand  the  issues  of  quest 
implementation  in  the  classroom,  so  that  I might  be  able  support  teacher  adoption  in  the  future. 

Literature  Review 

Research  on  the  effectiveness  of  virtual  quests  is  lacking.  However,  it  is  possible  to  examine  research  on  the  Internet  in  the 
classroom,  writing,  problem  based  learning,  situated  cognition,  and  anchored  instruction. 

Michael  Ritter,  a professor  of  geography,  did  a descriptive  case  study  on  virtual  field  trips  with  his  students  who  visited  key 
locations  on-line  by  following  a path  of  hyperlinked  pages,  gathered  and  analyzed  data  from  photos  and  text,  and  kept  field 
journals  (1998).  Ritter  stated  that  the  value  of  virtual  field  trips  for  learning  geography  is  that  it  brings  remote,  dangerous  or 
expensive  locations  into  the  classroom. 

Lee  and  Songer  looked  at  the  effect  of  on-line  collaboration  on  students’  discourse  about  weather  science,  through  the  One 
Sky  project  (1999).  They  found  that  this  collaborative  on-line  experience  connected  student  personal  experience  with  scientific 
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learning.  It  made  science  learning  more  authentic.  Students  became  local  experts  and  knowledge  producers  and  developed  a 
higher  level  of  language  applied  to  science. 

Jean  Lave,  the  key  proponent  of  situated  cognition,  recognizes  that  learning  occurs  as  a function  of  activity  and  that  it  is 
situated  in  the  context  of  the  learning  environment  (1988).  The  model  of  situated  cognition  involves  a multidisciplinary, 
collaborative  approach  to  complex  problem  solving  with  embedded  data.  Brown,  Collins  and  Duguid  point  out  that  traditional 
learning  is  often  disconnected  from  the  context  of  content  (1989).  For  example,  students  learn  the  vocabulary  of  science  by  using 
the  dictionary,  writing  sentences  or  reading  textbooks.  This  approach  is  devoid  of  any  structure  or  clues  that  might  help  the 
student  make  connections  to  real  meaning.  In  a situated  learning  environment,  students  might  be  asked  to  work  like  scientists, 
experimenting,  observing,  hypothesizing  and  recording.  Situated  cognition  has  learning  occur  within  a community  or  culture  of 
practitioners.  Brown,  Collins  and  Duguid  note  that  activity,  concept,  and  the  culture  of  the  learning  community  are 
interdependent  (1989).  Students  who  work  and  study  like  scientists  are  more  likely  to  have  a structure  on  which  to  build  knew 
scientific  knowledge. 

Anchored  instruction,  a concept  developed  by  the  John  Bransford  at  the  Cognition  and  Technology  Group  at  Vanderbilt 
(CTGV)y  situates  authentic  problems  into  what  are  called  macrocontexts,  or  story -like  situations  (1993).  The  problems  are 
anchored  a^ound  these  realistic  situations  that  may  involve  an  adventure.  The  students  can  solve  the  challenge  by  using 
information  embedded  in  the  story.  The  story,  or  anchor,  engages  the  student  in  realistic  and  relevant  problem  solving.  Research 
done  by  the  CTGV  found  that  students: 

• performed  as  well  as  or  better  on  standardized  tests  than  the  control  group. 

• demonstrated  superior  performance  on  multi-step  word  problems. 

• showed  less  anxiety  about  mathematics. 

• were  more  likely  to  see  mathematics  as  relevant  to  their  lives  (CTGV,  1993). 

In  summary,  the  theory  of  situated  cognition  proposes  that  students  are  more  likely  to  develop  skills  that  will  transfer  to  new 
learning  situations,  if  they  are  engaged  in  authentic  activities  that  focus  on  development  of  problem  solving  skills.  Anchored 
instruction  has  applied  the  theory  of  situated  cognition  to  the  classroom  setting  and  promotes  the  development  of  activities  that 
create  situated  environments. 

Virtual  quests  such  as  those  produced  by  Classroom  Connect  and  The  JASON  Project,  are  anchored  learning  environments. 
Students  are  asked  to  help  scientists  in  the  field  find  solutions  to  real  world  problems.  They  place  students  into  a community  of 
learners  where  they  can  share  their  observations,  responses,  data  and  questions  with  an  authentic  audience.  They  can  focus  on 
solving  complex  problems  with  the  support  of  the  virtual  community.  Learning  content  such  as  math,  biology,  geography,  or 
culture  may  even  be,  in  many  cases,  unintentional  or  incidental. 

The  quests  produced  by  Classroom  Connect,  are  examples  of  comprehensive  virtual  quests.  Its  Adventure  Learning  Division 
produces  the  expeditions,  and  adventure  is  key  to  their  quest  experiences.  Their  team  of  explorers  travels  exotic  places  on 
bicycle,  a mode  of  transportation  to  which  students  can  easily  relate.  America  Quest  has  taken  students  to  the  Four  Comers 
region  of  the  United  States,  Australia,  Central  America,  Asia,  Africa  and  Galapagos  Islands. 

Each  of  their  quests  must  be  organized  around  a real  mystery,  or  what  CTGV  (1990)  would  refer  to  as  an  anchor  or 
macrocontext.  The  mystery  of  the  Spring  2000  America  Quest  expedition  focused  on  the  sudden  disappearance  of  the  Anasazi 
people  from  their  homes  over  700  years  ago.  Experts  in  the  field  don’t  have  a clear  understanding  of  what  happened  to  the 
Anasazi.  No  one  knows  the  answers  at  the  outset,  students,  teachers  or  field  experts,  and  it  is  unlikely  that  they’ll  know  the 
answers  when  the  expedition  ends.  The  problem  of  the  disappearance  of  the  Anasazi  is  a very  complex  one.  Clues  are  available 
but  they  conflict.  Some  experts  say  that  the  Anasazi  were  violent  people;  perhaps  they  were  running  from  their  enemies.  Others 
say  the  opposite  is  tme;  they  were  peaceful  and  were  affected  by  environmental  changes.  As  participants  in  America  Quest, 
students  are  exposed  to  the  complexity  of  real  world  problems  and  the  reality  that  solutions  come  from  considering  many 
perspectives. 

Student  participants  are  addressed  to  directly  by  the  America  Quest  expedition  team.  Daily  journal  postings  are  written  with 
them  in  mind,  including  descriptive  language,  interesting  topics,  and  lots  of  multimedia.  Students  are  considered  to  be  members 
of  the  team,  and  are  solicited  for  their  input  in  a number  of  ways.  Many  of  the  postings  from  the  expedition  team  include  polls. 
Students  can  email  the  team,  the  on-line  experts  and  each  other.  Responses  are  posted  to  the  site  bulletin  board  where  students  are 
also  invited  to  post.  The  audience  of  archaeologists,  anthropologists,  environmentalists  and  biologi  sts  value  their  responses  acting 
as  mentors  to  the  students’  developing  knowledge.  Immersed  in  the  culture  of  exploration,  students  are  witnessing  the  language 
and  behavior  of  experts  in  the  field.  They  are  called  upon  by  the  team  to  share  their  input  and  are  being  acknowledged  for  their 
thought  processes. 

In  order  to  make  informed  statements  and  decisions,  students  need  to  do  some  study  and  research.  America  Quest  also 
provided  a large  amount  of  archived  topical  information  on  its  site.  Students  are  encouraged  to  use  whatever  online  or  offline 
resources  are  available  to  them,  in  the  same  manner  that  their  teammates  in  the  field  do.  Immersed  in  the  culture  of  archaeology 
and  anthropology,  students  were  building  knowledge  about  Anasazi  culture,  environment  and  their  modem  day  successors.  They 
are  acquiring  knowledge  as  a tool  rather  than  a collection  of  facts. 

Although  most  classroom  participants  are  not  prepared  to  tackle  an  investigation  of  this  scale,  the  contributions  they  do  make 
are  clearly  part  of  the  process.  There  are  also  several  smaller  problems  that  students  are  called  upon  to  solve.  On  the  America 
Quest  expedition,  Dan  Buettner  presented  the  students  with  a weekly  ethical  dilemma  called  Dan’s  Dilemma,  for  example 
whether  or  not  to  photograph  the  remain  in  Hopi  gravesites.  At  the  end  of  the  week  a summary  of  students’  responses  was  posted, 
along  with  a sample  of  supporting  explanations  of  their  choices.  A weekly  dilemma  calls  on  them  to  think  outside  of  themselves, 
while  calling  on  their  own  personal  experience  to  make  informed  decisions. 
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As  with  any  trip,  time,  distance,  cost,  location,  and  supplies  are  important  things  to  be  aware  of. 
Figure  1.  Example  of  AmericaQuest  Daily  Data 


GPS  position: 

35°  36  minutes  N 

106°  20  minutes  W 

Sunrise: 

5:54  AM 

Sunset: 

6:25  PM 

Dental  floss  used: 

133  ft. 

Toilet  Paper  Used: 

407  ft. 

Pictures  Taken: 

4608 

Miles  Biked: 

260 

Miles  Hiked: 

52 

Flat  Tires  Fixed: 

3 

Ailments: 

Fatigue,  sliver  in  Dan’s  finger 

The  America  Quest  expedition  team  has  placed  a variety  of  data  within  the  context  of  the  expedition.  Figure  1 shows  an 
example  of  the  team  s daily  postings  (Buettner,  2000b).  This  authentic  data,  both  serious  and  whimsical,  is  provided  as  a resource 
for  study  of  geography,  weather,  health,  measurement,  and  mathematical  computation.  The  numbers  have  meaning  within  the 
context  of  the  quest,  whereas  traditional  approaches  often  stretch  to  create  context  for  manipulation  of  data. 

Personal,  archaeological,  biological  and  anthropological  information  are  presented  within  the  expedition  team’s  posted 
journal  entries.  A weekly  entry  by  the  team  biologist,  called  ‘‘Gross  and  Disgusting,”  described  items  that  attract  the  attention  of 
students  but  also  address  issues  of  a biological  nature  (2000).  For  example,  the  team  biologist’s  entry  on  March  9, 2000  discussed 
the  discovery  of  road  kill. 

“ The  other  day  I was  . . . searching  for  mule  deer.  I came  around  a comer  and  there  ...  was  my  first  animal ! 

As  I got  closer  I saw  it  was  a cat  but  it  was  so  bloated  and  putrid  that  I didn’t  want  to  get  too  close.  (Allen,  2000) 

She  goes  on  to  discuss  what  kinds  of  animals  are  more  likely  to  become  victims  of  road  kill  and  why,  and  that  bacteria  is  largely 
responsible  for  road  kill  clean  up.  It’s  easy  to  see  that  both  the  topic  and  language  can  pull  the  reader  in,  drawing  attention  to 
what’s  “gross”  and  away  from  the  challenges  of  reading. 

In  the  case  of  road  kill,  the  team  biologist  asked  students  to  think  about  the  issue  of  road  kill  in  Arizona  and  apply  what 
they’d  learned  to  the  entire  United  States.  She  polled  them  on  how  the  United  States  should  reduce  the  road  kill  problem.  The 
expedition  experience  gave  the  students  a structure  upon  which  informed  opinions  could  be  made.  Situated  cognition  may  also 
help  with  the  transfer  of  problem  solving  skills  to  new  situations  (CTGV,  1993). 

Multimedia  is  a major  supporting  factor  in  the  America  Quest  experience.  Each  weekly  Kid  Profile,  for  example,  included  a 
video  of  the  host  student  and  a recording  of  them  speaking  or  singing.  In  some  cases  they  were  speaking  in  the  Navaho  language. 
As  virtual  members  of  the  expedition  team,  they  can  experience  more  closely  what  the  team  in  the  field  is  experiencing.  Another 
aspect  of  the  audio-visual  nature  of  the  quest  site  is  that  it  supports  the  varied  learning  styles  of  the  participants  and  provides  cues 
for  students  who  may  have  learning  difficulties. 

Brown,  Collins  and  Duguid  stress  the  value  of  the  enculturation  of  the  learner  into  the  domain,  or  content  (1989).  By 
observing  the  members  of  the  new  culture,  in  the  case  of  the  America  Quest  expedition  team,  archeologists,  anthropologists  and 
biologists,  students  can  learn  the  behavior  and  language  of  the  field  experts.  The  environment  encourages  student  interaction  with 
the  adult  experts  as  well  as  with  other  student  team  members.  Students  can  communicate  through  email,  message  boards  and 
occasional  scheduled  chat  rooms. 

Collaboration  and  cooperation  are  key  characteristics  of  an  anchored  learning  environment  (CTGV,  1993).  Real  world 
problems  are  solved  in  environments  where  many  people  contribute  to  the  search  for  solutions.  Complex  problems  often 
necessitate  the  input  of  many.  Students  with  weaker  skills  can  contribute  to  the  process  while  they  benefit  by  learning  from  their 
peers.  The  process  of  problem  solving  within  a group,  requires  articulation  of  thought.  Students  must  be  clear  about  their 
thinking’.  When  posting  to  the  site,  the  America  Quest  team  chose  models  of  articulate  thought,  what  they  referred  to  as  the  most 
interesting  questions  and  responses,  to  post  on  the  bulletin  board.  They  are  able  to  view  good  models  of  communication. 

The  America  Quest  experience  also  builds  a sense  of  community.  At  each  location  the  team  selects  a student  for  the  Kid 
Profile  to  be  interviewed  and  introduced  on  the  website.  Students  often  have  misconceptions  about  people  from  different  parts  of 
the  world  or  from  different  cultures.  The  Kid  Profile  helps  children  develop  some  global  awareness  and  cultural  sensitivity. 
Profiles  include  information  about  the  student’s  family,  daily  routine,  school  day,  personal  interests  and  goals.  Students  can  relate 
to  another  child  s experiences,  and  can  see  the  world  from  a different  perspective.  This  experience  can  change  a student’s,  and 
even  a teacher’s,  global  awareness. 

The  Challenges  of  Quests  in  the  Classroom 

Adoption  of  a new  instructional  approach  is  a challenge  for  any  classroom  teacher.  Accountability  for  standardized  test 
scores  makes  it  difficult  to  shift  attention  from  mandated  curriculum  schedules.  For  many,  a cultural  shift  also  needs  to  take  place 
in  the  classroom.  Teachers  and  students  must  move  from  the  more  traditional  didactic  to  a more  process  oriented  leamine 
approach  (SCOPE,  200 1 ).  ^ 
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Research  has  shown  that  successful  adoption  of  innovations  is  dependent  on  six  factors:  relative  advantage,  compatibility, 
complexity,  trialability,  observability  and  reinvention  (Rogers,  p,15-16).  Adopters  must  perceive  that  the  new  technology  has  a 
relative  advantage  over  the  usual  way  of  doing  things.  Teachers  must  believe  that  the  quest  experience  offers  enough  value  to 
justify  modification  of  mandated  curriculum.  They  need  time  to  explore  the  quest  and  talk  to  other  teachers  who  believe  in  the 
quests’  value. 

Teachers  considering  participation  in  a virtual  quest,  want  to  know  if  this  new  learning  environment  is  compatible  with  their 
own  their  personal  values  about  education.  Teachers  who  favor  problem-based,  student -centered  and  process  oriented  approaches 
may  be  best  suited  for  virtual  quests.  The  complexity  of  an  innovation  has  an  impact  on  its  adoption.  Innovations  perceived  as 
too  complex  or  difficult  to  understand  are  less  likely  to  be  successfully  adopted.  A teacher  approaching  a quest  for  the  first  time 
must  be  comfortable  with  the  technology  and  its  management  in  the  classroom.  They  must  decide  which  quest  experiences  to  use, 
what  instructional  focus  to  choose,  and  how  much  time  to,  Trialability  is  another  factor  impacting  successful  adoption.  It  gives 
the  uncertain  adopter  an  opportunity  to  experiment,  and  become  comfortable  with  the  new  technology.  Teachers  have  the 
opportunity  to  visit  archived  virtual  quests  on  the  Quest  Channel  at  Classroom  Connect’s  website  prior  to  the  expedition 
(www.classroom,com),  or  to  visit  the  quest  website  before  the  quest  starts.  Important,  particularly  to  adoption  of  any  innovation 
in  the  educational  setting,  is  the  degree  to  which  the  impact  of  an  innovation  is  observable  to  others,  i.e.  teachers,  parents  and 
administrators.  Improved  student  motivation  makes  the  adoption  of  innovation  more  likely.  Once  an  innovation  is  adopted,  if  the 
adopter  is  able  to  modify  or  reinvent  the  innovation  to  customize  it  to  their  needs,  then  it  is  more  likely  to  continue  to  be  used. 
Quests,  such  as  America  Quest,  are  designed  so  that  the  breadth  of  the  experience  and  resources  leaves  room  for  teachers  with 
varying  student  or  curriculum  needs  to  make  decisions  about  what  they  will  focus  on  and  how  it  will  be  used.  The  more  often  a 
teacher  participates  in  quests,  the  more  familiar  they  become  with  its  components  and  the  more  easily  they  can  customize  the 
experience  for  their  students. 

The  introduction  of  the  virtual  quest  into  the  classroom  environment  brings  with  it  the  issue  of  teacher  readiness  to  use 
technology  as  an  instructional  resource.  Roger’s  model  for  adoption  of  innovation  touches  on  this  issue  when  considering  the 
adopter’s  perception  of  innovation’s  complexity,  i.e.,  whether  or  not  the  teacher  is  technically  ready  to  integrate  technology  into 
the  instructional  process.  According  to  Sandholtz,  Ringstaff  and  Dwyer,  adoption  of  technology  integration  happens  in  five 
stages:  entry,  adoption,  adaptation,  appropriation  and  invention  (1997). 

Teachers  at  the  entry  stage  are  struggling  with  learning  how  to  operate  computers  and  are  more  likely  to  rely  on  teacher 
directed  activities  away  from  the  computer.  At  the  adoption  stage,  teachers  have  developed  some  personal  skills  with  computers 
and  begin  to  show  some  interest  in  integrating  technology  into  their  lessons.  Teachers  at  these  stages  may  find  a virtual  quest 
overwhelming,  and  are  least  likely  to  participate. 

It’s  in  the  last  three  phases  that  change  in  instructional  practices  begins  to  take  place.  In  the  adaptation  phase,  teachers  begin 
to  realize  that  computers  can  save  time,  that  students  are  highly  motivated  and,  therefore,  more  productive.  As  teachers  develop 
confidence  in  and  mastery  of  their  own  technical  skills,  they  begin  to  realize  the  natural  association  of  technology  and  work.  At 
this  stage,  appropriation,  students  are  working  at  the  computer  more,  interacting  more  collaboratively  and  are  more  involved  in 
project-based  learning.  At  the  height  of  the  integration  continuum  is  the  invention  stage.  It  is  at  this  point  teachers  have  moved  to 
a more  constructive  approach  to  learning  in  the  classroom,  have  begun  to  question  more  traditional  teaching  approaches  and  are 
frequently  collaborating  with  other  teachers.  It  seems  that  teachers  at  these  last  three  stages  would  be  most  ready  to  consider 
participation  in  virtual  quests,  particularly  those  in  the  invention  stage.  Their  collaborative  and  project-based  classrooms  are  well 
suited  for  anchored  instruction. 

In  1993,  the  Cognitive  and  Technology  Group  at  Vanderbilt  (CTGV)  revisited  their  1990  classroom  implementation  of  the 
Jasper  Woodbury  series  and  looked  at  the  challenges  of  teaching  an  anchored  curriculum.  They  noted  that  one  of  the  greatest 
challenges  for  the  teacher  was  the  shift  in  their  role  as  provider  of  information  to  coach. 

Teachers  in  CTGV’s  research  group  questioned  the  value  of  the  Jasper  series  activities  and  wondered  where  it  fit  into  their 
curriculum.  They  also  worried  that  taking  their  students  out  of  the  mandated  curriculum  for  a month  would  impact  their 
achievement  test  scores.  CTGV  found  that  achievement  test  results  of  participating  students  were  equal  or  better  than  those 
students  that  did  not  participate  (CTGV,  p.  58).  The  Classroom  Connect  quest  experiences  are  designed  with  national  curriculum 
standards  in  mind  and  should  make  their  acceptance  as  supporting  local  curriculum  easier. 

Anchored  instruction,  problem-based  learning  and  virtual  quests  lend  themselves  to  portfolio  and  performance  based 
assessment  (SCOPE,  2001  & CTGV,  1993).  Virtual  quests,  such  as  America  Quest,  have  assessment  opportunities  built  in;  the 
Weekly  Quiz  is  an  online  review  of  the  event  of  the  week;  performance-based  rubrics  are  available  in  the  Teacher’s  Lounge;  and 
suggestions  for  developing  student  portfolios  can  be  found  in  the  teacher’s  guide. 

Students,  too,  who  are  used  to  a more  didactic  approach  to  learning,  may  not  know  how  to  participate  in  a learning 
environment  that  is  more  process  oriented  and  where  the  path  to  solving  problems  is  less  clear.  Used  to  working  as  individuals, 
the  shift  to  cooperative  learning  may  take  some  training  and  practice. 

In  conclusion,  virtual  quests  are  excellent  examples  of  learning  environments  that  engage  students  in  real-world  problem 
solving  in  authentic  settings.  Research  has  shown  that  students  engaged  in  situated,  anchored  learning  environments  performed  as 
well  as  or  better  on  standardized  tests  than  the  control  group  (CTGV,  1993). 

Issues  of  virtual  quest  adoption  in  the  classroom,  however,  include  consideration  of  a teacher’s  readiness  to  try  an  innovative 
approach  to  instruction,  their  perception  of  its  compatibility  with  their  teaching  style,  the  ability  to  which  he  or  she  can  try  it  out, 
see  how  well  it  works  and  modify  it  to  fit  their  curriculum  and  instructional  style.  Because  virtual  quests  are  by  nature 
technologically  supported,  adoption  readiness  is  also  contingent  on  the  teachers’  level  of  technology  adoption.  Teachers  who  are 
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comfortable  with  technology  themselves  and  as  a classroom  resource  are  more  likely  to  find  that  virtual  quests  fit  in  their 
classrooms. 

Methodology  and  Results 

The  2001  MayaQuest  expedition  was  conducted  with  six  elementary  school  teachers  from  the  author’s  school.  The  teachers 
registered  their  classes  at  the  Classroom  Connect  web  site  and  were  committed  to  incorporate  MayaQuest  as  part  of  their 
instructional  day.  These  six  teachers  were  then  interviewed  about  their  instructional  characteristics,  issues  surrounding  the  use  of 
quests  in  the  classroom,  and  their  level  of  comfort  with  technology.  Although  conducted  as  objective  research,  the  author’s 
enthusiasm  for  the  topic  introduced  some  subjectivity.  One  fifth  grade  teacher  chose  to  pair  up  with  a third  grade  teacher  and 
held  a peripheral  role  in  the  daily  planning.  All  six  teachers  agreed  to  the  interviews,  which  lasted  25-45  minutes  and  were 
conducted  during  their  planning  time.  The  MayaQuest  project  was  in  its  second  and  third  week  when  the  interviews  were  started. 

The  interviews  were  audio  taped,  transcribed  and  coded  in  reference  to  the  three  focus  areas  for  this  study:  1)  teacher 
instructional  characteristics  including  their  comfort  with  and  use  of  technology  as  an  instructional  tool,  2)  student  outcome  as  a 
result  of  the  MayaQuest  experience  and  3)  issues  surrounding  successful  implementation  of  the  quest  in  the  elementary 
classroom. 

Each  classroom  at  Sawnee  Elementary  School  has  5 networked,  Internet  accessible  multimedia  computers.  Each  teacher  has 
her  own  multimedia  notebook  computer  with  a video  card  that  allows  for  projection  to  the  classroom  TV,  and  that  is  also  network 
able.  In  addition,  there  are  three  mobile  mini-labs  that  contain  ten  notebook  computers  per  cart.  These  notebooks  have  a wireless 
connection  to  the  network.  Students  shared  the  notebooks  or  worked  individually  in  an  activity  center  arrangement. 

Teacher  Instructional  Characteristics 

All  teachers  taught  regular  classrooms  that  included  four  fourth  grades,  one  third  grade,  and  one-fifth  grade  classroom.  Three 
of  the  teachers  had  tried  previous  Classroom  Connect  virtual  quests  with  limited  success. 

When  asked  about  their  comfort  level  with  technology  as  an  instructional  tool,  on  a scale  from  0 (no  comfort)  to  5 (total 
comfort),  5 teachers  in  the  group  placed  themselves  well  above  average  (4-5)  and  one  teacher  rated  herself  as  average  (3).  (See 
the  chart  below.) 


Table  1 .Characteristics  of  Teachers  Participating  in  the  Study 


Teacher 

Grade 

Level 

Taught 

Years  Teaching 
Experience 

Prior  Online 

Project 

Experience 

Technology 
Comfort  Self 
Rating  Out  of  5 

Self-stated 
Role  in  Student 
Learning 

Student  Desk 
Grouping 

Anne 

3 

7 

No 

4 

Facilitator 

Groups 

Betty 

4 

11 

Yes 

4 

Guide 

Groups 

Carol 

4 

11 

Yes 

4.5 

Facilitator 

Groups 

Diane 

4 

4 

Yes 

4 

Facilitator/ 

Direct 

Instruction 

Rows 

Edith 

4 

5 

No 

5 

Facilitator/ 

Direct 

Instruction 

Rows/ 

Grouped 

Faye 

5 

25 

Yes 

3 

Manager 

Rows 

As  a result  of  classroom  observation  and  exp^ience  working  with  these  teachers,  this  author  would  rate  their  apparent  comfort 
with  technology  about  the  same  as  or  slightly  higher  than  they  rated  themselves. 

The  teachers  were  asked  about  their  approach  to  instruction,  i.e.,  what  they  saw  as  their  role  in  student  learning.  Ann  and 
Carol  defined  their  role  as  facilitators.  Betty  referred  to  herself  as  a guide.  Faye,  on  the  other  hand,  referred  to  herself  as  a 
manager.  All  of  the  teachers  stated  that  they  used  cooperative  group  work  as  part  of  their  normal  daily  instruction.  Diane  and 
Faye  said  that  they  used  cooperative  group  work  either  weekly  or  bi-weekly.  Edith  noted  that  she  used  computers  more  often 
after  the  beginning  of  Maya  Quest. 

Cooperative  group  work  in  a classroom  is  best  facilitated  by  a desk  arrangement  that  allows  for  ready  communication.  Within 
a week  of  starting  the  expedition,  Edith  had  rearranged  her  classroom  to  four  desk  groupings. 

Betty,  Carol  and  Diane  mentioned  that  they  had  attempted  previous  Classroom  Connect  quests  with  limited  success,  and  Betty 
described  her  past  experience  (AmericaQuest  2000)  as  “overwhelming.” 

All  teachers  indicated  that  prior  to  MayaQuest  their  students  used  computers  for  word  processing,  i.e.  journaling,  letter 
writing,  cooperative  story  writing  and  research  reports;  all  took  advantage  of  Encarta  Multimedia  Encyclopedia  ’97  and  the 
Internet;  and  several  used  notebooks/TV  connections  for  whole  group  demonstration. 

Throughout  the  year  Betty,  Carol,  Diane,  Edith  and  Faye  used  Power  Point  presentations  for  morning  announcements  that 
were  projected  on  the  classroom  TV.  These  included  reminders  of  what  the  students  should  do  when  they  arrived  in  the 
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classroom,  journal  prompts,  and  learning  center  management.  Several  also  created  hyperlinked  learning  centers,  for  example, 
creating  a single  slide  to  explore  electricity  and  magnetism  by  linking  to  several  related  websites  (students  rotated  through  the 
computers  stations  with  this  teacher  created  focus  sheets). 

Looking  at  these  teachers’  level  of  adoption  of  technology  in  reference  to  the  ACOT  model  none,  it  is  possible  to  place  them 
loosely  along  the  continuum  of  growth.  Faye’s  instructional  emphasis  placed  her  in  the  adoption  stage,  whereas  Anne,  Carol, 
Diane  and  Edith  style  of  instruction  placed  them  at  the  Appropriation  stage. 

Student  Outcome 

Each  of  the  teachers  was  asked  to  comment  on  student  outcome,  as  a result  of  MayaQuest  participation.  Several  noticed  an 
increase  in  the  students’  independent  use  of  technology.  Students  developed  independent  problem  solving  skills  through  use  of 
the  Mystery  Photo.  For  example,  the  “Mystery  Photo”  is  a picture  of  an  object  taken  at  close  range  so  that  its  size  and  context  are 
questionable  Students  were  asked  to  guess,  or  infer,  what  it  was  with  the  least  number  of  clues.  Betty  explained: 

“Of  course,  the  kids  love  the  Mystery  Photo...  We  looked  at  it,  we  got  our  clues... and  we  still  didn’t  have  a clue  what  it  was. 
But  they  knew  it  was  some  kind  of  plant.  It  said  it  wasn’t  a bush,  so  they  guessed  it  was  some  kind  of  tree.  And  then  it  said 
something  about  ants.  So,  then  I showed  them  how  they  could  go  into  the  Quest  library  and  start  looking  for  websites  about 
plants  and  helped  them  to  come  up  with  what  do  we  need  to  look  for... We  had  to  look  in  several  places,  and  they  found  the 
tree.  They  couldn’t  WAIT  to  get  over  to  the  computer  to  write  their  answer... they’ll  call  me  over,  I found  it!  I figured  it  out! 
Instead  of  them  just  guessing  and  seeing  what  the  answer  is,  which  is  what  mine  did  in  the  past  because  I didn’t  do  it  that  way 
and  show  them  how  to  find  the  answer.  They’re  getting  so  much  better  at  research  skills,  on  the  Internet  sites,  on  the  Encarta 
sites.  They’re  skimming  for  information.” 

Increased  student  motivation  was  noticed  by  all  of  the  teachers.  Betty  noted  that  her  class  would  be  so  involved  in  what  they 
were  doing  that  they  would  forget  to  go  to  lunch.  Anne,  a third  grade  teacher,  noted  a student  who  had  come  into  third  grade 
barely  reading  at  all  who  was  able  to  read  MayaQuest  entries  and  “state  five  facts  he’d  read.”  She  also  stated  that  reading  this 
type  of  text,  i.e.,  expository  text  helps  to  “...train  the  reader’s  eye,  to  read  for  understanding.” 

Edith  used  words  from  the  quest  for  the  weekly  vocabulary.  Faye  had  observed  her  students  reading  the  journal  entries  on  the 
quest  website  and  that  “...they  read  through  the  text  with  vocabulary  I know  they  don’t  know  but  continued  to  read  anyway  to  get 
the  gist  of  what  they  were  saying.” 

Several  teachers  noted  that  learning  curricular  content  in  the  context  of  the  quest  helped  students  to  make  connections  to  their 
learning.  Betty  and  Carol  used  ecosystems  as  their  focus.  Betty  said, 

“They’re  talking  about  producers.  They’re  talking  about  consumers  and  ...they’re  using  it  in  conversation  now.  They  read 
something  about  an  animal...!  guess  it  was  about  the  termite  eating  rotten  wood  and  I heard  them  say,  ‘Oh,  so  that’s  a 
decomposer.’  They’re  making  the  connection.” 

All  of  the  teachers  commented  on  the  importance  of  the  realism  and  opportunities  for  decisions  making  that  are  part  of  the 
quest  experience.  Their  participation  in  the  quest 

“...was  not  just  a scenario  in  a book.  It’s  real.  Someone’s  really  talking  to  us  and  thinking  we’re  important.  They  think  that 
what  we  have  to  say  matters.  And  these  fourth  graders,  they  feel  so  important.  That  they  want  to  hear  what  their  vote  is  and 
that  they  make  a difference.”  [Betty] 

The  Kid  Profile  had  a particular  impact  on  their  students.  The  students  were  most  intrigued  by  Alfredo,  a 9year-old 
shoeshine  boy  in  Guatemala  who  has  lived  on  his  own  for  3 years  (Buettner,  2001).  These  teachers  did  a compare  and  contrast 
activity  comparing  basic  characteristics  of  their  lives  and  Alfredo’s.  The  difference  was  “stark,”  as  Carol  put  it. 

Table  2.  Teacher  Selected  Curricular  Focus  for  MayaQuest 


TEACHER 

Science 

Social  Studies 

Math 

Language  Arts 

Anne 

Animal  classification 

Reading  maps  and  charts, 

communities, 

compare/contrast 

Gathering  data, 
graphing 

Expository  writing,  reading  for 
facts 

Betty 

Ecosystems,  food  webs 

Reading  maps,  charts, 
latitude  & longitude, 
compare/contrast 

Gathering  data, 
graphing 

Journaling,  writing  research 
reports 

Carol 

Ecosystems,  food  webs 

Reading  maps,  charts, 
latitude  & longitude, 
compare/contrast 

Gathering  data, 
graphing 

Journaling,  writing  research 
reports 

Diane 

Animal  classification, 
weather 

Writing  research  reports 

Edith 

Animal  classification, 
weather 

Reading  maps,  charts, 
compare/contrast 

Gathering  data, 
graphing 

Writing  research  reports 

Faye 

(Mentoring) 

Reading  expository  text 

404 


Issues  Surrounding  MayaQuest  Implementation 

During  the  year  of  this  study,  teachers  had  access  to  5 classroom  computers,  a teacher  laptop  and  three  mobile  notebook 
computer  carts  available  on  sign  out  from  the  media  center.  When  asked  what  would  keep  them  from  participating  in  future 
quests,  Betty  and  Carol  stated  that  access  to  the  technology  would  be  a major  factor. 

Table  2.  Teacher  Selected  Curricular  Focus  for  MayaQuest 

All  teachers  mentioned  the  need  to  find  a curricular  focus  for  the  quest  (See  Table  2).  Several  teachers  felt  pressure  to  meet  the 
established  curriculum  and  prepare  for  the  standardized  tests. 

In  response  to  this  perceived  pressure,  however,  some  teachers  noted  that  their  students  learned  helpful  content  and  skills  in 
preparation  for  standardized  testing.  Betty  noted  that,“If  I hit  these  objectives  [the  QCC’s]  through  the  quest,  they’re  going  to  like 
it  better...  if  I can  find  the  right  [curricular]  focus  I can  do  any  quest.” 

Assessment  of  student  outcome  was  an  issue  raised  by  several  of  the  teachers,  even  though  no  question  addressed  it  directly. 
“Can  I assess  all  of  their  growth  in  understanding  of  the  world,  their  communication?  No!”  [Anne]  Portfolios,  rubrics  and  use  of 
the  quest’s  on-line  quizzes,  however,  were  ways  that  most  of  the  teachers  measured  student  growth. 

Discussion 

Teacher  and  student  response  to  the  MayaQuest  experience  was  uniformly  positive.  Teachers  spoke  excitedly  about  their 
students’  response  to  the  experience  and  about  their  own  enjoyment  with  teaching  with  MayaQuest.  Comments  throughout  the 
five  weeks  included  how  engaged  the  students  were  and  how  ordinarily  reluctant  learners  were  participating  fully.  They  remarked 
that  their  students  made  connections  between  their  newly  introduced  content  skills  and  the  quest  experience.  At  the  end  of 
MayaQuest,  all  participating  teachers  agreed  that  it  was  a valuable  experience  that  they  plan  to  repeat  the  following  school  year. 

The  teachers  in  this  study  were  all  relatively  experienced;  even  the  least  experienced  had  taught  for  four  years  (see  Table  1). 
As  a result,  they  had  a good  understanding  of  school  culture,  curriculum  management,  the  Georgia  QCC’s,  and  basic  classroom 
management.  Their  self-proclaimed  comfort  level  with  technology  was  relatively  high.  These  teachers  were  well  along  the 
continuum  of  technology  adoption  as  defined  by  Sandholtz,  Ringstaff  and  Dwyer’s  model  (1997). 

All  of  the  teachers  said  that  given  adequate  resources,  they  would  sign  up  for  another  quest.  For  these  teachers,  learning  best 
takes  place  in  an  environment  that  is  interdisciplinary,  involves  authentic  problem  solving,  is  collaborative  and  supports  skill 
development.  Some  teachers  have  found  previous  quests  to  be  overwhelming.  However,  once  they  determined  a curricular  focus, 
they  looked  at  the  quest’s  complexity  more  as  a rich  resource  than  a roadblock.  Betty,  Carol  and  Diane  had  tried  previous  quests. 
This  certainly  helped  them  with  understanding  what  was  involved.  They  all  reviewed  the  teacher’s  guide  thoroughly  before 
starting  and  Anne  said  that  this  was  key  in  her  finding  the  focus  for  her  instruction.  Key  to  their  final  adoption  of  MayaQuest,  and 
possibly  future  quests,  was  their  1)  ability  to  reinvent  or  modify  the  experience  to  fit  their  instructional  style  and  curricular  needs 
and  2)  the  impact  that  it  had  on  their  students’  academic  growth. 

Perhaps  the  most  significant  common  instructional  characteristic  of  these  teachers  was  how  they  defined  their  role  in  student 
learning.  They  proclaimed  themselves  as  facilitators  focused  on  creating  a learning  environment  for  their  students  rather  than 
simply  imparting  information  or  delivering  curriculum.  They  actually  verbalized  a preference  for  an  interdisciplinary  approach  to 
instruction,  i.e.  teaching  in  this  manner  helped  students  make  connections  across  the  curriculum. 

Their  classroom-learning  environment  regularly  included  student  to  student  and  student  to  teacher  collaboration  and 
cooperation.  They  recognized  the  classroom  as  a community  of  learners  and  organized  student  activities  with  that  in  mind  daily. 

All  of  the  teachers  used  the  classroom  desktops,  and  Anne,  Betty,  Carol  and  Edith  also  used  the  student  notebooks  to  support 
MayaQuest.  Classroom  student  assistants,  called  TechnoBuddies,  were  responsible  for  pick  up,  setup,  troubleshooting,  break 
down  and  return  of  the  notebook  carts  to  the  media  center.  They  developed  independence  from  daily  technology  use,  which  made 
it  possible  for  the  teacher  to  focus  on  facilitating  the  various  classroom  activities.  In  the  classroom,  students  worked  in  pairs,  or 
sometimes  individually  at  the  computer  to  interact  with  the  MayaQuest  web  page,  do  research  or  publish  their  Power  Point 
presentations,  reports  and  journal  entries.  Because  these  teachers  supported  collaboration,  their  students’  skills  with  both  the 
technology  and  the  content  grew.  The  students  frequently  learned  technical  and  content  information  from  each  other. 

Key  to  the  success  of  MayaQuest  for  these  teachers  was  access  to  the  technology.  All  but  Diane  and  Faye  noted  that  if  they 
had  not  had  access  to  the  notebooks  on  a daily  basis,  that  the  impact  of  the  MayaQuest  experience  would  not  have  been  as  great. 
There  was  new  information  on  the  site  daily,  and  because  there  was  so  much  that  was  interesting,  both  the  students  and  teachers 
felt  a need  to  keep  up  with  team’s  adventures.  Daily  access  also  allowed  for  students  to  respond  daily  to  journal  postings, 
continue  research,  draft,  revise  and  produce  quality  responses  to  their  learning.  Missing  a day  was  not  a choice  they  wanted  to 
make.  When  asked  if  they  would  do  MayaQuest  with  just  the  five  desktops  in  their  room,  the  teachers  said  that  they  would,  but 
felt  that  fewer  students  would  be  able  to  interact  with  the  site  daily  or  produce  the  same  quantity  or  quality  of  work.  As  Betty 
said,  “They  have  to  touch  it  everyday  or  you  lose  them.” 

The  teachers’  view  of  students  as  part  of  a community  of  learners  and  their  willingness  to  participate  in  MayaQuest 
demonstrated  a broad  view  of  education.  They  all  had  the  established  curriculum  to  teach  and  goals  to  meet  but  were  flexible 
enough  in  their  view  of  education  to  be  able  to  blend  these  goals  into  the  quest  experience.  Curricular  focus  was  very  important 
to  their  adoption  of  the  quest,  and  as  Betty  stated,  “If  I can  find  the  curricular  focus  I can  do  any  quest.” 

The  MayaQuest  2001  experience  was  an  excellent  example  of  anchored  instruction.  Students  were  placed  in  a learning 
environment,  or  macrocontext,  that  was  truly  authentic.  They  worked  with  a team  of  field  experts  over  a period  of  time  to  find 
clues  about  the  mystery  of  the  fall  of  Mayan  civilization,  a complex  problem  with  few  clear  answers.  Students  worked  as 
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practitioners.  They  shared  their  research  and  theories,  collaborating  not  only  with  their  classroom  peers,  but  also  with  students 
around  the  world  and  with  field  experts.  Considered  part  of  the  research  team,  their  input  was  valued,  which  added  to  the 
authenticity  of  their  participation. 

The  most  significant  outcome  of  the  MayaQuest  experience  for  these  teachers  was  the  impact  that  it  had  on  student  learning. 
According  to  the  teachers,  student  response  to  MayaQuest  was  unanimously  positive.  Even  students  considered  to  be  at  risk,  were 
excited  enough  by  the  content  that  they  seemed  to  forget  that  they  were  reading,  writing  and  doing  mathematics.  Teachers 
noticed  increased  scores  on  vocabulary  tests,  increased  reading  comprehension  and  writing  production.  Anne’s  third  grader,  a 
non-reader  at  the  beginning  of  the  year,  was  reading  web  site  postings  and  able  to  state  facts  about  what  he’d  read  during  the 
quest.  All  teachers  noted  that  student  critical  thinking  showed  growth,  as  evidenced  by  improved  problem  solving  and  journal 
reflections. 

The  range  of  topics  and  the  authenticity  of  the  issues  blurred  the  boundaries  between  mathematics,  science,  reading  and  so 
on.  Aided  by  their  teachers,  and  often  independently,  students  made  connections  to  curricular  goals  and  applied  new  learning  to  a 
real  context.  Students  meeting  children  through  the  Kid  Profile  were  often  moved  by  their  counterparts’  lifestyle  and  easily  made 
connections  to  their  own  lives  certainly  developing  a broader  global  view. 

Implications 

Those  teachers  who  view  themselves  as  classroom  facilitators  of  student  learning,  take  a flexible  approach  to  instruction, 
have  a clear  direction  in  terms  of  curricular  goals,  and  prefer  to  help  students  make  connections  to  learning  and  real  life  through 
an  interdisciplinary  approach  to  learning,  are  more  likely  to  adopt  virtual  quests.  Teachers  with  well-developed  and  frequently 
used  technology  skills  are  also  more  likely  to  tie  quests  into  curriculum  goals.  The  results  point  toward  these  teachers  as  the  best 
candidates  for  successful  quest  experiences. 

Teachers,  who  participate  in  a virtual  quest  for  the  first  time,  might  be  encouraged  to  speak  with  other  teachers  who  have 
participated  in  previous  quests.  Communication  could  happen  through  the  Classroom  Connect  Quest  Channel  where  previous  and 
current  quest  bulletin  boards  are  available  for  teachers. 

The  high  level  of  student  motivation,  increase  in  student  work  production,  and  good  work,  cannot  be  ignored.  It  is  the  most 
exciting  outcome  of  virtual  quest  participation.  By  their  nature  the  quests’  adventure,  authenticity  and  interaction  with  the  quest 
team  in  the  field,  are  so  engaging  that  students  seem  to  learn  without  realizing  it.  The  teachers  in  this  study  noted  that  the  amount 
and  type  of  reading  that  their  students  were  doing,  i.e.  daily  high  level  expository  reading,  writing,  and  the  development  of  their 
students’  critical  thinking  skills  through  quest  activities,  did  more  to  prepare  their  students  for  standardized  testing  than  did  drill 
and  practice.  The  students  developed  skills  that  were  transferable  to  new  academic  settings. 

Access  to  technology  as  part  of  virtual  quest  success  was  an  issue  brought  up  by  teachers  in  this  study.  Although  they  had 
five  networked  multimedia  desktops  in  their  classrooms,  four  of  them  felt  that  the  high  level  of  student  achievement  was  related 
to  the  daily  use  of  the  additional  ten  wireless  notebook  workstations.  The  implications  are  that  teachers  with  less  technology 
access  will  find  quest  implementation  more  challenging. 

Further  Research 

The  results  of  this  study  lead  to  some  new  questions  for  further  research.  First,  would  a larger  sample  of  interviewees  show 
the  same  teacher  characteristics  for  quest  participants?  Second,  would  a sample  of  teachers  who  do  not  know  the  author  provide 
different  results? 

The  next  step  for  this  study  would  be  to  measure  student  outcome  more  quantitatively.  How,  and  how  much,  does  student 
achievement  increase  as  a result  of  participation  in  a virtual  quest?  Student  data  on  language  arts,  mathematics,  science,  social 
studies  and  critical  thinking  skills  prior  to  and  following  quest  participation  would  give  data  for  comparative  study.  Standardized 
test  scores  of  students  participating  in  the  quests  might  be  compared  to  those  who  use  drill  and  practice  preparation. 

Final  Comments 

This  study  demonstrates  the  value  of  the  anchored  instructional  approach  to  learning,  as  represented  by  MayaQuest.  Student 
participants  in  this  study  were  highly  engaged,  motivated,  and  appeared  to  benefit  from  increased  skill  development  and  work 
production.  Students  experienced  models  of  problem  solving,  teamwork,  thought  processes  and  language  in  an  authentic  manner, 
not  just  as  a scenario  developed  to  simulate  reality.  That  authenticity  was  made  possible  by  access  to  the  technology,  bringing  the 
field  experience  into  the  classroom  while  transporting  students  to  the  field. 

Adoption  of  virtual  quests  can  motivate  and  rejuvenate  teachers,  but  certain  teachers  are  also  more  likely  to  adopt  virtual 
quests.  They  are  teachers  who  value  a student -centered  approach,  view  themselves  as  facilitators  of  learning,  are  flexible  enough 
to  modify  the  prescribed  curriculum  to  fit  the  quest  experience,  and  who  are  comfortable  with  technology. 

Virtual  quests  are  a promising  instructional  approach.  It  is  not  recommended  that  a school  or  district  adopt  them  globally. 
Rather,  teachers  should  be  given  the  opportunity  to  adopt  or  not  adopt  virtual  quests.  It  is  also  important  that  they  be  provided 
both  technical  and  instructional  support  as  needed.  For  the  appropriately  oriented  teacher,  anchored  instruction,  in  the  form  of 
virtual  quests,  is  a veiy  exciting  addition  to  the  learning  environment. 
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Abstract 

A popular  belief  is  that  a learner's  degree  of  self-directedness  impacts  the  success  of  students  learning  at  a distance  Also 
conceivable  is  that  self-ejficacy  may  affect  student  satisfaction  in  distance  courses.  Additionally,  satisfaction  may  help  to  identify 
success  in  distance  education  students.  However,  the  relationship  between  these  variables  has  yet  to  be  explored.  In  the  summer 
of 2001,  subjects  enrolled  in  web-based  courses  were  surveyed  to  determine  their  self-directedness  and  self-efficacy  and  general 
satisfaction  with  their  learning  experience.  Results  of  the  statistical  analyses  and  suggestions  for  future  research  are  discussed. 

Introduction 

It  is  a widely  held  believe  that  a learner’s  degree  of  self-directedness  plays  a role  in  the  success  of  students  learning  at  a 
distance  (e.g.  Kearsley,  1995,  Major  & Levenburg,  1999,  Moore,  1986).  It  is  also  thought  that  perceived  self-efficacy  may  have 
an  effect  on  student  satisfaction  in  distance  courses  (Clark,  1999,  Schneider  & Reinhart,  1995).  Additionally,  satisfaction  may  be 
important  in  identifying  future  success  of  distance  education  students  (Simonson  et  al,  2000). 

The  relationship  between  these  three  variables  has  yet  to  be  explored.  In  particular,  the  relationship  between  perceived  self- 
efficacy,  self-directedness  and  satisfaction  in  a web-based  course  may  play  a major  role  in  whether  or  not  a student  enrolls  in 
courses  at  a distance,  continues  with  study  in  a on-line  degree  program  or  recommends  such  courses/programs  to,  other  students. 
This  could  have  far-reaching  implications  for  distance  degree  programs  and  administration  considering  such  programs. 

Purpose  of  the  Study 

There  were  several  purposes  for  this  study.  The  first  was  to  determine  whether  there  is  a relationship  between  perceived  self- 
efficacy  and  self-directedness  in  students  enrolled  in  an  on-line  Masters  program.  A second  purpose  was  to  ascertain  whether 
there  was  a relationship  between  perceived  self-efficacy  and  satisfaction.  The  third  purpose  was  to  determine  whether  there  is  a 
relationship  between  self-directedness  and  satisfaction.  The  fourth  was  to  discover  whether  either  perceived  self-efficacy  or  self- 
directedness  played  a role  in  student  satisfaction. 

Subjects 

Subjects  were  graduate-level  students  enrolled  in  two  separate  web-based  courses:  ‘‘Foundations  of  Instructional 
Technology”  and  “Advanced  Technology  in  HPER”  (Health,  Physical  Education  and  Recreation).  Subjects  in  Foundations  of 
Instructional  Technology  are  working  toward  a Master  of  Science  in  Instructional  Design  and  Technology.  Subjects  in  Advanced 
Technology  in  HPER  are  working  toward  a Master  of  Science  in  Health,  Physical  Education  and  Recreation.  Both  Masters 
programs  are  fully  on-line  degree  programs  at  the  same  midwestem  university.  Both  classes  are  taken  near  the  beginning  of 
course  work  in  each  respective  major. 


Methodology 

The  survey  was  a combination  of  three  separate  assessments:  a perceived  self-efficacy  psychometric  instrument  (Jerusalem 
and  Schwarzer),  a self-directedness  questionnaire  (Guglielmino)  and  a survey  designed  to  measure  satisfaction  with  the  subject’s 
own  learning  and  satisfaction  with  the  learning  environment  (Summerville).  Subjects  were  surveyed  in  the  fifth  week  of  a six- 
week  summer  course.  Subjects  were  sent  an  e-mail  message  requesting  that  they  take  a survey  to  help  future  students  who  take 
courses  at  a distance.  This  survey  consisting  of  eighty-eight  individual  items. 

Instrumentation 

The  Generalized  Self-Efficacy  Scale 

The  Generalized  Self-Efficacy  Scale  is  a 10-item  survey  designed  in  1981  by  Jerusalem  and  Schwarzer  to  assess  perceived 
self-efficacy.  It  was  originally  developed  in  the  German  language  and  subsequently  translated  into  thirteen  different  versions. 
The  measure  was  tested  for  each  version  and  the  Chronbach  alpha  for  the  English  version  was  .90.  It  is  available  on-line  at 
http://userpage.fii-berlin.de/~health/selfscal.htm. 

Self-Directed  Learning  Readiness  Scale 

The  Self-Directed  Learning  Readiness  Scale  was  designed  by  Guglielmino  to  measure  a subject’s  readiness  to  engage  in  self- 
directed  learning.  The  SDLRS  has  been  used  by  hundreds  of  organizations,  translated  into  more  than  14  languages  and  used  in 
more  than  90  dissertations  since  its  development  in  1977.  Permission  was  given  by  the  author  to  use  the  survey  in  an  on-line 
format.  This  researcher  tallied  the  number  of  surveys  received  and  the  author  received  compensation  for  each  survey  completed 
by  the  subjects. 
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Summerville  Satisfaction  Questionnaire 

A satisfaction  questionnaire  was  developed  for  administration  in  a doctoral  dissertation  (Summerville,  1997)  and  adapted  for 
use  in  this  study.  Questions  were  designed  to  determine  if  students  were  satisfied  with  the  learning  environment,  were 
comfortable  with  the  learning  environment,  and  to  self-assess  learning  from  the  experience.  For  every  question,  an  equal  and 
opposite  question  was  written  to  ascertain  which  wording  best  asked  the  question  to  be  answered  (e.  g.,  “I  liked  learning  via  the 
Internet”  vs.  “I  did  not  like  learning  via  the  Internet”).  Two  statistical  consultants  evaluated  the  questions  for  face  and  construct 
validity.  These  questions  were  then  pilot  tested  with  a group  of  approximately  sixty  subjects  and  subsequently  scored  to 
determine  how  effective  the  instrument  was  at  measuring  satisfaction.  Finally,  the  completed  satisfaction  instrument  was 
subjected  to  a Cronbach’s  Coefficient  Alpha  and  a factor  analysis  to  establish  statistical  validity  and  reliability.  The  resulting 
Coefficient  Alpha  for  the  satisfaction  questionnaire  was  .89. 

Data  Collection 

Data  collection  began  during  the  fifth  week  of  a six-week  summer  course.  The  subjects  were  sent  an  email  message 
detailing  the  purpose  of  the  study,  requesting  subject  participation  and  the  message  included  the  URL  of  the  combined  survey. 
Due  to  the  agreement  reached  with  the  author  of  the  Self-Directed  Learning  Readiness  Scale,  the  subjects  were  given  48  hours  to 
complete  the  survey  before  it  was  removed  from  Internet.  The  surveys  were  returned  by  CGI  server  to  the  researcher  via  e-mail. 
There  were  twelve  useable  surveys  returned. 

Analysis  Procedures 

Returned  surveys  were  collected  and  data  was  coded  for  analysis  by  this  researcher.  SPSS  Statistical  software  was  used  for 
analysis.  Due  to  the  unknown  relationship  between  The  Generalized  Self-Efficacy  Scale,  the  Self-Directed  Learning  Readiness 
Scale,  and  the  Summerville  Satisfaction  Questionnaire,  a Pearson  Correlation  was  used  for  analysis  on  the  total  scores  for  each 
subject  on  each  instrument  to  determine  if  there  was  a correlation  between  the  instruments.  If  a relationship  was  found,  a 
stepwise  regression  procedure  would  be  used  to  determine  if  either  the  Generalized  Self-Efficacy  Scale  or  the  Self-Directed 
Learning  Readiness  Scale  impacted  satisfaction. 

Analysis  of  Pearson  Correlation 

A Pearson  Correlation  procedure  was  used  to  determine  if  there  was  a relationship  between  total  scores  on  the  perceived  self- 
efficacy,  self-directedness  and  satisfaction  instruments,  as  measured  by  the  combined  survey.  The  result  of  the  procedure  is  as 
follows: 


Perceived  Self- 
Efficacy 

Self-Directedness 

Satisfaction 

Perceived  Self- 

Pearson  Correlation 

1.000 

.770** 

.322 

Efficacy 

Sig.  (2-tailed) 

.003 

.308 

N 

12 

12 

12 

Self-Directedness 

Pearson  Correlation 

.770** 

1.000 

.577 

Sig.  (2-tailed) 

.003 

.050 

N 

12 

12 

12 

Satisfaction 

Pearson  Correlation 

.322 

.577* 

1.000 

Sign.  (2-tailed) 

.308 

.050 

N 

12 

12 

12 

**Correlation  is  significant  at  the  0.01  level  (2-tailed). 
*Correlation  is  significant  at  the  0.05  level  (2-tailed). 
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Analysis  of  Stepwise  Regression 

ANOVA  from  the  Regression  Model  for  Independent  Variable  Self-Directedness  and  Dependent  Variable  Satisfaction 


Model 

SSQ 

DF 

MSQ 

F 

Regression 

253.572 

1 

253.572 

4.978* 

Residual 

509.344 

10 

50.934 

Total 

762.917 

11 

*Denotes  significance  at  0.05  level 


Perceived  self-efficacy  was  not  found  to  be  a significant  predictor  of  satisfaction  in  this  model. 

Future  Research 

Since  there  was  a significant  correlation  between  self-directedness  and  student  satisfaction  and  degree  of  self-directedness 
was  found  to  be  a significant  predictor  of  student  satisfaction,  this  researcher  plans  to  investigate  tie  relationship  further. 
Additionally,  since  perceived  self-efficacy  and  self-directedness  were  so  highly  correlated,  the  relationships  between  those 
variables  will  also  be  investigated  to  a greater  degree. 

In  particular,  this  researcher  plans  to  survey  other  students  enrolled  in  on-line  courses  but  use  a different  schedule  for  survey 
administration.  The  perceived  self-efficacy  and  self-directedness  surveys  will  be  administered  at  the  beginning  of  the  semester 
while  and  the  perceived  self-efficacy  survey  will  be  readministered  and  satisfaction  survey  will  be  administered  at  the  end  of  the 
course. 

In  the  future,  this  researcher  plans  to  develop  prescriptions  for  helping  students  with  low  perceived  self-efficacy  and/or  low 
self-directedness  scores  have  a g"eater  degree  of  satisfaction  with  their  own  learning.  Additionally,  the  relationship  between 
satisfaction  and  achievement  may  also  be  explored  and,  if  warranted,  prescriptions  will  be  developed  for  students  with  difficulty 
in  these  areas. 
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Abstract 

This  Study  examined  the  learner  characteristics  of  enrol  lees  in  the  Job  Corps  training  program.  The  independent  variables  in 
this  study  were  length  of  time  out  of  school,  age,  gender,  ethnicity,  and  scores  on  the  Tests  of  Adult  Basic  Education  (TABE).  The 
dependent  variables  were  training  completion,  job  placement  and  training  completion  time.  Research  questions  investigated  the 
relationship  between  the  independent  and  dependent  variables.  Data  for  all  variables  was  obtained  from  the  US  Department  of 
Labor,  and  represented  70,049  enrollees  from  July  1,  1996  through  July  I,  1997. 

Results  of  the  study  indicate  older  students  who  have  more  training  prior  to  enrolling  in  the  Job  Corps  and  those  who  score 
higher  on  the  TABE  have  higher  training  completion  and  job  placement  rates.  Success  in  these  two  outcome  variables  was 
achieved  by  over  half  of  all  enrollees.  The  strongest  finding  of  the  study  is  the  impact  of  literacy  training  on  the  outcomes  under 
investigation. 

Identification  of  learner  characteristics  is  a prominent  step  in  various  instructional  design  models  (e.g,  Dick  and  Carey, 
Reigeluth,  Jonassen  and  Hannum,  Morrison,  Kemp  & Ross).  Despite  this  step  being  emphasized,  a review  of  the  literature  in  the 
field  of  Educational  Technology  reveals  very  few  studies  in  this  area.  Recommendations  for  further  research  are  included. 

Overview 

Instructional  design  experts  have  consistently  advocated  the  investigation  of  learner  characteristics  prior  to  the  design  and 
development  of  instruction  so  that  these  characteristics  can  inform  the  design  process.  In  this  study,  the  researchers  investigated 
learner  characteristics  of  at-risk  learners  enrolled  in  a large,  federally  funded  vocational  training  program,  the  Job  Corps,  to 
determine  if  a relationship  existed  between  certain  learner  characteristics  and  student  success  in  the  program.  The  researchers 
examined  extant  data  related  to  learner  characteristics  that  is  routinely  collected  by  the  Job  Corps  when  students  enter  the 
program.  The  intent  was  to  determine  which  of  these  characteristics  were  related  to  success  in  the  Job  Corps  vocational  training 
program. 

Instructional  design  models  typically  provide  information  on  examining  learner  characteristics.  It  is  one  of  the  early  steps  in 
the  Dick  and  Carey  (2000)  instructional  design  model,  a model  widely  used  in  the  field  of  Educational  Technology.  The  authors 
provide  much  detail  related  to  gathering  and  analyzing  learner  characteristics,  and  how  these  characteristics  should  then  drive  the 
design  of  instruction.  This  need  to  have  learner  characteristics  serve  as  the  basis  for  instructional  design  is  echoed  by  Reigeluth 
(1999),  Jonassen  and  Hannon  (1995),  Morrison,  Kemp  & Ross  (2001),  as  well  as  many  other  instructional  design  experts. 

Purpose  of  the  Study 

The  purpose  of  this  study  was  to  examine  the  characteristics  of  at-risk  learners  who  participated  in  the  U.S.  Department  of 
Labor's  Job  Corps  program.  The  independent  variables  in  this  study  were  those  characteristics  the  student  had  upon  entering  the 
program  as  collected  by  the  Job  Corps.  These  variables  were:  (a)  length  of  time  out  of  school,  (b)  age  of  the  student  upon 
entering  the  program,  (c)  gender,  (d)  ethnicity,  and  (e)  entry-level  academic  skills  as  measured  by  the  Tests  of  Adult  Basic 
Education  (TABE).  The  dependent  variables  were:  (a)  whether  the  student  completed  the  training  (training  completion),  (b) 
whether  the  student  was  placed  in  a job  (job  placement),  and  (c)  the  length  of  time  students  took  to  complete  the  program 
(training  completion  time). 

Review  of  the  Literature 

The  at-risk  adult  learner  has  many  obstacles  to  success:  traumatized,  often  unstable  backgrounds,  low  literacy  levels,  and  a 
lack  of  faith  in  the  educational  system  being  able,  or  willing  to  meet  their  needs.  The  negative  impact  of  these  obstacles  is  the 
limitation  of  economic  achievement  for  this  group. 

The  workplace  qualifications  of  at-risk  adults  are  in  stark  contrast  to  the  needs  of  business.  Gone  are  the  days  of  low-level 
jobs  where  minimal  skills  were  acceptable.  The  current  and  future  needs  of  business  call  for  literate  workers  who  can  function  as 
a part  of  a team,  who  can  access  and  analyze  information,  and  who  can  function  independently  with  little  supervision. 

In  order  for  at-risk  learners  to  participate  in  the  workplace  of  the  future,  they  require  remedial  training  in  basic  skills  and 
vocational  training.  This  training  cannot  effectively  take  the  form  of  traditional  training,  for  the  traditional  style  of  training  has 
failed  this  group  in  the  past.  A different  educational  setting  is  required  that  can  accommodate  the  unique  needs  of  this  group, 
where  the  instruction  is  individualized  and  self-paced,  yet  structured  and  supportive.  The  Job  Corps  provides  this  kind  of 
opportunity. 
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Research  Questions  and  Methodology 

The  Job  Corps  was  chosen  for  this  study  because  it  targets  youths  who  are  economically  and  educationally  disadvantaged. 
Job  Corps  training  centers  offer  basic  education  leading  to  a high  school  diploma  or  GED  and  vocational  skills  training.  The 
70,049  records  used  in  this  study  represented  all  Job  Corps  enrollees  from  July  1,  1996  to  July  1,  1997.  Data  for  the  study  were 
supplied  by  the  U.S.  Department  of  Labor.  There  were  five  research  questions  that  guided  this  study: 

1.  Do  length  of  time  out  of  school,  age,  gender,  ethnicity  and  entry-level  academic  skills  as  measured  by  scores  on  the  Tests  of 
Adult  Basic  Education  (TABE)  in  English  and  math  discriminate  between  Job  Corps  students  who  complete  training  and 
Job  Corps  students  who  do  not  complete  training? 

2.  Do  length  of  time  out  of  school,  age,  gender,  ethnicity  and  entry-level  academic  skills  as  measured  by  scores  on  the  Tests  of 
Adult  Basic  Education  (TABE)  in  both  English  and  math  discriminate  between  Job  Corps  students  who  are  placed  in  jobs 
and  Job  Corps  students  who  are  not  placed  in  jobs? 

3.  Is  there  a relationship  between  length  of  time  out  of  school,  age,  gender,  ethnicity  and  entry-level  academic  skills  as 
measured  by  scores  on  the  Tests  of  Adult  Basic  Education  (TABE)  in  both  English  and  math  for  Job  Corps  student  who 
differ  on  training  completion  time? 

4.  Is  there  a relationship  between  the  Tests  of  Adult  Basic  Education  (TABE)  English  scores  and  training  completion,  job 
placement,  and  length  of  time  to  complete  training? 

5.  Is  there  a relationship  between  the  Tests  of  Adult  Basic  Education  (TABE)  math  scores  and  training  completion,  job 
placement,  and  length  of  time  to  complete  training? 

The  first  two  questions  were  investigated  using  discriminate  analysis.  Research  Questions  3,  4 and  5 were  investigated  using 
multiple  linear  regression. 

Results 

The  results  of  the  discriminate  analyses  for  Research  Questions  1 and  2 indicated  that  the  five  independent  variables  were  all 
predictive  of  students  completing  training  and  being  placed  in  jobs.  The  results  of  multiple  linear  regression  for  Research 
Questions  3,  4,  and  5 indicated  significant  relationships  between  all  of  the  independent  variables  and  training  completion  time.  In 
other  words,  significance  was  found  for  all  independent  variables  in  their  relationship  with  the  dependent  variables  in  this  study. 


Discussion  of  the  Results 

In  this  study  the  most  important  learner  characteristic  to  emergp  was  literacy  level.  Many  at-risk  individuals  are  unable  to 
secure  jobs  paying  more  than  minimum  wage  because  they  don't  have  the  literacy  levels  to  understand  business  documents. 
Business  documents  can  be  written  memos,  training  manuals,  reference  lists,  emails,  inventory  documents  and  financial 
statements.  Communication  flounders  when  literacy  skills  are  weak  or  missing.  Poor  communication  skills  negatively  impact 
success  in  the  workplace. 

The  data  indicated  that  enrollees  who  scored  600  or  more  on  the  TABE  English  or  math  tests  were  more  likely  to  complete 
training.  This  group,  functioning  at  the  2nd  grade  literacy  level,  possesses  some  foundational  skills  that  can  be  enhanced  through 
remediation.  The  results  of  this  study  also  contradict  a common  misconception  that  if  a people  cannot  read  by  the  third  grade, 
they  are  probably  unwilling  or  unable  to  learn.  As  Knowles  stated,  people  learn  when  the  training  is  relevant.  By  combining 
vocational  training  and  literacy  training,  the  Job  Corps  establishes  this  relevance. 

One  issue  regarding  high  school  diplomas  or  GEDs  and  their  relationship  to  literacy  is  necessary  to  understand  the  success  of 
the  Job  Corps  program.  The  presence  of  a diploma  or  GED  does  not  necessarily  mean  that  the  student  is  highly  literate.  At  the 
Job  Corps  all  enrollees  are  tested,  and  if  an  enrollee  has  a high  school  diploma  or  equivalency  and  still  scores  below  the  8th  grade 
5th  month  on  the  TABE,  remediation  is  prescribed.  It  might  be  worth  stating  the  8th  grade  5th  month  literacy  level  is  important 
because  that  is  the  level  at  which  most  business  correspondence  is  conducted.  Newspapers,  popular  magazines,  government 
forms,  publications,  and  other  types  of  documents  are  written  at  this  level.  Other  programs  with  which  this  researcher  is  familiar 
do  not  strive  to  move  the  literacy  level  of  program  participants  to  this  8.5  level  which  is  typically  required  to  be  successful  in  the 
workplace.  As  a result,  it  can  be  inferred  that  one  of  the  hallmarks  of  the  Job  Corps  success  is  the  fact  that  students  complete  the 
program  with  a solid  literacy  level  required  by  employers. 

The  Job  Corps  is  not  currently  training  knowledge  workers.  It  may  be  necessary  for  this  program  to  examine  what  it  would 
take  to  train  individuals  to  compete  in  the  new  economy.  Certainly,  higher  literacy  levels  and  problem  solving  skills  are 
necessary  in  order  for  Job  Corps  participants  to  gain  entry  into  the  knowledge  workforce.  This  workforce  commands  higher 
salaries  which  in  turn  contribute  to  a higher  standard  of  living.  In  the  future,  the  Job  Corps  program  may  want  to  consider  raising 
their  literacy  standards  and  provide  the  training  necessary  to  produce  knowledge  workers. 

The  Job  Corps,  through  testing  and  remediation,  stands  as  an  excellent  example  of  how  learner  characteristics  can  be 
exploited  at  the  front  end  of  a training  program  in  order  to  achieve  success  at  the  end  of  a training  program.  The  future  success  of 
the  Job  Corps  program  may  hinge  on  its  ability  to  produce  knowledge  workers  for  the  new  economy. 

Recommendations  for  Further  Research 

The  identification  of  learner  characteristics  needs  to  be  more  rigorously  addressed.  Variables  examined  in  this  study  are  those 
collected  by  the  Job  Corps  and  are  demographically  oriented.  Case  studies  night  reveal  additional  learner  characteristics  not 
discussed  in  this  study.  One  possible  method  would  be  to  select  several  Job  Corps  enrollees  with  different  entry  skills  and  literacy 
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levels  and  conduct  a series  of  comparative  case  studies.  These  could  include  thick  descriptions  of  their  experiences  in  the  training 
program  and  how  their  learner  characteristics  impact  these  experiences.  Observations,  interviews,  identifying  key  informants- 
-perhaps  instructors  or  mentors  involved  closely  with  the  subjects-as  well  as  journal  records  and  demographic  data  could  be  used 
as  data  sources  for  the  case  studies. 

One  aspect  of  particular  interest  to  this  researcher  is  the  residential  component  of  the  Job  Corps,  and  its  effect  on  training 
completion.  There  are  several  day  program  sites  in  the  Job  Corps  where  enrollees  report  every  day  to  training  as  they  would  to  a 
Job.  A study  that  might  be  informative  from  a program  design  standpoint  would  compare  a residential  program  with  a day 
program  to  see  whether  there  is  a difference  in  their  impact  on  success. 

Several  longitudinal  studies  are  suggested  by  the  results  of  this  research.  Studies  could  be  designed  to  compare  programs  that 
emphasize  literacy  training  with  programs  which  do  not  emphasize  literacy  training.  How  do  their  program  participants  fare  over 
time?  Are  there  comparable  job  placement  rates?  Are  wage  rates  comparable?  Again,  the  long-term  employment  outcomes  of 
participants  may  provide  information  that  could  then  be  used  to  enhance  program  design. 

Conclusions 

The  independent  variables  in  this  study -length  of  time  out  of  school,  age,  gender,  ethnicity  and  scores  on  the  Tests  of  Adult 
Basic  Education  in  English  and  math- were  examined  to  determine  if  a relationship  exists  between  those  characteristics  and 
training  completion,  job  placement  and  length  of  training.  Data  provided  by  the  US  Department  of  Labor  contained  records  for 
70,049  enrollees  who  participated  in  the  Job  Corps  from  July  1,  1996  through  June  30,  1997.  The  relationships  between  these 
variables  vsere  tested  using  discriminate  analysis  and  multiple  linear  regression.  Results  of  these  statistical  tests  indicate  a 
relationship  between  entry  characteristics  and  training  completion.  The  data  also  highlights  the  importance  of  combining  literacy 
training  with  vocational  training,  especially  for  those  individuals  who  have  been  classified  as  at-risk  because  of  their  low  literacy 
skills. 

The  data  in  this  study  support  the  importance  of  investigating  learner  characteristics  and  allowing  that  knowledge  to  drive  the 
program  design.  A review  of  literature  reveals  an  acute  shortfall  of  studies  that  investigate  learner  characteristics  and  their  impact 
on  program  design.  This  researcher  recommends  that  practitioners  of  instructional  design  focus  more  attention  on  this  aspect  of 
the  design  process. 
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Abstract 

The  rapid  emergence  of  e learning  in  business  and  industry  has  been  accompanied  by  a number  of  problems  when 
instructional  design  concerns  are  incorporated  into  the  overall  curriculum  development. 

This  paper  examines  two  distinct  geographical  extremes  of  Shin-Ju,  Taiwan  and  Central  Florida  to  see  if  when  comparisons 
are  made,  there  appear  any  issues  in  common  to  both  locations  and  cultures. 

Background: 

The  first  company  is  located  in  Shin-Ju,  Taiwan.  It  is  one  of  many  diverse  high  tech  manufacturing  companies  located  in 
the  Shin-Ju  Science  Center.  The  major  manufactured  products  are  computer  components.  Their  clients  range  from  distribution 
channels  in  Taiwan  and  beyond  on  a worldwide  basis.  There  are  several  thousand  employees  at  the  Shin-Ju  location,  with  the 
following  workforce  demographics: 

Males:  60%  Females:  40% 

Nationality:  100  % Taiwanese 
Average  Employee  Age:  28 

Average  number  of  years  employed  at  the  company:  1 -2  years 
Average  education  level  of  employees:  Bachelor’s  degree 

The  second  company  is  in  Lake  Mary,  Florida  with  corporate  headquarters  in  Germany. 

There  are  several  thousand  employees  located  at  the  Lake  Mary  plant  with  the  following  demographics: 

Males:  60%  Females:  40% 

Nationality:  Diverse,  i.e.  Hispanic,  American-born  Caucasian,  African-American,  Asian,  Northern  European,  etc.  with  a 
majority  being  American-born  Caucasian. 

Average  Employee  Age:  34 

Average  number  of  years  employed  at  the  company:  5-10  years 

Average  education  level  of  employees:  High  School  diploma-Two  year’s  college 

The  Florida  corporation  has  ISO  9000  Certification. 

The  Taiwanese  company  has  a top-down  management  style  that  maintains  stringent  accountability  and  performance 
standards.  Employees  have  a three-shift  daily  24-hour  production  line  quota  that  is  enforced.  If  the  production  line  fails  to  meet 
piecework  numbers  and  a random  sampling  of  quality  levels  exceeds  a specified  percentage,  the  line  is  shut  down  until  the 
problem  is  corrected. 

The  Florida  company  tolerates  a specified  percentage  of  rework  numbers  and,  should  this  number  increase  beyond  tolerances 
management  is  on  the  floor  investigating  the  cause.  The  management  structure  reflects  a “Northern  European”  work  ethic 
relative  to  stringent  cleanliness,  time  on  and  off  the  clock,  and  attention  to  detail.  It  is  also  top  down  but  participatory  in  nature. 

The  Issues  and  Statement  of  the  Problem 

In  the  Shin-Ju  company  two  basic  quality  management  tools  are  in  place  to  assist  in  the  efficient  operation  of  the  plant. 
These  tools  are:  Quality  Control  Seven  Tools  (QC  7)  which  hvolves  analysis  of  tasks  related  to  Zero  Defects,  Significant 
Milestones,  Statistical  Process  Control,  etc.  and  a Failure  Mode  Effectiveness  Analysis  (FMEA)  table  that  tracks  and  analyzes 
production  line  failure. 

The  Florida  company  bases  its  analysis  on  identified  ISO  9000  standards  for  quality  control  of  all  production  line  task  rates. 
In  both  instances,  management  mandates  the  use  of  both  analysis  tools  and  production  standards  and  the  degree  to  which  they  are 
actually  used  in  practice  varies  considerably. 

Degrees  of  variance  relate  to  the  employee’s  understanding  of  the  necessity  to  use  such  tools,  their  perception  of  value  in 
using  them,  and  the  inherent  motivation  accorded  the  employee.  Courses  on  how  to  use  the  management  tools  were 
unsuccessful.  During  formal  classes  offered  as  to  how  to  use  the  tools,  it  was  found  both  management  and  production  line 
employees  did  not  understand  them. 
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A Potential  Solution 

In  both  companies,  employee  training  as  a possible  solution  the  problem  was  addressed.  It  was  felt  that  if  sufficient  training 
could  be  provided,  the  management  tools  already  in  place  would  be  used. 

The  Taiwan  organization  has  been  provided  with  a curricular  plan  based  on  a new  instructional  design  model  devised  by  the 
lead  author  through  a series  of  formulae  and  matrices  that,  once  training  elements  are  identified,  provide  management  with  a 
specific  action  plan. 

The  Florida  organization  began  its  training  analysis  by  first  looking  at  its  workforce  demographics  and  capabilities.  A first 
step  was  the  provision  of  basic  communication  and  mathematic  skills  for  the  line-workers,  thus  the  beginning  of  ‘‘Project  Gold”. 

Issues  from  the  East 

This  project  focused  on  updating  training  needs  and  building  a web  based  learning  support  system.  A year  after  the  system 
was  adapted  in  this  manufacturing  company  a reduction  was  found  the  number  of  employees  who  participated  in  web-based 
training.  Each  employee  is  required  to  take  at  least  take  one  course  via  web  based  training  but  many  finally  quit  learning  from 
the  web. 

An  interview  and  survey  was  conducted  about  the  web  based  training  issues  from  employee  and  management.  It  was  found 
that  employees  are  willing  to  participate  in  web  training  courses,  the  Internet  access  is  good,  and  each  course  is  well  designed. 
The  connection  between  courses  was  the  problem.  For  example,  in  the  quality  control  class,  learners  did  not  have  the  requisite 
statistics  back  ground  to  successfully  complete  the  class.  When  the  employees  enrolled  in  the  quality  control  class,  there  was  no 
English  class  provided  to  assist  learners  in  reading  the  English  menu  or  to  operate  machines  that  have  English  control  panels. 

Finding  the  training  needs 

The  worker-oriented,  job-oriented,  and  cognitive  task  analyses  were  performed  as  tools  for  determining  the  required  skills, 
knowledge,  and  abilities  of  employees  to  accomplish  the  required  tasks  and  for  developing  the  trainingcurriculum. 

Worker-oriented  Task  Analysis 

The  purpose  of  the  worker-oriented  task  analysis  is  to  determine  the  knowledge  and  skills  of  employees.  The  process  of 
worker-oriented  task  analysis  includes  interviews  and  discussions  with  employees,  job  tasks  performance  observations,  review  of 
tasks  by  management,  and  surveys  to  specify  the  knowledge  and  skills  needed  for  accomplishing  tasks  (Clifford  1994). 

Job-oriented  Task  Analysis 

Job-oriented  task  analysis  is  used  to  collect  procedural  knowledge  related  to  specific  tasks  required  for  each  job.  This 
analysis  relies  on  employees  and  supervisors  who  can  clearly  state  the  job  task  sequence  step -ly -step  (Texas  Higher  Education 
Coordinating  Board  1 995). 

Cognitive  Task  Analysis 

Cognitive  task  analysis  involves  observation  and  worker  interviews  and  is  a process  used  to  gather  information  on  worker 
behavior  in  problem  solving  (Llorente  1996). 

The  second  focus  of  the  project  was  building  a web  based  learning  support  system  that  constructs  a cognitive  map  to  depict 
the  amount  of  training  needed.  The  training  offered  on  the  web  contains  approximately  2000  courses  such  as  statistical  process 
control,  quality  control  tools,  and  specialized  problem-solving  skills  and  skill  required  for  machine  operations.  The  “map”  is  able 
to  help  learners  to  obtain  direction  as  to  the  large  number  of  courses,  and  shows  both  the  pre-requisite  and  advanced  courses. 

The  core  principle  of  the  learning  support  system 

The  learning  support  system  employs  characteristics  found  in  our  Interpretive  Structure  Model.  “The  Tangible  product  of  an 
Interpretive  Structure  Model  exercise  is  a structural  model  called  a “map”  which  is,  in  general,  a multilevel  structure.  In 
condensed  form,  the  structure  is  a hierarchy”  (Warfield,  1977).  From  the  cognitive  map  learners  have  a visualized  concept  that 
shows  the  course  structure  and  the  hierarchy  of  courses,  thus  enabling  learners  to  track  their  learning  path  and  direction  without 
getting  lost.  The  following  information  illustrates  the  core  principles  that  allow  the  computer  to  automatically  develop  the 
graphic  structure  of  training  courses. 

Step  one:  Determine  the  training  needs. 

From  the  figure  1,  the  “1”  represents  the  specific  department  and  the  following  l.l,  1.2,  1.3,  1.4  numbers  represent  the  four 
assembling  lines  under  department  “1”.  Training  needs  determined  through  worker,  job,  and  cognitive  task  analyses,  are 
depicted  as  Qi,  Q2,  Qs-.-Qijk^s  shown  in  figure  1. 
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Fig.  1.  Needs  assessment 


Step  two:  Forming  the  system  matrix 

This  step  compares  every  course  or  learning  objective  with  every  other  course  or  learning  objective  in  the  matrix  to  identify 
any  subordinate  relationships.  When  completed,  a system  matrix  that  embodies  the  subordinate  relationships  between  and  among 
the  courses  and  learning  objectives  is  the  result. 
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Fig.  2.  System  matrices 


Step  three:  The  rationalizing  procedure.  The  system  matrix  is  then  transferred  to  connected  matrices. 

This  step  is  intended  to  determine  the  number  of  system  matrix  hierarchies.  The  aim  of  the  rationalizing  procedure 
produce  a set  of  matrices  called  connected  subordination  matrices,  each  of  which  represents  one  hierarchy. 


0 0 0 0 0 
1 0 0 0 0 
110  10 
110  0 0 
0 0 10  0 
0 0 0 0 0 
0 0 0 0 0 
0 0 10  1 
0 0 0 0 0 
0 0 0 0 0 
0 0 0 0 0 
0 0 0 0 0 
0 0 0 0 0 
0 0 0 0 0 
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10  10 
10  0 0 
0 0 0 1 
0 0 0 0 
0 0 0 0 
0 0 0 1 
0 0 0 0 
0 0 0 0 
0 0 0 1 
0 0 0 0 
0 0 0 1 
0 0 0 1 


0 0 0 0 0 
0 0 0 0 0 
0 0 0 0 0 
0 10  0 0 
0 0 0 0 0 
0 0 0 0 0 
0 0 10  0 
0 0 0 0 1 
10  10  0 
0 0 0 0 1 
10  0 0 1 
0 0 0 0 0 
0 0 0 0 1 
10  10  0 


Step  four:  Transfer  connected  matrices  to  reachable  matrix 


Definition  B = A-^  I 

Where  A ~ Connected  matrices 
/=  Identity  matrices 

[Boolean  factor] 

0+0=0  0x0=0 
0=1=1  0x1=0 
1=0=1  1x0=0 

1=1=1  1x1=1 

reachable  matrix 
B.B 

reachable  matrix:  T=  B' 


[Generating  matrix] 
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Step  five:  Transform  reachable  matrix  to  hierarchical  matrix 
[Definition] 

Sj.  the  number  of  the  element 
R (j,)  reachable  set 

^ (^ii  Q (si) 

[Example] 

Si  = \ 

R (sf  1 6 
Q (Sij:  1 2 3 4 5 8 
R {Sij  n Q (Si):  1 
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BEST  COPY  AVAILABLE 
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Fig.  3.  Hierarchy  Analysis 


Step  six:  Transform  the  hierarchy  matrix  to  the  Interpretive  Structure  Model 

In  this  step  the  task  is  to  produce  a cognitive  map  and  graphic  hierarchy  structure  among  courses  or  learning  objectives. 
After  this  step,  the  “map”  automatically  generates  by  computer  and  is  displayed  on  the  web  page.  Learners  will  have  a visualized 
concept  on  the  course  structure  and  the  hierarchy  of  the  course  and  be  able  to  track  their  learning  path  and  direction  without 
getting  lost. 
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Level  1 


Level  5 

Fig.  4.  Cognitive  map 


Issues  from  the  West 

The  global  corporate  climate  in  which  American  businesses  find  themselves  competing  today  requires  a high  workplace  rich 
in  skills  and  technology.  Today's  manufacturing  workforce  requires  continual  training  to  keep  abreast  of  technological  advances. 
Today's  corporations  need  flexible  people,  trained  in  many  different  functions  (specialized  training  with  cross  training). 

Technology  waits  for  no  one!  It  is  progressing  at  warp  speed,  and  workers  (especially  production  workers)  are  hard  pressed 
to  keep  up.  Multinational  manufacturing  companies  are  working  toward  gaining  or  retaining  global  standardization  of  product 
quality  (ISO  9000,QS  9000,  etc).  What  happens  to  a worker  with  low  basic  skills  and  an  inability  to  train  rapidly  enough  to  keep 
pace  with  these  changes?  They  are  lost  during  downsizing  or  merger.  It  is  usually  the  least  educated  workers  in  low-end 
manufacturing  jobs  that  become  unemployed  first  and  remain  so  for  the  longest  amount  of  time.  While  many  people  consider  the 
1990-1991  recession  to  have  been  the  first  white-collar  one,  it  hit  minorities  and  low-skilled  workers  especially  hard.  They 
ended  up  getting  clobbered  and  the  least  educated  employees  got  clobbered  once  again.  (Hilsenrath,  September  2001) 

A survey  by  the  Gallup  Organization  for  the  National  Federation  of  Independent  Business  found  that  two-thirds  of  companies 
of  the  260  polled  said  they  have  gone  without  needed  skilled  employees.  More  than  half  said  they  had  to  cut  work  hours,  limit 
production  or  turn  away  business.  Many  companies  have  told  researchers  that  the  softening  market  now  allows  them  to  be  more 
selective  in  hiring  than  they  were  last  year.  (Grimsley,  August  2001)  Most  organizations  agree  that  letting  less-productive 
workers  go  and  replacing  them  with  better-skilled  ones  during  a time  of  economic  crisis  is  an  opportunity  they  are  quick  to  use  to 
their  advantage.  Current  employees  are  increasingly  aware  of  this  situation. 

Management  Plays  a Vital  Role  In  Training 

Management  plays  a vital  role  in  for  it  sets  the  tone  when  it  comes  to  employee  training  and  professional  development. 
Management  teams  of  many  American  companies  have  already  begun  to  align  their  training  programs  with  the  company  goals 
and  to  follow  the  human  resource  management  practice  continuums.  They  are  attempting  to  progress  from  short  to  long-term 
term  focus,  from  narrow  to  broad  application,  from  a productivity  emphasis  to  quality  of  work  life  emphasis,  from  spontaneous, 
unplanned  training  to  a planned,  systematic  one,  from  individual  orientation  to  group  orientation,  from  low  participation  to  high 
participation  and  from  zero  employee  involvement  to  a higher  degree  of  involvement.  (Evans  & Lindsay,  1999,  pp.  290-3) 

In  America  companies,  motivational  focus  is  usually  on  rewards  and  recognition.  They  establish  minimums  for  promoting 
cooperation,  create  or  modify  recognition  systems,  compensation  systems,  and  mechanisms  for  broadening  employee 
responsibilities.  They  create  education  and  training  opportunities  for  employees  to  learn  and  use  skills  that  go  beyond  current  job 
assignments  through  the  redesigning  process.  They  form  relationships  with  educational  institutions  to  continue  to  develop 
employees  — thus,  ensuring  a supply  of  well-prepared  employees.  They  actively  seek  employee  involvement.  However, 
depending  on  the  type  of  management  in  the  organization,  the  degree  of  employee  involvement  varies  widely.  The  table  below 
depicts  levels  of  employee  involvement.  * 
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Level 

Action 

Primary  Outcome 

Information  sharing 

Managers  decide  - then  inform 
employees 

Conformance 

Dialogue 

Managers  get  employee  input  - 
then  decide 

Acceptance 

Special  problem 

Managers  assign  problem  to 

Contribution 

solving 

specific  employees 

Intragroup  problem 
solving 

Intact  groups  meet  to  solve  it. 

Commitment 

Intergroup  problem 

Cross-functional  groups  solve 

Cooperation 

solving 

problems 

Focused  problem 

Intact  groups  deepen  involvement 

Concentration 

solving 

in  specific  issue 

Limited  self-direction 

Teams  function  full  time  with 
Supervision 

Accountability 

Total  self-direction 

Executives  facilitate  self-manage- 
ment in  All-team  Company 

Ownership 

*Source:  Copyright@  Jack  D.  Orsburn,  Linda  Moran,  Ed  Musselwhite,  and  John  H.  Zenger,  Self-Directed  Work  Teams  (Bun- 
Ridge,  IL:  Business  One  Irwin,  1990),  p.  34.  All  rights  reserved. 

Increased  employee  involvement  results  in  empowered  employees  who  have  the  wisdom  to  know  what  to  do  and  when  to  do 
it,  possessing  both  the  right  motivation  and  tools.  This  requires  significant  change  in  work  systems  within  companies. 
Employees  must  be  provided  education,  resources,  and  encouragement  from  management.  However,  empowerment  also  means 
that  managers  must  be  willing  to  relinquish  some  power.  This  power  shift  often  creates  fears  in  management  that  workers  will 
abuse  their  privileges.  Experience  usually  shows  that  front-line  workers  generally  are  more  conservative  than  their  managers. 
(Evans  & Lindsay,  1999,  pp.  290-3) 

Experience  has  shown  that  feedback  to  management  is  vital  from  the  facilitator  or  educational  institute  providing  the 
training,.  With  computer-assisted  instruction  and  e-leaming,  this  task  is  easier  because  monthly  reports  on  employee  training 
progress  (generic  progress  - all  names  and  vital  information  are  omitted)  can  be  sent  to  management  on  a regular  basis. 

Employee  Motivation  Plays  an  Important  Role  in  Training  Programs 

Students  in  basic  skills  updating  training  programs  report  an  immediate  boost  in  self-esteem.  As  they  set  and  meet  goals,  it  is 
not  long  until  a habit  is  formed  and  transferred  to  their  own  lives  in  the  workplace  and  at  home.  Facilitators  report  comments 
such  as,  from  an  older  student,  “I  enjoy  coming  to  class  because  it  helps  me  stay  more  mentally  alert”.  Students  realize  that  they 
get  increased  respect  at  home  because  they  are  now  able  to  help  their  children  with  their  homework.  Supervisors  report  seeing  an 
increased  confidence  and  professional  development  focus  that  soon  becomes  an  incentive  to  volunteer  for  additional  training  or 
responsibilities.  Motivation  for  specific  skill  training  is  easier  to  generate  because  once  employees  comprehend  “what’s  in  it  for 
them”  they  are  excited  about  learning. 

In  order  to  motivate  employees  and  managers  of  an  organization,  trainers  or  facilitators  must  establish  a sound  professional 
relationship  with  the  company  management  and  patiently  attempt  to  spark  the  interest  of  the  workers.  Listening  and  being 
sensitive  to  their  needs  can  aid  in  this  task.  If  it  is  a manufacturing  firm,  understand  that  downtime  is  money  lost.  Being  open  to 
creative  scheduling  can  be  a big  help  - both  for  meetings  with  management  and  classes.  Motivational  flyers  and  pamphlets  can 
be  produced  and  strategically  placed  around  the  plant  and  open  houses  can  be  held  with  specific  times  set  aside  for  questions  and 
answers. 

Examples  of  Successful  Training  Programs 

In  1993,  American  production  workers  found  that  without  continual  training,  they  were  unable  to  adequately  meet  the 
challenges  of  rapidly  progressing  technology,  global  competition,  and  corporate  mergers.  Project  Gold  was  a three-year  Federal 
Workplace  Education  Grant  aimed  at  these  production  workers.  Collaborative  partnerships  arose  between  a community  college, 
a private  non-profit  organization,  and  a mid-size  telecommunications  equipment  manufacturing  plant  in  Central  Florida.  This 
project  was  only  a small  part  of  a nationwide  collaborative  effort  to  assist  companies  in  addressing  a lack  of  American  global 
competitiveness. 

This  project  was  unique  in  that  it  had  a dual  focus:  One  part  was  a computer-assisted  instructional  learning  center  to  update 
or  enhance  basic  skills.  In  this  center,  PLATO  and  I Speak  English  software  was  used:  PLATO  for  the  basic  skills,  as  well  as  for 
some  specialized  problem-solving  skills  such  as  Statistical  Process  Control  (SPC)  and  I Speak  English  for  the  English  as  a second 
language  students  (ESL).  Another  example  of  training  received  in  the  learning  center  was  GRE  preparation.  A residual  benefit 
to  Project  Gold  was  that  all  of  these  students  become  quite  skilled  at  working  on  computers.  This  was  a plus  that  the  company 
had  not  considered. 
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The  procedure  was  the  following: 

1.  Students  would  enter  and  receive  the  Test  of  Adult  Basic  Education  (TABE)  in  order  to  assess  their  basic  skill  levels. 

2.  PLATO  would  draw  up  an  individualized  education  plan  (lEP)  for  each  based  on  their  specific  needs. 

3.  They  would  then  commence  their  self-paced  training  with  regular  progress  reports  being  generated  - which  proved  to 
be  very  motivational  to  the  students,  as  well  as  to  management. 

4.  On-the-job  follow-ups  were  then  carried  out,  as  meetings  were  set  up  with  management  to  discuss  visible  results. 

The  second  focus  of  the  project  was  on  customized  training  geared  specifically  to  meet  industry  needs.  First,  a certified  job 

profiler  conducted  a job  analysis  though  interviews,  surveys,  plant  tours,  and  meetings  with  subject  matter  experts  (SME's)  and 
task  analysis.  A customized  curriculum  was  created  and  methods  and  strategies  selected,  as  well  as  securing  the  best  media, 
materials,  and  instructor.  Finally,  creative  scheduling  was  undertaken.  Given  that  timing  is  everything  in  manufacturing,  we 
worked  closely  with  management  in  designing  a training  program  that  would  not  cause  undue  scheduling  stress.  Some  examples 
of  customized  training  offered  in  this  project  were  Bright  Ideas  (creative  problem  solving),  American  Production  and  Inventory 
Control  Society  (APICS  I & II),  Statistical  Process  Control  (SPC),  teamwork.  Regular  Problem  Solving,  HAZCOM,  and  New 
Technology  Training. 

A second  collaborative  program  was  being  conducted  simultaneously  with  Project  Gold.  It  was  the  Electronic  Technology 
Advanced  Program  (ETAP)  that  involved  a partnership  with  the  manufacturing  plant,  the  public  school  system,  and  a community 
college. 

ETAP  was  a dual  system  combining  theory  with  practice  that  originated  in  Germany  a hundred  years  ago.  It  begins  with 
basic  skills  and  knowledge  and  progresses  to  a highly  technical  level.  The  ETAP  program  includes  a “Pre  apprenticeship” 
program  for  high  school  students  that  were  tied  to  the  “Tech  Prep”  initiative.  The  second  was  an  apprenticeship  program  for 
community  college  students  to  obtain  an  Associates  of  Science  Degree  in  Electronic  Engineering  Technology,  an  electronic 
industry  certification,  and  Government  Certification. 

ETAPw^ns  designed  to  search  find  qualified,  highly  motivated  high  school  students,  and  offer  them  a scholarship  and  hands- 
on  applied  training.  Then  upon  graduation,  accept  them  into  the  actual  program  where  they  would  then  be  paid  for  two  On-the- 
job-training  periods  and  receive  tuition  and  fee  reimbursement  for  a two-year  Associate  Degree  - all  of  this  with  the  ultimate  goal 
of  employment  within  this  company.  The  program  would  enable  students/employees  to  compete  using  globally  benchmarked 
world-class  standards  with  recognized  credentials,  train  in  critical  manufacturing  technologies  and  customer  service,  and  train  an 
incumbent  new  work  force,  existing  workforce,  and  instructors/mentors. 

Virtual  Universities 

Today  the  world  is  rapidly  turning  toward  electronic  learning.  Virtual  universities  are  popping  up  everywhere:  Florida, 
California,  Kentucky,  New  Jersey,  Israel,  Germany,  and  Canada  to  name  a few.  The  manufacturing  plant  where  the  two  above- 
mentioned  programs  operated  also  has  a virtual  university.  Employees  can  be  motivated  to  learn  if  they  understand  that  it  will 
possibly  increase  job  security.  It  was  proven  with  Project  Gold  and  the  ETAP  program  successfully  proved  this  concept  in  our 
high  schools  and  community  colleges.  ETAP  is  now  called  ACE-NET  involving  the  same  partnership  but  now  it  leads  to  an  AS 
degree  in  Computer  Engineering  Technology. 

This  educational  history  of  this  manufacturing  company  has  led  up  to  the  E-leaming  that  is  now  taking  place.  Workers  can 
still  attend  the  basic  skill  updating  classes  at  the  community  college.  However,  today,  this  company  has  a Virtual  University  of 
its  own.  Their  goal  during  2001  is  to  convert  at  least  60  % of  all  training  programs  to  either  Computer  Assisted  Instruction 
(CAI)  or  E Learning.  Next  year,  the  goal  is  for  75%  and  finally  100%  in  20031  Thus,  as  is  readily  apparent  — Technology 
marches  on! 

Conclusions 

We  have  examined  two  organizations,  one  in  the  east,  a second  in  the  west.  Both  faced  training  issues  that  impacted  line- 
employees  where  communication  and  work-related  skill  assessment  and  development  were  key  to  their  future  success. 
Management  issues  accompanied  the  decisions  as  to  what  training  needs  were  and  the  basis  upon  which  such  decisions  would  be 
made.  Despite  being  separated  by  oceans  and  miles,  the  problems  encountered  were  identical  while  the  proposed  solutions 
differed  only  in  their  complexity.  Both  solutions  were  designed  to  achieve  success  on  the  part  of  all,  with  quality  control  being 
the  common  element.  The  progress  made  in  the  Florida  scenario  paid  off  with  results  that  not  only  are  being  continued,  albeit  in 
another  location,  but  amplified  within  the  organization  as  well.  The  jury  remains  out  in  the  instance  of  the  Taiwanese 
manufacturing  plant  until  sufficient  time  elapses  to  test  the  model  as  proposed. 

As  has  often  been  concluded  in  the  past,  while  cultures  may  differ,  the  basic  issues  related  to  e- learning,  training  as  a whole, 
and  employee  demographics  are  often  far  more  alike  than  different.  This,  we  feel,  is  an  accurate  reflection  of  today’s  rapidly 
advancing  techno- economy. 
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Abstract 

An  experiment  was  conducted  to  investigate  the  influence  of  the  variables  'realism'  and  'context'  on  the  performance  of 
biology  students  on  a visual  test  about  the  anatomy  of  a rat.  The  instruction  was  primarily  visual  with  additional  verbal 
information  like  Latin  names  and  practical  information  about  the  learning  task:  dissecting  a rat  to  gain  insight  in  the  anatomy  of 
a mammal.  Students  were  tested  on:  (a)  recognition  of  anatomical  objects,  (b)  labeling  of  these  objects  and  (c)  relations  between 
objects.  Results  indicate  that  the  amount  of  realism  and  context  used  in  the  text  influences  test  performance  depending  on  the 
learning  tasks.  Test  results  also  show  a learning  hierarchy  in  the  different  learning  tasks  with  the  recognition  task  being  the 
easiest  and  the  relations  task  the  most  dijflcult. 

Introduction  and  the  Context  of  the  Problem 

In  several  courses  that  are  taught  in  the  faculty  of  Biology  at  the  State  University  of  Groningen  in  the  Netherlands,  it  is 
considered  necessary  that  students  learn  about  the  anatomy  of  a mammal  by  dissecting  the  body  of  a rat.  The  students  perform 
this  task  in  a two-day  practical  in  which  they  dissect  dead  animal  material  (the  rat)  layer  by  layer  to  identify  the  different 
anatomical  structures  such  as  the  different  muscles  and  organs.  This  practical  will  always  exist  in  this  way  as  the  faculty  staff 
takes  the  firm  position  that  the  students  need  the  experience  of  the  stepwise  dissection  of  a real  rat  to  gain  thorough 
understanding  of  these  anatomical  structures.  Careful  dissection  of  the  animal  enables  the  student  to  examine  every  part  of  the  rat 
in  detail  and  to  get  a grip  on  what  they  have  to  learn. 

The  problem  occurred  due  to  the  teaching  approach  in  which  the  students  had  to  report  about  their  observations  by  making 
anatomical  drawings  of  the  anatomical  structures  according  to  drawing  rules  that  were  part  of  the  instructions  of  the  course.  The 
drawings  of  the  anatomical  structures  had  to  depict  all  parts  involved  with  their  relative  proportions  and  their  interconnections, 
together  with  the  Latin  names  of  the  parts  in  a legend  in  the  margin  of  the  drawing.  After  two  days  work,  each  student  had 
produced  a set  of  drawings  of  a rat.  The  problem  was  that  a limited  number  of  staff  members  had  to  correct  the  work  of  large 
numbers  of  students.  There  are  about  150  students  yearly,  distributed  over  two  courses  for  first-year  biology  students  and  second- 
year  pharmacy  students,  who  produced  about  ten  drawings  each.  The  correction  of  these  drawings  is  so  time  consuming,  that  the 
students  sometimes  had  to  wait  for  months  before  they  ^t  feedback  on  their  performance.  This  situation  was  educationally 
undesirably.  Further  the  variety  of  drawing  skills  of  the  students  introduced  a subjective  element  in  accepting  or  rejecting  a 
drawing  as  a correct  presentation  of  a given  anatomical  structure.  Moreover,  different  members  of  the  teaching  team  put  different 
accents  while  rating  student  performance.  Uniform  evaluation  of  the  drawings  could  therefore  not  be  guaranteed.  This  situation 
was  highly  unsatisfactory.  So  the  problem  was  that  the  variety  of  the  students  drawing  skills  leads  to  a subjective  judgment. 
Besides  this  the  spread  of  the  correction  work  over  several  teachers  caused  a non-uniform  evaluation  and  last  the  students 
received  delayed  feedback. 

These  limitations  resulted  in  the  need  of  the  teaching  staff  to  look  for  other  possibilities  for  assessment.  That  need  motivated 
this  research  project.  The  identified  assessment  problem  is,  however,  more  general  than  the  problem  with  the  drawings  in  the 
particular  practical  about  the  anatomy  of  the  rat.  The  learning  outcomes  that  have  to  be  assessed  are  largely  in  the  visual  domain 
(the  anatomical  parts  and  their  relative  positions  and  interconnections),  which  led  to  the  choice  to  study  the  assessment  problem 
on  a more  general  level  in  the  field  of  visual  testing.  The  concrete  problem  of  the  practical  about  the  anatomy  of  the  rat  will  then 
be  used  as  the  test  bed  for  that  study. 

Visual  learning  - Visual  testing 

In  searching  for  alternatives  for  the  assessment  on  the  basis  of  student  drawings,  the  construct  to  be  learned  during  the 
instruction  had  to  be  redefined.  According  to  Cronbach  and  Meehl  (1955)  "a  construct  is  some  postulated  attribute  of  people  that 
is  assumed  to  be  reflected  in  test  performance.  In  test  validation  the  attribute  about  which  we  make  statements  in  interpreting  a 
test  is  a construct".  In  the  case  of  the  anatomy  of  the  rat  the  construct  to  be  learned  consists  of  (a)  the  decisive  visual 
characteristics  of  anatomical  objects  like  shape,  size,  texture  and  color  of  the  object,  (b)  the  spatial  relationships  between  the 
objects  and  the  relative  size  of  each  object  in  relation  to  other  objects,  (c)  the  verbal  labels  and  descriptions  of  the  objects,  and  (d) 
the  functional  relationships  of  objects.  This  construct  is  essentially  visual,  whereby  the  verbal  labels  and  descriptions  extend  the 
qualities  of  the  construct  into  the  verbal  domain,  allowing  for  verbal  communication  about  the  spatial  structure  and  its 
components 
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The  definition  of  the  learning  construct  resulted  in  the  definition  of  learning  tasks  for  the  instruction.  After  following  the 
course,  the  students  should  be  able  to  (a)  recognize  the  different  anatomical  structures,  (b)  label  each  structure  by  its  Latin  name, 
(c)  specify  their  spatial  orientation  and  (d)  specify  their  functional  interconnections.  The  learning  outcomes  can  subsequently  be 
assessed  by  requiring  the  students  to  demonstrate  that  they  are  able  to  fulfill  three  tasks:  (a)  the  recognition  task,  (b)  the  labeling 
task,  and  (c)  the  relations  task.  In  the  recognition  task  the  students  must  recognize  the  correct  object  according  to  its  visual 
characteristics.  In  the  labeling  tasks  the  students  has  to  identify  an  object  by  its  Latin  name.  The  relations  task  is  a task  in  which 
the  student  has  to  show  understanding  of  the  spatial  relations  between  objects  by  their  functionality. 

Although  the  instruction  is  primarily  visual,  it  is  relevant  to  know  whether  the  assessment  should  also  be  visual.  According  to 
the  stimulus  generalization  theory  (Hartman,  1961)  learning  increases  as  the  testing  mode  approximates  the  mode  in  which  the 
information  was  presented.  Dwyer  (1978)  gave  an  example  in  which  he  stated  that  instruction  presented  via  a visual  modality  but 
evaluated  in  a conventional  pencil-and-paper  assessment  would  probably  not  provide  an  accurate  representation  of  the  total 
amount  of  learning  that  has  occurred.  The  effectiveness  of  instruction  presented  to  the  students  through  a visual  channel  might 
most  appropriately  be  measured  by  employing  criterion  measures  assessing  contributions  of  the  visual  mode  of  instruction. 

As  in  all  learning  also  in  visual  learning  the  more  complex  contructs  contain  concepts  and  relationships  that  are  hierarchically 
ordered.  Smith  and  Ragan  (1999)  summarized  theories  about  learning  hierarchies  which  all  prescribe  a similar  pattern:  learners 
first  should  be  able  to  recognize  a concept  in  order  to  identify  that  concept  by  its  name.  The  acquired  concepts  then  form 
prerequisites  for  more  difficult  tasks  such  as  problem  solving.  Dwyer  (1978)  made  a distinction  in  phases  in  a learning  hierarchy 
for  visual  learning,  starting  with  facts  and  definitions  in  a content  area  that  are  familiar  to  a person.  In  this  way  the  person  is 
prepared  to  relate  and  combine  known  and  new  elements  to  form  new  concepts.  The  more  concepts  a person  possesses,  the  easier 
it  is  to  form  generalizations  and  rules.  These  processes  are  again  prerequisites  for  problem  solving.  Dwyer  (1978)  states  that:  "the 
implication  to  be  derived  from  the  concept  of  a learning  hierarchy  is  that  since  there  are  different  kinds  of  educational  objectives 
there  also  are  different  kinds  of  learning,  each  requiring  students  to  perform  different  kinds  of  activities  and  each  possessing 
unique  conditions  for  optimum  learning  to  occur".  This  assumes  that  in  designing  a visual  test,  it  is  required  to  look  more  closely 
to  the  type  of  item  format  that  will  assess  these  different  kinds  of  learning  in  a valid  way.  In  this  case,  the  recognition  task  of  the 
visual  concepts  is  a prerequisite  for  identifying  that  concept  with  the  correct  name.  Together  they  are  prerequisites  for  the 
relations  task. 

Item  types 

For  visual  testing,  Dwyer  (1978)  designed  the  PSE-test  (PSE:  Program  of  Standardized  Evaluation)  in  which  he  tested  the 
learning  effect  of  instruction  given  verbally  with  additional  visualization  versus  verbally  alone.  The  subject  of  instruction  is  the 
anatomy  of  the  heart.  The  test  consisted  of  four  parts:  (a)  a terminology  test,  (b)  an  identification  test,  (c)  a drawing  test  and  (d)  a 
comprehension  test.  The  terminology  test  measures  the  students  knowledge  of  specific  facts,  terms  and  definitions.  The 
identification  test  measures  the  students  ability  to  identify  parts  or  positions  of  an  object.  The  drawing  test  measures  the  students 
ability  to  construct  and/  a reproduce  items  in  their  appropriate  context  and  the  comprehension  test  measures  the  students 
understanding  of  the  heart,  its  parts  and  its  internal  functioning.  These  four  tests  combined  form  an  overall  test  for  measuring  the 
student’s  total  understanding  of  all  the  content  material.  The  identification  test,  terminology  test  and  comprehension  test 
consisted  of  multiple  choice  items  only.  The  test  items  for  the  comprehension  test  was  the  most  difficult  since  they  were  designed 
to  measure  the  student’s  understanding  of  complex  procedures  and  processes. 

The  use  of  multiple  choice  items  is  sometimes  being  criticized.  Martinez  (1999),  for  instance,  who  is  also  active  in  the  field 
of  visual  testing,  claims  that  multiple  choice  items  often  elicit  low-level  cognitive  processing.  He  designed  constructive  figural 
response  items  (CFR)  which  would  evoke  complex  thinking  and  therefore  be  more  appropriate  for  testing  the  student’s 
understanding  of  complex  procedures  and  processes  (Martinez,  1994).  These  items  differ  from  traditional  items  in  two  ways:  (a) 
they  require  mental  construction  of  a response,  rather  then  selection  among  options,  and  (b)  they  require  demonstration  of 
proficiency  in  a figural  medium.  Martinez  (1990)  argued  that  comparison  of  multiple  choice  items  with  their  figural  constructive 
response  counterparts  showed  that  CFR  items  were  more  difficult,  more  discriminating  and  more  reliable.  Martinez  (1999)  argues 
that  the  use  of  CFR  items  is  best  for  items  that  evoke  complex  thinking.  Other  research  outcomes  (Martinez  & Jenkins,  1993) 
were  that  CFR  items  were  better  able  to  distinguish  between  novices  and  experts.  CFR  items  are  sometimes  referred  to  as  free 
response  or  open-ended  items. 

Martinez  (1993)  and  Parshall,  Davey  and  Pashley  (2000)  recognized  the  fact  that  as  technology  improves  it  becomes  more 
useful  in  visual  testing  because  it  gives  possibilities  to  innovate  item  formats  for  visual  testing  by  using  objects  and  media  in  item 
formats.  This  may  bring  an  interactive  aspect  in  the  item,  for  instance  by  requiring  students  who  take  the  test  to  scale  object  size 
by  dragging  with  the  mouse  or  to  move  objects  to  required  positions. 

Variables  for  the  study 

The  visual  aspects  of  the  construct  to  be  learned  are  decisive  visual  characteristics  like  shape,  size,  texture  and  color  of  the 
objects.  The  representation  of  these  characteristics  in  visuals  is  affected  by  the  amount  of  realism  of  those  visuals.  'Realism'  is 
mostly  associated  with  photographic  pictures.  More  schematic  representations  such  as  line  drawings  are  regarded  to  be  less 
realistic,  although  they  may  be  more  effective  to  articulate  certain  visual  characteristics  than  realistic  pictures  do.  Dwyer  (1978) 
claims  that  the  effectiveness  of  realism  in  this  sense  depends  on  the  learning  tasks  and  the  instructional  method.  According  to 
Mandler  and  Ritchey  (1977,  in:  Anderson,  1994),  however,  people  are  better  in  remembering  the  meaning  of  a visual  then  the 
details  of  that  particular  visual.  Which  would  here  mean  that  realism  would  not  add  to  visual  testing  because  the  details  are  not 


416 


425 


important  since  people  only  remember  the  meaning.  This  uncertainty  was  reason  to  choose  'realism'  as  the  first  variable  for  this 
study. 

The  construct  to  be  learned  also  contains  the  recognition  of  objects  with  their  spatial  positions  between  other  objects  and  the 
relative  size  of  the  objects  in  relation  to  each  other.  The  availability  of  the  other  objects  appears  to  influence  the  recognition  of 
objects.  Cave  and  Kosslyn  (1993)  conducted  an  experiment  in  which  they  speak  of 'holistic  pictures'  when  objects  are  embedded 
in  a visual  context.  They  found  that  the  recognition  of  objects  in  holistic  pictures  resulted  in  higher  mean  scores  then  the  scores 
for  isolated  objects.  Research  in  the  field  of  face  recognition  (Tanaka  & Farah,  1993)  resulted  in  a similar  outcome.  Tanaka  and 
Farah  demonstrated  that  recognition  of  facial  components  was  facilitated  by  the  presence  of  the  facial  contour.  Isolated  facial 
objects  were  difficult,  in  most  cases  impossible,  to  recognize  correctly. 

The  results  of  Mandler  and  Ritchey  as  well  as  those  of  Tanaka  and  Farah  suggest  that  'visual  context'  is  a relevant  variable  for 
visual  testing.  'Context'  was  thus  selected  as  the  second  variable  for  this  study. 

Hypotheses 

The  effect  of  the  variables  'Realism'  and  'Context  is  studied  on  the  basis  of  three  hypotheses.  The  contradictory  results  of 
earlier  research  about  the  amount  of  realism  led  tot  the  first  hypothesis: 

Hl:There  will  be  no  difference  in  mean  score  between  items  with  realistic  color  pictures  and  items  with  schematic  drawings. 
Hypotheses  of  this  kind  have  mostly  been  studied  with  types  of  instruction  that  are  primarily  verbal  with  additional  visuals.  In 
our  case,  the  instruction  is  primarily  visual. 

The  variable  'Context'  is  studied  by  testing  the  second  hypothesis: 

^ H2:  Items  with  contextual  information  will  result  in  higher  mean  scores  then  items  without  contextual  information. 

Hypothesis  three  is  based  on  the  theory  of  learning  hierarchy.  The  requirement  that  concepts  of  anatomical  structures  have  to  be 
learned  as  a prerequisites  for  understanding  functional  relations,  lead  to  the  assumption  that  this  letter  task  is  more  difficult: 

H3:  Items  testing  the  relations  task  will  be  more  difficult  then  items  testing  recognition  and  labeling. 

Item  formats 

Item  formats  were  chosen  based  on  the  insights  of  Martinez  (1999)  and  Martinez  and  Jenkins  (1993)  that  recognition  could 
be  assessed  with  multiple  choice  items  whereas  items  requiring  more  complex  tasks  such  as  relating  objects  based  on  functional 
relationships  should  be  tested  with  CFR  items.  For  the  purpose  of  the  study  eventually  the  following  item  formats  were  used: 

1 . Multiple  choice  items 

2.  Labeling  items 

3.  Connect-the-dots  items 

The  multiple  choice  items  all  contained  four  alternatives  in  which  the  correct  object  had  to  be  recognized.  In  the  labeling  items 
the  examinee  had  to  identify  an  object  by  its  Latin  name  which  could  be  chosen  from  a long  list  in  an  index.  The  connect-the-dots 
items  are  examples  of  CFR  items.  With  the  connect-the-dots  items,  examinees  had  to  make  connections  between  anatomical 
objects  and  also  give  the  direction  of  these  connections  according  to  their  functionality. 


Experimental  Design 

Figure  1 gives  an  overview  of  the  2*2-experimental  design  in  which  there  were  two  independent  variables  being  the 
amount  of  realism  and  the  context  information.  For  the  variable  'Realism'  the  choice  was  made  to  look  at  the  influence  of  colorful 
realistic  pictures  (photo’s)  of  anatomical  structures  versus  the  influence  of  schematic  drawings  cf  the  same  structures.  The 
variable  'Context'  was  operationalized  by  looking  at  the  influence  of  the  presence  of  contextual  information  versus  the  absence  of 
context.  Figure  I summarizes  the  experimental  design. 


Realism 

Picture 

Drawing 

Context 

Yes 

Picture  + context 

Drawing  + context 

No 

Picture  - context 

Drawing  - context 

Figure  L Independent  variables  and  conditions 

De  dependent  variables  are  test  scores  for  each  of  the  three  learning  tasks  of  the  course  and  a total  test  score  for  all  test  items 
together: 

1 . The  mean  test  score  on  recognition  items 

2.  The  mean  test  score  on  labeling  items 

3.  The  mean  test  score  on  relation  items 

4.  The  mean  test  score  on  the  overall  test 
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Method  and  Procedure 

The  sample  population  consisted  of  134  first  year  biology  students.  The  students  followed  the  dissection  course  for  two  full 
days.  During  these  two  days  the  rat  was  dissected  in  ten  layers.  The  students  were  working  in  groups  of  thirteen  and  every  student 
got  an  own  rat  for  dissection.  Before  the  course  each  student  received  a study  guide  with  the  instructional  objectives  and  the 
dissection  steps  explained  according  to  a cued  instruction.  The  cues  that  were  used  in  the  guide  were  all  Latin  names  of  the 
anatomical  objects  that  they  had  to  learn.  The  cued  instruction  for  each  dissection  layer  followed  a certain  procedure  in  which  the 
students  as  a group  got  a live  demonstration  of  the  dissection  of  that  particular  layer  by  a teaching-assistant.  The  teaching 
assistant  used  the  Latin  names  to  identify  the  different  anatomical  objects,  show  the  important  visual  characteristics  and  explain 
the  different  functions  of  each  object  in  accordance  to  their  interconnections.  After  the  demonstration,  the  students  were  required 
to  dissect  the  current  layer  individually  and  accordingly  observe  the  anatomical  objects.  Students  had  the  freedom  to  take  notes 
during  the  observation  by  making  drawings  of  their  observations.  After  completion  of  the  individual  dissection  tasks,  the  students 
were  asked  to  join  the  group  and  have  a group  dscussion  about  the  observation  with  the  help  of  a poster  that  contained  the 
outline  drawing  of  the  dissected  layer.  Every  student  had  to  label  at  least  one  anatomical  object  with  its  Latin  name.  At  the  end  of 
the  course  on  the  second  day,  the  student  performed  a computerized  test.  The  test  contained  57  items  of  which  19  items  were 
based  on  recognition  tasks,  20  items  were  based  on  labeling  tasks  and  18  items  were  based  on  relations  tasks.  The  test  was 
specifically  developed  for  the  purposes  of  the  experiment.  Three  groups  of  questions  were  designed  to  measure  achievement  on 
the  three  different  learning  tasks:  (a)  multiple  choice  items,  (b)  labeling  items,  and  (c)  connect -the*dots  items.  For  the  picture- 
version  (the  realistic  version  of  the  variable  ’Realism')  photographs  were  taken  of  all  needed  visual  materials.  Image-processing 
software  was  used  to  produce  the  correct  alternative  and  also  the  distracters  for  the  multiple  choice  items,  for  isolating  objects  for 
the  no-context -condition,  and  to  add  a graphical  layer  with  dots  to  be  connected  for  the  connect-the-dots  items.  For  the  drawing- 
version  of  the  test  students  of  an  art  school  were  recruited  to  make  the  drawings.  The  testing  program  itself  was  programmed  in 
C++. 

Students  were  randomly  assigned  to  one  of  the  four  conditions  being:  (a)  picture  with  context,  (b)  picture  without  context,  (c) 
schematic  drawing  with  context  and,  (d)  schematic  drawings  without  context.  At  the  start  of  the  test,  each  candidate  got  an 
example  of  each  item  format  to  get  acquainted  with  the  style  of  questioning.  The  results  of  the  test  were  analyzed  quantitatively 
in  SPSS  10.0  by  comparison  of  means,  ANOVA,  Univariate  Analysis  of  Variance  and  reliability  tests. 


Results 

Table  1 gives  an  overview  of  the  results  of  the  univariate  analysis  of  variance  on  the  variables  ‘realism’  and  ‘context’.  The 
table  gives  an  general  overview  of  the  significant  differences  for  every  learning  task  and  the  overall  test. 


Table  1,  Univariate  Analysis  of  Variance  on  variables  'realism ' and  ‘context’ 

Recognition  Labeling  Relations  Overall  test 

F Significance  F Significance  F Significance  F Significance 


Realism 

11.266 

.001 

2.076 

.152 

6.727 

.011 

7.877 

.006 

Context 

38.129 

.000 

.485 

.487 

1.762 

.187 

.811 

.370 

Realism  * Context 

36.218 

.000 

.170 

.680 

.049 

.049 

.128 

.721 

The  results  are  discussed  below  by  realism,  context  and  item  difficulty. 


Realism 

Figure  2 shows  the  mean  scores  of  the  examinees  for  the  condition  realism  of  drawing  versus  colorful  pictures  on  the 
different  learning  tasks  and  on  the  overall  test.  These  mean  scores  are  on  a scale  of  0 to  10. 
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Figure  2.  Results  of  the  mean  scores  on  the  condition  realism:  picture  vj.  drawling. 

As  can  be  concluded  from  Figure  2 and  Table  1 , there  were  differences  between  drawing  (for  which  the  exact  mean  is  M= 
6.59)  and  picture  (M=  7.48)  on  the  recognition  task  in  favor  of  picture  with  a significant  difference  (p  = 0.001).  For  the  labeling 
task  there  is  a slight  difference  between  the  conditions  in  favor  of  drawing  but  this  difference  is  not  significant  (p  = .152). 
However,  the  difference  found  on  the  relations  task  between  drawing  (M=  5.98)  and  picture  (M=  5.19)  was  significant  (p  = .01 1) 
in  favor  of  drawing  and  for  the  overall  test  the  difference  between  drawing  (M=  7.10)  and  picture  (M=  6.42)  was  also  significant 
(p=  .006)  and  in  favor  of  drawing. 

Context 

Figure  3 gives  an  overview  of  the  mean  scores  for  the  presence  of  context  versus  the  absence  of  context  on  the  different 
learning  tasks  and  on  the  overall  test.  The  scale  of  the  mean  scores  is  from  0 to  10. 


Figure  3.  Results  of  mean  scores  on  the  condition  context:  with  context  vj.  without  context 

As  can  be  concluded  from  Figure  3 and  Table  1,  there  were  differences  between  the  presence  of  contextual  information  in  the 
test  where  examinees  score  higher  for  the  recognition  of  objects  (M=  7.84)  then  in  the  absence  of  contextual  information  (M= 
6.21).  This  is  the  only  significant  difference  (p  = 0.00)  that  was  found.  The  slight  differences  that  were  found  for  the  remaining 
tasks  were  not  significant  which  resulted  in  the  overall  test  also  showing  no  significant  differences. 

Realism  * Context 

An  interaction  effect  was  found  between  realism  and  context  for  picture  (M=  7.48)  and  the  existence  of  context  (M=  7.84) 
with  a level  of  significant  of  p=  0.00. 
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Item  difficulty 

Table  2 gives  an  over  view  on  test  reliability  for  Cronbach's  alpha  and  the  standard  error  of  mean. 


Table  2.  Test  reliability  with  Cronbach's  alpha  and  Standard  Error  of  Mean 


Ss 

N (items) 

Cronbach's  alpha 

Standard  Error  of  Mean 

Overall  test 

134 

57 

.8583 

.066 

Recognition  items 

134 

19 

.5873 

.097 

Labeling  items 

134 

20 

.7788 

,061 

Relations  items 

134 

18 

.7113 

,097 

Table  2 shows  that  the  overall  test  is  reliable  with  Cronbach's  alpha  being  .8583  and  an  SEM  of  ,066.  The  least  reliable  are 
the  recognition  items.  Labeling  and  relations  items  are  equally  reliable.  Figure  4 shows  the  item  difficulty  for  the  three  learning 
tasks.  The  scale  on  the  horizontal  axis  is  from  0 to  10  and  gives  an  overview  of  the  proportions  of  students  that  answered  a 
particular  item  correctly  on  the  different  learning  tasks. 

Figure  4 shows  a difference  in  item  difficulty  with  recognition  tasks  being  the  easiest  and  relations  tasks  the  most  difficult. 
Table  2 and  Figure  4 complement  each  other.  Figure  4 shows  that  low  reliability  of  recognition  items  may  be  caused  by  a ceiling 
effect.  The  recognition  items  are  obviously  rather  easy.  The  more  difficult  tasks  are  discriminating  much  better  between  learners 
who  know  and  those  who  have  learned  less.  The  distribution  of  the  items  on  the  scale  for  the  relations  tasks  shows  that  this  task 
was  the  most  difficult  and  the  most  discriminating.  As  this  task  was  (thus)  also  very  reliable,  the  results  support  the  contention  of 
Martinez  (1990)  who  argues  that  these  characteristics  are  typical  for  figural-constructive-response  items.  CFR  items  seem  a 
proper  choice  for  distinguishing  between  experts  and  novices. 


Figure  4:  Item  difficulty  for  each  learning  task 
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Discussion  and  Conclusions 

Returning  to  the  hypotheses  stated  earlier  in  the  paper  and  the  experimental  results,  some  conclusions  can  be  drawn  on  visual 
aspects  of  visual  testing. 

Hypothesis  one 

The  first  hypothesis  states  that  we  may  expect  no  differences  in  mean  scores  between  items  with  realistic  color  pictures  and 
items  with  schematic  drawings.  This  hypothesis  must  be  rejected.  We  did  find  a significant  difference  in  favor  of  picture  over 
drawings  for  the  recognition  task.  Also,  a significant  difference  was  found  in  favor  of  drawing  over  picture  for  the  relation  task 
and  the  overall  test.  These  results  suggest  possible  guidelines  which  are  that  for  constructs  that  are  primarily  visual,  it  is  best  to 
test  recognition  of  objects  with  pictures  and  to  test  lelations  between  objects  with  drawings.  The  richer  amount  of  stimuli  in 
pictures  thus  seems  to  help  recognition  of  necessary  visual  characteristics,  while  for  the  more  abstract  relations  task,  the  main 
characteristics  of  the  objects  are  sufficient.  Drawings  can  be  focused  on  these  main  characteristics.  These  findings  comply  with 
the  results  of  Mandler  and  Ritchy  (1977)  who  found  that  people  are  more  sensitive  to  remember  the  meaning  of  a visual  then  the 
details. 

Hypothesis  two 

The  second  hypothesis  expects  that  items  on  contextual  information  will  result  in  higher  mean  scores  then  items  without 
contextual  information.  This  expectation  was  only  true  for  the  recognition  of  objects.  Thus  it  seems  that  for  recognition  tasks  the 
examinees  are  helped  by  'holistic  visuals'  in  the  sense  of  Cave  and  Kosslyn  (1993)  to  succeed  in  answering  the  test  items.  For  the 
other  tasks  the  isolated  objects  gave  sufficient  information  to  complete  the  task. 

Hypothesis  three 

The  third  hypothesis  states  that  the  relations  tasks  will  be  more  difficult  then  the  recognition  and  labeling  tasks.  This 
hypothesis  cannot  be  rejected.  The  relation  tasks  appears  to  be  the  most  difficult  one.  This  finding  supports  the  findings  of  Dwyer 
(1978)  stating  that  recognition  and  labeling  are  prerequisites  for  relations.  There  is  also  the  possibility  of  Martinez  and  Jenkins 
(1993)  stating  that  CFR  items  are  better  able  to  distinguish  between  novices  and  experts  then  multiple  choice  items  which  is 
supported  in  our  results  in  which  the  items  on  the  relations  task  were  the  most  difficult  but  had  a wide  spread  in  scores  in 
contradiction  to  the  items  of  the  recognition  and  labeling  tasks. 

The  main  conclusions  of  the  experiment  are  that: 

1 . Anatomical  structures  were  better  recognized  as  picture  than  as  drawings. 

2.  Drawings  yielded  better  results  for  test  items  on  relations  between  anatomical  structures  than  pictures. 

3.  The  results  for  the  overall  test  favor  the  use  of  drawings  over  pictures. 

4.  Context  helped  to  recognize  anatomical  structures. 

5.  Scaling  of  item  difficulty  from  easy  to  difficult  shows  the  following  order:  Recognition  - Labeling  - Relations. 

As  far  as  guidelines  for  constructing  visual  tests  are  at  stake,  the  results  for  'Realism'  and  'Context"  gave  reason  for  two 
guidelines: 

1 . The  use  of  contextual  information  and  color  pictures  can  facilitate  the  recognition  of  objects. 

2.  In  order  to  assess  the  knowledge  of  relations  between  objects  based  on  spatial  relationships  or  object  functions,  the  use 
of  schematic  drawings  can  be  sufficient. 

In  regard  to  item  types,  a guideline  about  CFR  items  seems  to  be  justified: 

1 . For  visual  tests  to  distinguish  between  experts  and  novices  constructive  figural  response  items  can  be  useful. 

These  guidelines  need  further  study  to  get  a more  precise  insight  in  the  relationships  between  learning  tasks  and  visual 
aspects.  New  technological  possibilities  for  innovative  item  types,  may  extend  definitions  of  visual  constructs  in  ways  yet  to  be 
found.  The  field  of  CFR  items  is  thereby  interesting  as  new  assessment  methods  to  measure  particular  constructs  are  there  already 
technically  available.  Methods  for  scoring  these  items  are  a challenge  and  need  further  research  for  the  development  of  suitable 
scoring  algorithms.  The  current  study  was  just  a modest  step  to  set  the  stage  for  further  research  on  visual  tests. 
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Abstract 

At  the  University  of  Twente  a new  curriculum  on  educational  science  and  technology  has  been  introduced.  That  occasion  was 
used  to  try  to  develop  an  apprenticeship  model  in  which  the  students  are  regarded  as  young  professionals  from  the  very 
beginning.  In  that  model  the  students  are  expected  to  govern  their  professional  growth  by  actively  collecting  evidence  of  acquired 
competencies  in  electronic  portfolios.  This  activity  should  stimulate  teachers  to  adapt  their  teaching  style  to  the  requests  from 
students  for  feedback  on  products  that  the  students  would  like  to  put  into  their  portfolios.  After  three  iterations  of  development  in 
three  consecutive  academic  years,  however,  the  use  of  portfolios  is  still  not  successful.  The  reasons  why  are  discussed  and  steps 
to  be  taken  are  suggested. 

The  Project 

The  Faculty  of  Educational  Science  and  Technology  at  the  University  of  Twente  in  the  Netherlands  is  educating  educational 
designers  with  specializations  in  seven  directions:  curriculum  development,  instructional  technology,  instrumentation  (media, 
computers,  and  internet  in  education),  educational  organization  and  management.  Human  Resources  Development  (HRD), 
educational  testing,  and  social  science  research.  The  Faculty  intends  to  switch  to  a competency -oriented  curriculum  for  its  three- 
year  Bachelor  program. 

The  term  ’competency’  is  partly  to  be  understood  in  terms  of  the  knowledge  and  skills  that  comprise  the  competency  profile 
of  a profession  and  partly  as  the  ability  “...to  operate  in  ill-defined  and  ever-changing  environments,  to  deal  with  non-routine  and 
abstract  work  processes,  to  handle  decisions  aid  responsibilities,  to  work  in  groups,  to  understand  dynamic  systems,  and  to 
operate  within  expanding  geographical  and  time  horizons”  (Keen,  1992).  Knowledge  and  skills  can  be  assessed  in  traditional 
ways.  The  kind  of  behavior  as  described  by  Keen  is  more  difficult  to  observe.  This  at  the  same  time  causes  a problem  in  terms  of 
the  extent  to  which  competency  in  Keen’s  terms  should  dominate  the  whole  philosophy  and  organization  of  the  program.  To  get 
around  this  problem,  competency  development  was  seen  as  an  activity  that  can  take  place  in  concert  with  doing  traditional 
coursework,  as  long  as  the  students  would  be  able  to  handle  the  related  learning  experiences  from  a meta-cognitive  level  that 
complies  with  Keen’s  definition  of  competent  behavior. 

The  use  of  portfolios  offers  a possibility  to  build  evidence  of  the  development  of  competencies.  The  main  attraction  of 
portfolios  is  formed  by  its  potential  to  assess  progress  and  process  as  essentials  for  learning  (Saywer,  1994).  In  the  new 
curriculum  portfolios  are  introduced  for  self-assessment  by  the  students  as  well  as  for  monitoring  their  progress. 

Starting  in  the  academic  year  1999,  electronic  portfolios  were  introduced  to  provide  students  with  a tool  for  actively  working  on 
competency  development.  The  introduction  was  monitored  and  the  use  of  the  portfolios  was  developed  in  a few  steps  aiming  at 
answering  the  following  question: 

”Do  student  portfolio's  yield  sufficient  support  to  help  students  to  direct  their  own  development  in  a competency-oriented 
curriculum,  and  if  so,  under  what  conditions?" 

The  literature  shows  growing  consensus  that  educational  reform  efforts  are  doomed  to  fail  unless  the  teachers'  cognitions, 
including  their  beliefs,  intentions,  and  attitudes,  are  taken  into  account  (Haney,  Czemiak,  & Lumpe,  1996).  The  introduction  of 
student  portfolios  is  not  only  planned  to  serve  the  purpose  of  the  students,  but  also  as  a tool  for  the  implementation  of  the  new 
curriculum.  The  latter  relates  to  the  readiness  and  willingness  of  teachers  to  adopt  a new  approach,  for  which  the  following 
question  should  be  answered: 

"Does  student -directed  collection  of  evidence  of  professional  growth  influence  teaching  style  of  academic  staff?" 

The  educational  concept  "initiation  in  the  academic  profession" 

The  new  curriculum  approach  is  based  on  a concept  that  is  labeled:  "initiation  in  the  academic  profession’’  (Verhagen,  2000). 
It  is  an  apprenticeship  model  in  which  the  students  who  enter  the  university  directly  from  secondary  school  are  regarded  as  young 
colleagues  from  the  very  beginning.  A central  principle  of  the  concept  is  that  the  interaction  between  students  and  teachers  should 
take  place  in  a professional  context  as  much  as  possible  instead  of  interaction  in  an  instructional  context.  Assignments  should  as 
much  as  possible  mirror  professional  practice  to  help  students  to  develop  professional  behavior.  The  result  of  a literature 
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assignment  should  take  the  form  of  the  literature  part  of  a scientific  article;  communication  products  of  design  assignments  such 
as  proposals,  budget  estimates,  blueprints  or  evaluation  reports  should  take  a form  that  would  be  appropriate  to  present  to  clients; 
and  so  forth.  Teachers  should  primarily  be  regarded  as  experts  who  enable  the  development  of  academic  insights  by  scientific 
discourse. 

To  make  this  work,  a new  approach  of  mentorship  has  been  introduced  together  with  the  new  curriculum.  Each  student 
becomes  a member  of  a mentor  group  that  is  chaired  by  a staff  member  of  the  faculty.  A mentor  group  consists  of  about  1 2 
students  from  all  three  years  of  the  Bachelor  program,  about  four  students  from  each  generation.  The  mentor  groups  provide  a 
social  structure  in  which  the  master  (the  mentor/staff  member)  and  the  experienced  students  from  the  second  and  third  year  help 
the  first  year  students  (the  learners)  to  be  initiated  in  the  professional  culture  of  academic  professionals.  In  these  groups  the 
discussion  of  personal  development  towards  becoming  a competent  professional  is  a standing  issue.  The  group  members  are 
required  to  collect  evidence  of  professional  growth  that  lends  itself  for  discussion  with  the  mentor  and  in  the  mentor  groups. 

The  students  collect  the  evidence  of  their  professional  development  in  their  electronic  portfolios.  Each  portfolio  consists  of 
four  parts:  (a)  An  introduction  of  the  owner  (student),  (b)  a text  based  curriculum  vitae,  where  the  student  is  expected  to  put 
personal  information,  information  about  his  or  her  school  career,  information  concerning  professional  development  in  or  outside 
of  the  official  program  (like  having  a job  in  the  field)  and  other  information  such  as  involvement  in  sports  and  hobbies;  (c)  an 
archive,  where  the  student  puts  evidence  of  his  or  her  professional  development  during  the  study;  and  (d)  a showcase  where  the 
student  chooses  to  present  a selection  of  his  or  her  best  work.  The  archive  is  the  central  tool  for  the  student  to  perform  self- 
assessments  about  specific  accomplishments  as  well  as  to  reflect  on  personal  development  in  general.  Self-assessment  is 
considered  to  be  a form  of  metacognition  that  is  essential  for  self-regulation  (Simon  & Forgette-Giroux,  2000). 

Additional  features  of  the  new  curriculum  are  that  the  development  of  information  and  communication  skills  is  integrated 
into  courses  and  that  all  courses  are  organized  using  a Web-based  course  management  system  (TeleTOP,  a home-made  Lotus 
Domino  application). 

The  implementation  of  the  new  approach  requires  a substantial  change  of  teacher  behavior.  Most  teachers  are  used  to  teacher- 
controlled  instructional  formats.  Some  teachers,  however,  share  the  philosophy  of  the  new  curriculum  concept.  They  should  act 
as  the  pioneers  and  early  adopters  who  provide  a critical  mass  of  authentic  professional  tasks  that  allow  student  to  develop  the 
necessary  skills  for  self-regulation  of  their  academic  education.  These  tasks  and  the  interaction  in  the  mentor  groups  are  expected 
to  shape  the  attitude  and  abilities  of  the  student  into  characteristics  of  self-reliant  young  professionals.  The  extent  to  which  this 
will  appear  to  be  true  will  answer  the  first  research  question. 

Approaching  the  students  in  such  a way  that  they  perceive  the  need  to  adopt  a professional  attitude  is  expected  to  cause 
students  to  work  conscientiously  on  compiling  evidence  of  their  professional  development  in  the  form  of  portfolio  products. 
They  then  will  seek  feedback  of  the  teachers  on  their  work  and  ask  for  comments  on  the  added  value  of  assignments  for  the 
objectives  of  a course  and  in  the  framework  of  the  competency  profile  of  the  Bachelor  program  as  a whole.  It  is  expected  that  this 
behavior  will  influence  the  teaching  style  of  the  academic  staff  and  will  help  teachers  who  are  reluctant  to  invest  in  the  new 
educational  concept  to  move  in  the  desired  direction.  The  extent  to  which  this  effect  occurs,  will  answer  the  second  research 
question. 

The  first  experiences 

Preparing  for  the  academic  year  1999-2000 

The  principles  of  the  new  approach  have  been  presented  to  the  teaching  staff  on  several  occasions,  at  first  to  estimate  whether 
the  concept  was  appealing  to  them.  The  overall  impression  was  sufficiently  positive  to  start  the  preparation  of  the  introduction  of 
the  new  approach  in  the  program.  A few  months  before  the  academic  year  1999-2000,  a group  of  student -friendly  staff  members 
was  invited  as  mentors.  Together  with  the  faculty  management  they  developed  procedures  and  a related  manual  to  start  the  new 
mentor  groups.  Involving  the  mentors  at  this  stage  resulted  in  their  ownership  of  the  concept  for  the  new  mentor  groups  and  the 
way  of  working  in  those  groups.  Elements  of  the  approach  are  that  competency  development  was  related  to  three  major  roles  of 
professionals  in  our  field:  designer,  researcher,  and  consultant;  and  that  explicit  attention  should  be  paid  to  generic  competencies 
such  as  planning,  self  management,  interpersonal  skills,  communication,  and  academic  reflection.  In  respect  to  the  individual 
development  of  the  students,  a list  of  products  that  should  be  collected  in  the  portfolios  was  specified.  They  concern  results  from 
assignments  in  courses  that  may  be  considered  as  evidence  of  acquired  knowledge  and  skills,  thus  contributing  to  the  competency 
profile  of  the  student.  Monthly  professional  meetings  of  the  mentor  groups  were  planned  to  discuss  progress.  During  these 
meetings  also  attention  was  paid  to  the  quality  of  the  program  as  experienced  by  the  students.  It  was  expected  that  the  students  in 
their  role  as  beginning  educational  designer  should  be  interested  in  strengths  and  weaknesses  of  the  courses  in  which  they 
participate.  The  input  from  the  mentor  groups  was  also  considered  as  valuable  for  the  formal  evaluation  of  running  courses.  The 
teaching  staff  was  informed  about  the  intended  approach  and  invited  to  work  accordingly. 

Outside  the  mentor  groups,  information  and  communication  specialists  developed  their  curricula  in  close  cooperation  with 
teachers  from  selected  courses  to  arrive  at  the  integration  of  relevant  tasks  and  assignments  in  the  different  courses. 

Results  from  the  academic  year  1999-2000 

The  mentor  groups  appeared  to  be  handicapped  by  the  fact  that  it  was  the  first  year  and  thus  only  first-year  students  were 
members.  The  monthly  meetings  failed  also  because  the  students  had  such  a close  contact  with  each  other  throughout  the  week, 
that  no  substance  to  discuss  remained  for  the  meetings.  Course  evaluations  became  a formal  ritual  with  no  real  impact. 
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Teachers  and  students  appeared  to  behave  more  traditional  than  expected,  leading  to  much  interaction  in  an  instructional 
context  and  little  in  a professional  context.  The  instructivistic  teaching  style  in  many  courses  appeared  a dominant  factor  in 
shaping  student  behavior.  Instead  of  working  on  a professional  attitude  that  complies  with  the  model  of  the  students  as  young 
colleagues,  the  students  felt  that  they  went  to  school  to  take  lessons  and  make  tests. 

Moreover,  due  to  technical  problems  portfolio  software  was  introduced  to  the  students  at  a late  stage  (the  end  of  the  first 
semester).  Students  had  then  to  go  back  to  already  completed  courses  to  find  the  required  products  for  their  portfolios.  It  was 
unlucky  that  for  unclear  reasons  they  appeared  not  to  be  informed  about  the  list  of  required  products  that  existed  from  the 
beginning  of  the  academic  year.  And  when  they  learned  about  the  list,  several  students  became  annoyed  because  they  consider 
that  list  as  contradictory  to  the  concept.  If  the  portfolios  are  tools  for  governing  one’s  own  learning  process,  they  should  be  able 
to  decide  by  themselves  what  to  put  into  it.  At  that  time,  so  much  was  unclear,  that  most  students  failed  to  work  with  the 
portfolios  in  a proper  way.  In  conclusion,  the  electronic  portfolios  were  hardly  used. 

The  only  thing  that  really  worked  was  the  integration  of  information  and  communication  skills  into  courses.  The  carefully 
developed  set  of  tasks  on  information  and  communication  skills  made  the  students  acquire  the  related  skills  every  time  they 
needed  them  for  the  assignments  in  the  courses.  A literature  assignment  in  a course  on  pedagogy  was  used  to  teach  them  how  to 
find  literature,  an  assignment  to  write  a paper  for  an  other  course  was  used  to  explicitly  pay  attention  to  writing  skills,  and  so  on. 

2000-2001:  Some  changes 

The  insight  was  developed  that  students  should  not  be  forced  to  put  products  in  a portfolio,  because  this  is  contradictory  to 
self-management.  The  students,  who  were  interested  in  the  desired  approach,  told  that  to  us  more  than  once  and  they  were  right  at 
this  point.  The  mentors  were  asked  to  guide  the  students  in  developing  self-management  skills,  using  the  electronic  portfolio  as  a 
discussion  platform. 

Now  that  students  of  two  study  years  were  member  of  the  mentor  groups,  activities  were  specified  that  could  bring  the 
concept  to  life.  The  elderly  students  could  now  introduce  the  new  students  to  all  kinds  of  procedures  and  habits  in  the  faculty. 
And  group  discussion  could  now  aim  at  points  of  interest  for  which  the  vision  of  both  the  first  year  and  the  second-year  students 
was  relevant.  The  number  of  official  meetings,  however,  was  reduced  to  seven.  This  measure  was  taken  to  avoid  the  problem  of 
too  few  subjects  for  discussion  that  came  with  the  monthly  meetings  in  the  first  year. 

Changing  the  teaching  style  of  teachers  towards  competence  development  seemed  not  to  be  possible  directly.  So  a major  role 
for  the  mentor  was  envisioned  given  the  character  of  the  guild  model  in  the  mentor  groups.  The  mentors  were  asked  to  work  with 
the  students  on  helping  them  to  use  their  portfolios  for  self-governance. 

Results  from  the  academic  year  2000-2001 

This  time  the  mentors  started  to  resist  to  the  idea  that  they  should  work  with  the  students  in  such  a way  that  the  students 
would  develop  the  metacognitive  skills  to  monitor  their  own  professional  growth  with  the  electronic  portfolio’s  as  the  basic  tool. 
They  argued  that  the  curriculum  and  the  way  in  which  the  courses  are  taught,  should  have  this  effect. 

Again,  the  portfolios  were  hardly  used  although  the  software  was  now  available  almost  from  the  beginning.  But  the  early 
start  had  also  a disadvantage.  The  students  were  introduced  to  the  portfolio  software  in  a workshop  where  to  technical  skills  were 
practiced  without  real  products  to  put  into  the  portfolio.  The  first  products  that  would  be  suitable  had  still  to  be  produced  in  the 
courses  that  just  were  started.  By  the  time  that  the  portfolios  could  be  used,  most  students  had  forgotten  how  to  do  that.  As  there 
was  this  time  no  list  of  required  products,  only  few  students  appeared  to  motivated  to  start  filling  their  portfolios.  Many  other 
students,  however,  appeared  not  to  be  able  to  decide  what  they  could  put  into  the  portfolio.  Partly  the  reason  is  that  they  appeared 
to  be  very  critical  of  their  first-year  products,  considering  their  own  work  as  real  beginners  work  that  is  not  worth  to  be  put  into  a 
portfolio. 

The  cooperation  between  students  from  different  years  appeared  to  be  one-sided.  Only  in  the  beginning  of  the  academic  year 
were  the  older  students  active  when  introducing  the  new  students.  Subjects  of  mutual  interest  to  old  and  new  students  were  not 
identified.  The  new  students  could  also  not  bring  anything  of  relevance  for  the  older  student.  The  question:  “ What  are  my 
benefits?”  was  hard  to  answer  for  the  older  students. 

Still,  the  smdents  were  positive  about  the  meetings  of  the  mentor  groups.  During  the  meetings  the  students  discussed  general 
information  with  their  mentor  and  with  each  other  and  they  used  the  meetings  as  a platform  to  complain  about  organizational  or 
educational  problems  in  the  faculty.  They  could  speak  freely  about  anything,  which  gave  the  meetings  an  “I  am  not  alone  with 
this”  function. 

Putting  students  in  control 

2001-2002:  A last  chance  for  portfolio’s? 

Gradually  it  becomes  clear  that  the  basic  philosophy  of  the  concept  “Initiation  in  the  Academic  Profession”  does  not  really 
settle  in  the  faculty.  The  belief  that  the  concept  is  worthwhile  is  reason  for  a third  attempt.  This  is  where  the  approach  was 
developed  that  is  the  reason  for  this  paper.  Again  the  idea  is  that  the  students  should  develop  initiative  in  using  their  portfolios  as 
a tool  for  collecting  evidence  of  their  professional  growth.  In  the  first  week  of  the  academic  year,  they  were  trained  to  use  their 
portfolio’s  in  two  ways:  technically  to  learn  how  to  put  elements  in  the  portfolio  and  how  the  manage  the  portfolio;  and 
conceptually  on  how  to  use  the  portfolio  for  monitoring  and  managing  professional  progress.  An  adapted  guide  for  the  mentor 
groups  explained  the  purpose  and  the  philosophy:  It  was  recognized  that  not  all  students  are  ready  for  this  kind  of  metacognitive 
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activity.  They  are  Aerefore  allowed  to  use  or  not  use  their  portfolios  for  self-management.  Filling  the  archive  with  products  of 
courses,  however,  is  this  time  required  to  maintain  basic  portfolio  skills  until  the  moment  that  the  student  is  ready  and  willing  for 
the  intended  use. 

The  results  so  far 

The  number  of  students  that  works  with  the  portfolios  in  the  intended  way  is  negligible.  There  are  just  a few  students  who 
work  with  their  portfolios.  These  students  are  mostly  using  the  archive  function  just  for  their  own  purposes  and  not  to  reflect  on 
what  they  have  done  in  past  periods.  Also  most  mentors  still  don’t  use  the  portfolios  for  the  individual  meetings  with  the  student. 
Some  of  them  do,  but  they  leave  no  room  for  self-directedness  by  the  students  because  they  require  the  students  to  fill  the 
archive. 

In  the  meantime,  the  discussion  about  the  usefulness  of  portfolios  has  become  an  issue  in  a broader  perspective.  Students  who 
are  following  the  old  curriculum  (from  before  the  introduction  of  the  new  educational  concept)  start  asking  for  their  own 
portfolio’s  for  making  overviews  of  products  that  they  collect  in  courses  during  their  study.  So  these  oldest  students  see  the 
purpose  and  the  advantages  of  portfolios  from  a need  for  systematically  archiving  products.  Regrettably,  however,  when 
providing  them  with  the  portfolio  software,  they  don’t  find  the  time  to  really  do  it.  It  seems  a similar  phenomenon  as  with  the 
staff  members  who  like  the  idea  of  the  new  educational  concept,  but  do  not  really  change  their  methods  to  comply  with  it. 

When  student  and  staff  members  are  asked  what  they  think  about  the  portfolio  idea,  they  are  almost  all  very  positive.  But  still,  it 
did  not  work  out.  The  project  fails  and  we  have  to  find  out  what  we  may  learn  from  it. 

Discussion 

Why  portfolio’s  can  be  a success 

When  looking  for  a field  where  portfolios  do  work,  the  field  of  Human  Recourse  Development  (HRD)  is  an  obvious  one. 
Self-responsible  adults,  who  have  a job  and  related  responsibilities,  benefit  from  individual  learning  arrangements  that  are 
reported  by  collecting  evidence  of  achievements  in  a portfolio.  The  use  of  (electronic)  portfolios  is  in  that  context  appropriate 
because  the  learner  has  sufficient  metacognitive  (and  computer)  skills  to  use  the  portfolio  tool  properly.  The  educating  agency 
(training  department,  external  course  provider,  etc.)  shares  the  philosophy  of  competency -based  education  and  is  therefore  open 
to  assessment  on  the  basis  of  individual  portfolios.  This  context  is  essentially  different  from  the  university  situation  where  young 
students  who  enter  the  university  directly  from  secondary  school,  do  not  have  the  maturity,  the  experience  and  the  interest  to 
work  along  these  lines.  This  is  the  starting  point  for  discussing  why  portfolios  may  fail. 

Why  portfolio’s  fail 

Portfolio’s  fail  when  the  students  don’t  see  the  value.  Portfolio  proponents  tend  to  deny  the  psychological  developmental 
stage  of  the  students.  But  many  (young)  students  are  not  prepared  or  willing  to  look  at  themselves  in  the  metacognitive  way  that 
IS  required  for  proper  dealing  with  portfolios.  Further  it  seems  that  the  spontaneous  fun  in  studying  theory  is  hampered  by  precise 
questions  about  the  requirements  that  have  to  be  fulfilled  for  a competency-based  curriculum.  A student,  who  is  really  involved 
m a subject,  has  to  make  a severe  mental  switch  when  he  or  she  has  to  step  outside  that  subject  to  analyze  on  a metacognitive 
level  whether  what  he  or  she  is  doing  is  a contribution  to  the  development  of  competence. 

And  even  when  students  see  some  value,  for  instance  for  building  a comprehensive  archive  of  their  work  during  their  study, 
they  may  misinterpret  the  function  of  the  archive  by  denying  products  of  which  they  are  not  very  proud,  like  the  first-year 
products  when  they  feel  themselves  still  beginners.  To  reflect  on  professional  growth,  however,  these  products  are  needed  for 
reasons  of  comparison  with  later  accomplishments.  Proper  guidance  of  the  students  by  mentors  could  help,  but  this  requires  that 
the  mentors  are  convinced  of  the  value  of  portfolios  as  a tool  for  monitoring  progress.  In  our  case,  we  were  clearly  not  able  to 
motivate  the  mentors  in  this  sense. 

And  there  is  also  a very  practical  reason  why  portfolios  may  fail.  That  is  when  the  software  causes  problems.  The  system 
may  need  to  many  steps  for  simple  tasks,  the  server  may  too  often  be  too  busy,  and  so  on.  In  our  case,  several  technical 
limitations  did  for  certain  not  stimulate  the  use  of  the  portfolios. 

Portfolios  also  fail  when  the  teachers  fail  to  adjust  their  teaching  accordingly.  Teachers  are  mainly  prepared  to  carry  out  a 
well-defined  course.  When  they  have  to  step  beyond  the  concrete  patterns  to  adjust  themselves  to  individual  trajectories,  many 
teachers  fail  to  comply  with  that  fact.  The  idea  that  a concept  such  as  “initiation  in  the  academic  profession”  can  be  put  into 
practice  outside  the  courses  by  regarding  the  traditional  courses  as  occasions  for  gathering  portfolio  products  that  are  used  by  the 
students  and  mentors,  does  not  work.  The  concept  and  the  use  of  the  portfolios  have  to  be  operationalized  within  the  courses. 


Where  to  go  from  here? 

Who  wants  to  succeed  in  an  effective  learning  process,  ought  to  be  able  to  coordinate  his  own  learning  process  (McCombs, 
1988).  In  order  to  make  the  concept  “initiation  in  the  academic  profession”  successful,  we  will  have  to  arrange  a situation  in 
which  the  students  will  be  helped  to  get  ready  to  do  so.  And  this  has  primarily  to  happen  in  courses,  while  the  new  mentor  groups 
may  have  a support  function.  The  developmental  readiness  of  the  students  has  to  be  taken  into  account.  Alexander  (1995),  for 
example,  menUons  three  stages  for  the  evolution  for  the  learner.  In  the  habituated  stage,  the  student  has  a diminutive  knowledge 
level.  Having  just  a little  domain-specific  knowledge  the  student  appeals  on  common  strategies.  The  second  stage  is  the  ability  or 
competency  level.  The  students  get  more  comprehensive  and  coherent  knowledge  of  the  subject  and  there  is  a change  the  student 


426 


435 


will  select  the  correct  strategy  for  the  specific  situation.  The  third  stage  is  the  expert  level.  At  this  stage  the  students  have  ample 
knowledge  of  domain  specifics,  are  ready  to  regulate  themselves,  and  are  able  to  add  new  knowledge  to  the  domain.  These  stages 
ask  for  a curriculum  line  in  which  each  stage  has  a logical  place.  This  leads  in  our  case  to  a choice  for  courses  in  which  the  three 
stages  have  a natural  place:  the  series  of  courses  about  design  methodology  that  runs  form  the  first  to  the  third  year.  This  choice 
is  inspired  by  the  Design  Studio  as  it  works  at  the  Master’s  level  at  the  University  of  Georgia  at  Athens  (Rieber,  2001).  The 
principle  as  it  will  be  tried  in  our  program  is  that  third-year  students  will  take  responsibility  for  design  assignments  while  second- 
year  students  will  act  as  helpers  for  specific  tasks  that  need  already  proper  workmanship  (like  carrying  out  a literature  study  or  an 
evaluation),  and  first  year  students  will  be  used  for  very  concrete  tasks  for  which  it  is  not  necessary  to  be  very  knowledgeable 
about  the  specific  domain.  In  this  stream,  the  principles  of  the  concept  “initiation  in  the  academic  profession”  may  be  fully 
exploited,  together  with  the  use  of  portfolios.  Next  to  and  in  balance  with  this  stream,  theory  courses  may  still  be  taught  in  more 
traditional  ways  as  long  as  all  teaching  complies  with  the  seven  principles  for  good  practice  in  undergraduate  education  as  listed 
by  Chickering  en  Gamson  (1987,  quoted  by  Chickering  en  Ehermann,  http://www.aahe.org/technology/ehrmann.htm):  (a)  good 
practice  encourages  contacts  between  students  and  faculty,  (b)  good  practice  develops  reciprocity  and  cooperation  among 
students,  (c)  good  practice  uses  active  learning  techniques,  (d)  good  practice  gives  prompt  feedback,  (e)  good  practice 
emphasizes  time  on  task,  (f)  good  practice  communicates  high  expectations,  and  (g)  good  practice  respects  diverse  talents  and 
ways  of  learning. 

After  the  lessons  learned  with  our  attempts  to  introduce  electronic  portfolio’s  faculty  wide,  we  hope  that  the  more  modest 
approach  for  introducing  portfolio’s  and  competency -based  learning  in  the  design  stream  of  our  program,  will  appear  to  be  the 
right  step  to  help  to  initiate  our  students  into  the  academic  profession. 
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Video  Outside  Versus  Video  Inside  the  Web:  Do  Media  Setting  and  Image 
Size  Have  an  Impact  On  the  Emotion-Evoking  Potential  of  Video? 
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Abstract 

To  explore  the  educational  potential  of  video- evoked  affective  responses  in  a Web-based  environment,  the  question  was 
raised  whether  video  in  a Web-based  environment  is  experienced  differently  from  video  in  a traditional  context.  An  experiment 
was  conducted  that  studied  the  affect- evoking  power  of  video  segments  in  a window  on  a computer  screen  compared  to 
presenting  the  same  video  materials  on  a television  monitor.  The  study  first  demonstrates  that  affect-evoking  power  of  video 
exists.  Although  not  always  significant,  the  results  show  that  selected  positive  and  negative  video  clips  induce  positive  or  negative 
affective  responses  and  mood  changes  in  participants,  in  a Web-environment  as  well  as  in  a television  environment.  Smaller 
window  sizes,  however,  may  do  less  well. 

Introduction 

Technical  developments  make  it  continually  easier  to  use  streaming  as  well  as  downloadable  video  in  Web-based 
applications.  This  raises  growing  interest  in  the  use  of  video  segments  within  Web-based  multimedia  learning  environments  (see 
for  example  Collis  & Peters,  2000).  Video  has  the  potential  to  present  dynamic  content.  In  the  cognitive  domain  moving  images 
with  sound  and  accompanying  voice-overs  may,  for  instance,  provide  insight  in  technical  processes  and  procedures.  In  the 
affective  domain  video  materials  may  evoke  affective  responses  in  the  learner  that  relate  to  the  instructional  content  of  interest.  A 
well-known  example  is  the  use  of  trigger  video’s  that  present  emotional  demanding  situations  as  input  for  learning  experiences. 
These  examples  represent  two  characteristic  functions  of  video:  (a)  the  potential  to  facilitate  the  acquisition  of  knowledge  and 
skills,  and  (b)  the  use  of  video  to  bring  material  into  the  learning  environment  for  affective  learning  goals.  The  present  study 
relates  to  this  second  function  of  video. 

Affective  video  materials  can  show  realistic  situations  and  people  depicted  directly  from  reality  or  dramatised  in  story  lines. 
Footage  from  real  situations  as  well  as  dramatised  story  lines  make  it  possible  for  viewers  to  identify  themselves  with  persons  on 
the  screen  and  to  become  involved  in  the  situations  that  those  persons  meet.  For  educational  purposes  this  potential  of  video  is 
used  to  evoke  emotions  and  to  stimulate  discussion.  Topics  to  be  discussed  may  vary  from  historical  and  societal  to  ethical  issues. 
Not  only  the  content  of  the  video  is  important  for  evoking  affective  responses,  but  there  is  also  a contribution  from  production 
variables  such  as  music,  editing,  camera  handling  and  lighting  (resulting  in  a "video  structure").  Video  content  and  video 
structure  are  not  independent  but  interacting  variables.  Video-production  techniques  and  their  influence  on  the  way  video  content 
is  experienced  are  discussed  in  for  example  Wetzel,  Ratke  and  Stem  (1994)  who  approach  video  design  from  a research 
perspective,  and  Millerson  (1999)  and  Zettl  (1999)  who  draw  upon  the  trade  of  film  and  video  makers.  The  potential  of  video  to 
evoke  emotions  and  the  relationship  between  video  structure  and  video  content  is  also  a topic  in  communication  research  (for 
example:  Geiger  and  Reeves,  1991;  Lang,  Dhillon,  & Dong,  1995;  Lang,  NewHagen  & Reeves,  1996). 

Affective  video  materials  may  further  have  educational  potential  when  combined  with  specific  (creative)  tasks  that  follow  a 
segment.  Research  studies  by  Isen  (for  example  Isen,  Daubman,  & Nowicki,  1987)  and  Kaufmann  and  Vosburg  (for  exarnple 
Kaufmann  and  Vosburg,  1997,  Study  2)  show  that  video  segments  used  to  evoke  emotions  can  influence  the  performance  on  a 
task  that  follows  the  video  segment.  The  content  of  the  video  segments  that  they  used  was  not  related  to  the  task,  but  served 
merely  to  induce  an  affective  response  or  mood  in  the  subjects.  Their  work  shows,  however,  that  video  may  be  considered  as  an 
effective  emotion-evoking  or  "mood- induction-technique”,  a finding  which  is  confirmed  in  a review  by  Gerrards-Hesse,  St>ies, 
and  Hesse  (1994). 

The  tasks  involved  in  the  research  of  Isen  and  Kaufmann  and  Vosburg  are  (creative)  problem-solving  tasks,  varying  from 
categorization,  association,  developing  insight,  to  divergent  thinking  tasks.  Positive  and  negative  mood  states  seem  to  be  related 
to  the  way  a problem-solving  task  is  carried  out.  Their  results  suggests  that  a positive-affect  video  could  be  more  effective  for  a 
certain  type  of  task  and  a negative-affect  video  for  another  type  of  task.  For  a recent  model  about  the  influence  of  mood  on 
problem  solving,  see  for  example  Vosburg  and  Kaufmann  (1999). 

Affective  video  materials  implemented  in  a Web-based  learning  environment  are  expected  to  be  potentially  similarly 
effective  for  reaching  specific  pre-defined  affective  goals  and  for  having  a mood- inducing  function  that  stimulates  the  solving  of 
specific  problems.  As  an  additional  effect  affective  video  might  also  contribute  a sense  of  “warmth”  to  a technological  system 
like  a Web-based  environment  which  may  help  to  maintain  interest  in  the  learning  task. 
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This  study  is  a step  in  exploring  these  expectations  by  first  trying  to  confirm  that  video  has  indeed  the  potential  of  evoking 
affective  responses  and  mood  changes  in  subjects  in  a traditional  setting  for  watching  television.  Subsequently  we  will  examine 
whether  video  materials  presented  in  a Web-like  environment  have  a similar  effect. 

Video  materials  in  a Web-based  learning  environment  are  presented  in  a video  window  that  is  smaller  than  video  shown  on  a 
television  screen.  Therefore  a second  question  is  raised  whether  image  size  is  a relevant  design  factor  that  influences  the  affective 
potential  of  video  materials  in  a Web-environment.  Research  on  image  size  of  emotion-evoking  movie  fragments  shows  that 
screen  size  does  indeed  have  an  effect  on  how  the  movie  is  experienced.  Little  screens  are  less  liked,  produce  less  arousal  and  the 
content  is  remembered  less  compared  to  big  screens  (Reeves  & Nass,  1996;  Reeves,  Lang,  Kim  & Tatar,  1999).  Video  materials 
presented  in  a little  video  window  within  a Web-environment  may  thus  have  less  affective  potential  than  the  same  videos 
presented  in  a bigger  video  window. 

From  an  instructors  point  of  view,  who  may  be  considering  a shift  from  traditional  classroom-use  of  video  to  using  video 
within  a Web-environment,  it  may  be  relevant  to  know  whether  differences  exist  between  the  two  types  of  media  settings 
(watching  television  vs.  watching  video  in  a window  on  a computer  screen).  This  will  be  the  third  question  of  our  research. 

In  summary,  the  following  three  research  questions  are  stated  to  explore  the  affective  potential  of  video  materials  in  a Web- 
based  environment: 

1.  Are  video  materials  that  are  selected  for  their  potential  to  evoke  either  positive  or  negative  affective  responses  and 
mood  changes  in  participants,  effective  when  the  video  is  presented: 

a) within  a traditional  television-viewing  setting  (group  or  individual  setting),  or 

b) within  a Web-like  environment  (individual  setting). 

If  this  is  the  case: 

2.  Does  image  size  have  an  effect  on  the  magnitude  of  affective  responses  and  mood  changes  in  participants  (within  Web 
setting;  within  television  setting). 

3.  Does  the  media  setting  have  an  effect  on  the  magnitude  of  affective  responses  and  mood  changes  in  participants 
(between  television  and  Web)? 


Method 

To  study  these  questions,  an  experiment  was  carried  out  that  is  specified  below. 

Participants 

The  participants  for  the  experiment  were  1 17  first-year  university  students  in  communication  studies  (82  women  and  35  men, 
mean  age  = 18.5  years).  The  experiment  was  part  of  a course  that  introduces  the  field  of  media  communication  and  media 
research  and  was  introduced  as  an  introduction  to  doing  experiments.  The  results  of  the  experiment  and  the  experience  of  the 
students  in  participating  in  the  experiment  were  discussed  in  class  a month  after  the  experiment. 

Materials 

For  the  selection  of  affect-evoking  video  materials,  video  segments  of  about  two  or  three  minutes  were  used  from  existing 
films  or  television  programs.  In  a pilot  study  situated  in  a traditional  classroom  setting,  six  video  clips  were  tested  with  students 
for  the  affective  responses  they  evoked.  Some  of  the  clips  showed  positive  content  (content  that  is  expected  to  evoke  positive 
feelings),  other  clips  showed  negative  content.  The  pilot  showed  that  two  of  the  positive  video  clips  and  two  of  the  negative  video 
clips  were  effective.  For  both  affect  types  the  briefest  clips  where  selected  for  the  experiment.  Selecting  the  short  clips  seems 
justified  by  the  fact  that  Web  users  are  used  to  brief  presentations.  The  shorter  clips  are  thus  more  realistic  to  embed  in  a Web- 
environment  than  longer  clips  (5  minutes  or  longer).  In  the  experimental  setting  the  positive  video  content  consisted  of  a segment 
from  the  comedy  movie  ‘When  Harry  met  Sally*  (2’45”);  the  negative  video  content  was  a news-item  about  ‘Hunger  in  Ethiopia’ 
(2’04”).  Half  of  the  subjects  were  shown  first  the  negative  content  and  then  the  positive  content.  For  the  other  half  this  was  the 
other  way  around.  To  ‘buffer’  the  impact  from  one  type  of  affective  content  to  the  other  a video  segment  was  presented  in- 
between  (a  segment  from  an  instruction  video  on  welding,  2’27”).  To  establish  a comparable  starting  position  for  all  subjects  a 
video  segment  was  added  as  a first  video  clip  for  all  participants.  This  segment  was  a part  of  a documentary  about  birds  (‘  1 ’53”). 

Design  and  procedure 

An  experimental  factorial  pretest-posttest  design  with  three  factors  was  chosen.  Factor  one  is  the  ‘affective  content  of  the 
video’  (positive  versus  negative).  Factor  two  represents  the  ‘playback  size  of  the  video’  (little  versus  big;  the  bigger  size  is  linear 
two  times  the  smaller  size,  resulting  in  an  image  area  that  is  four  times  the  smaller  size).  Factor  three  is  ‘media  type’  (television 
monitor  versus  computer  monitor).  All  three  factors  were  studied  with  settings  for  individual  participants.  In  addition  to  the 
individual  settings  a group  setting  was  introduced  as  an  extra  factor  to  explore  social  effects  that  may  influence  the  magnitude  of 
affective  responses  to  video. 

Subjects  were  randomly  assigned  to  one  media  type  setting  and  within  this  setting  to  one  video  playback  size.  A matching 
procedure  was  used  for  the  variable  male/female.  Subjects  received  in  advance  information  about  the  location  of  the  experiment 
and  the  allocated  time  for  it.  At  the  entrance  of  the  experimental  room  the  subjects  were  given  a number  to  a specific  ‘seat’  in  the 
room.  For  each  seat  the  presentation  order  of  the  positive  and  negative  video  segment  was  predetermined.  The  two  versions  were: 

Version  1 : clip2=positive  and  clip4=negative,  and 

Version  2:  clip2=negative  and  clip  4 = positive. 
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All  subjects  encountered  both  positive  and  negative  video  contents.  Table  1 shows  the  order  of  clips  in  all  settings. 


Table  L Overview  of  the  experimental  conditions 


INDIVIDUAL  TELEVISION  SETTING 

INDIVIDUAL  COMPUTER  SETTING 

Small 

Small  television  monitor  (14”,  S-VHS-PAL) 

Small  video  window  (320*240  pixels,  screen 
resolution:  1024*768) 

docu  - pos  - instr  - neg 
docu  - neg  - instr  - pos 

docu  - pos  - instr  - neg 
docu  - neg  - instr  - pos 

Big 

Big  television  monitor  (28”,  S-VHS-PAL) 

Large  video  window  (640*480  pixels,  screen 
resolution:  1024*768) 

docu  - pos  - instr  - neg 
docu  - neg  - instr  - pos 

docu  -pos-  instr  - neg 
docu  - neg  - instr  - pos 

SOCIAL  TELEVISION  SETTING 

Big 

Big  television  monitor  (28”,  S-VHS-PAL) 

docu  - pos  - instr  - neg 
docu  - neg  - instr  - pos 

Note: 

docu  = segment  from  a documentary  about  birds 

instr  = segment  from  an  instruction  video  about  welding 

pos  = segment  from  ‘When  Harry  met  Sally’ 

neg  = segment  from  a news-item  about  ‘Hunger  in  Ethiopia’ 

The  experiment  was  conducted  in  four  rooms: 

• Room  1 : Big  television-monitor  setting; 

• Room  2:  Little  television- monitor  setting, 

• Room  3:  Computer  setting  (for  all  computer  conditions  for  small  and  large  video  windows); 

• Room  4:  Big  television  monitor  in  a social  setting. 

The  individual  arrangements  were  in  such  a way  that  the  viewing  angles  for  the  television  and  the  computer  conditions  were 
the  same:  Watching  the  big  television  monitor  took  place  with  the  same  viewing  angle  as  watching  the  large  video  window  on  the 
computer  screen;  watching  the  small  television  monitor  took  place  with  the  same  viewing  angle  as  watching  the  small  video 
window  on  the  computer  screen.  To  accomplish  this,  viewing  distances  were  carefully  controlled. 

Four  experiment  monitors  and  four  assistants  were  hired  and  instructed.  In  advance  students  of  the  course  were  informed 
about  the  research  group  (time,  and  location)  to  which  they  were  assigned.  They  were  asked  to  gather  in  a central  location  in  the 
building,  so  that  no  current  sessions  would  be  disturbed.  Each  experiment  monitor  for  a certain  condition  (room)  invited  his/her 
group  to  go  to  the  room.  The  assistants  waited  for  the  participants  at  the  room.  Upon  entering  the  room,  the  assistants  gave  each 
participant  their  seat  number  (see  above).  They  were  asked  to  read  the  information  about  the  experiment,  fill  in  their  name  and 
wait  silently  for  the  experiment  monitors  to  start  the  experiment.  They  also  had  to  sign  an  attendance  list  for  the  course 
administration,  A few  days  before  and  at  the  day  of  the  experiment  some  students  attended  that  were  not  in  the  register  that  was 
used  for  the  assignment  procedure.  Because  of  the  course  credits  they  were  allowed  to  participate  and  assigned  to  the  different 
settings. 

The  experiment  monitors  gave  a brief  introduction  to  the  experiment.  The  procedure  and  the  instruments  (questionnaires) 
where  automated  into  a (wd)-like)  computer-based  environment.  For  the  computer  setting  this  was  done  to  create  the  look  and 
feel  of  a Web-environment,  To  be  consistent  in  the  overall  procedure  the  individual  television-monitor  settings  also  used  this 
environment  for  filling  in  the  digital  questionnaires.  Thus  in  both  settings  the  same  procedures  were  used.  In  the  social  television- 
monitor  setting  printouts  of  the  instructions  and  questionnaires  were  used. 

In  the  computer  setting  the  video  clips  were  announced  and  shown  within  this  setting.  The  video  clips  were  shown  as  part  of 
a separate  almost  full-screen  window  to  provide  a neutral  (dark  grey)  background  and  was  followed  by  a questionnaire.  In  the 
television-monitor  setting  the  participants  used  the  computer  for  filling  in  questionnaires  and  switched  to  watching  a television 
monitor  when  a video  clip  had  to  be  started.  In  this  condition  the  video  clips  were  available  on  tape  in  a video  player.  The 
videotape  contained  all  four  video  clips.  Each  video  clip  was  preceded  by  10  seconds  of  black  screen  and  the  clip’s  title  (like 
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“Video  clip  I”)  to  assure  enough  time  for  the  VCR  to  start  playing  and  as  confirmation  for  the  participants  that  the  correct  clip 
was  started,  After  each  clip  a message  to  stop  the  tape  was  presented. 

Scoring 

Instruments.  A questionnaire  containing  bipolar  affect  and  mood  items  using  a 7-point  Likert  scale  for  response  was  used  to 
assess  the  affect -evoking  and  mood  change  potential  of  the  positive  and  negative  video  materials.  Non-relevant  items  about 
cognitive  responses  (evaluations  of  the  videos)  and  arousal  items  were  included  to  make  the  aim  of  the  experiment  less  obvious 
to  the  participants,  A questionnaire  consisted  of  three  sections:  A section  with  questions  about  the  mood  state  of  the  participants 
(including  also  arousal  items),  a section  about  the  feelings  evoked  by  the  video  clip  (affective  responses)  and  a section  with 
questions  about  the  opinions  (evaluations)  about  the  video  clip.  As  mentioned  in  the  design  and  procedure  section  the  instruments 
(questionnaires)  where  automated  into  a (web-like)  computer  based  environment  and  presented  after  viewing  a video  clip.  To 
prevent  that  participants  would  develop  a repetitious  way  of  filling  out  the  questionnaires,  the  order  of  the  sections  in  each 
questionnaire  was  varied  and  also  the  direction  of  the  items.  Since  the  second  and  the  fourth  video  clip  were  the  actual  affective 
video  clips  of  interest  in  the  experiment,  the  section  of  the  questionnaire  that  measured  the  mood  state  was  always  placed  in  such 
a way  that  a pretest  - posttest  measurement  of  these  clips  could  be  made.  The  last  section  of  the  questionnaire  belonging  to  the 
first  video  clip  was  the  mood  section,  because  this  was  meant  to  be  the  pre-test  measure  for  the  second  clip.  The  first  section  after 
viewing  the  second  clip  was  again  the  mood  section  as  a post-test  measure.  For  the  fourth  clip  the  same  strategy  was  used. 

The  final  Mood  scale  consisted  of  three  items,  the  final  Affect  scale  consisted  of  five  items  (Figure  1). 


Mood  scale:  three  items: 

Affect  scale:  five  items: 

Very  negative  (1) 

Very  positive  (7) 

Very  negative  (1) 

Very  positive  (7) 

Very  unpleasant  ( 1 ) 

Very  pleasant  (7) 

Very  unpleasant  ( 1 ) 

Very  pleasant  (7) 

Very  somber  (1) 

Very  cheerful  (7) 

Very  somber  (1) 

Very  cheerful  (7) 

Very  sad  (1) 

Not  at  all  sad  (7) 

Very  happy  (1) 

Not  at  all  happy  (7) 

Figure  1.  Rating  scales  for  mood  and feeling 

A reliability  analysis  of  the  mood  items  showed  that  the  three  items  together  produced  a mood  scale  with  an  average  Cronbach’s 
alpha  score  of  ,88,  Originally  there  were  six  feeling  items.  They  produced  a reliability  score  of  .85.  When  one  of  the  feeling  items 
(very  involved  / not  at  all  involved)  was  deleted,  the  alpha  score  increased  and  resulted  in  an  average  score  of  .90,  (See  Table  2), 
The  reliability  scores  for  the  Mood  scale  and  the  Affect  scale  are  thus  quite  acceptable. 

Table  2.  Cronbach  's  alpha  scores  for  the  final  mood  and  feeling  scale 


Mood  Scale 
(3  items) 

Affect  Scale 
(5  items) 

Questionnaire  1 (clipl) 

.7758 

.8487 

Questionnaire  2 (clip2) 

.9358 

,9811 

Questionnaire  3 (clip3) 

.8452 

.8007 

Questionnaire  4 (clip4) 

.9557 

.9842 

Mean  questionnaires 

.8781 

,9037 

Results 

Homogeneous  treatment  groups 

The  first  video  clip  and  accompanying  questionnaire  served  as  a pre-test  measure  for  all  treatment  groups.  An  analysis  of 
variance  (one  way  anova)  showed  no  significant  differences  between  the  groups  on  the  Mood  scale  and  the  Affect  scale  (Mood 
scale  F^9,l  16]=l,048,p=.407  and  Affect  scale  /^9,1 16]=.606, /?=,790).  The  treatment  groups  can  be  considered  homogeneous  on 
the  dependent  variables. 

Group  sizes  and  gender.  The  average  group  size  was  n=12  (1 1.7).  On  the  average  29.9%  of  the  participants  were  male  and 
71.1%  were  female.  Data  showed,  however,  that  two  treatment  groups  were  different  from  the  other  groups.  The  treatment  group 
“computer  / small  video  / version  2 (clip2=neg,  clip4=pos)”  appeared  to  have  a relatively  small  number  of  participants  (n=6).  For 
the  treatment  group  “television  / small  video  / version  2 (clip2=meg,  clip4=pos;  n=9)“  atypical  male/female  proportions  were 
observed  of  66.7%  en  33.3%  respectively. 
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Affective  responses  and  Mood  changes 

Within  television  settings.  First  we  wanted  to  confirm  whether  positive  and  negative  video  materials  could  evoke  affective 
responses  and  mood  changes  in  participants  within  a traditional  television  viewing  setting  (both  social  and  individual).  A paired- 
T-test  was  conducted  for  the  settings  with  television.  In  this  analysis  the  first  clip  (documentary)  was  compared  with  the  second 
clip  (being  positive  in  version  1 and  negative  in  version  2). 

For  the  positive  clip  significant  effects  were  found  in  all  television  settings  for  a change  in  evoked  affective  responses.  Only 
in  the  individual  small  television  setting  no  significant  mood  changes  for  the  positive  clip  were  found  (p-35%). 

The  negative  video  clip  produced  in  all  television  settings  significant  effects  for  a change  in  mood  state  and  in  evoked 
affective  responses. 

Within  Web  settings.  For  the  positive  clip  significant  effects  for  mood  change  and  feeling  change  were  found  in  the  Web 
setting  with  the  large  video  in  both  versions.  For  the  small  video's  no  significant  effects  were  found  on  mood  changes  (p=.ll9) 
and  changes  in  affective  responses  (p=.078).  The  negative  video  clip  produced  in  all  television  settings  significant  effects  for  a 
change  in  mood  state  and  in  evoked  affective  responses. 

Comparisons  of  image-size  within  a media-setting 

Procedure.  As  the  data  showed  that  two  treatments  groups  of  version  2 appeared  to  have  a relatively  small  number  of 
participants  or  an  atypical  male/female  proportion,  the  analyses  of  the  results  will  be  based  on  version  1.  In  that  version  the 
sequence  of  the  clips  was  clip  2 = positive  and  clip  4 is  negative.  For  the  exploration  of  image-size  effects  version  1 is  taken  as 
starting  point  for  within-media-type  comparisons.  Clip  1 (documentary)  had  a slightly  positive  score  for  mood  and  affect  and 
Clip  3 (instruction)  had  a slightly  negative  score  on  mood  and  affect.  Both  are,  however,  sufficiently  close  to  the  middle  of  the 
scale  to  be  regarded  as  “neutral  clips**. 

Both  for  the  Mood  scale  and  the  Affect  scale  only  change  scores  are  presented  (not  the  absolute  scores).  Change  scores  are 
more  informative  because  they  represent  the  pretest -posttest  differences  in  the  line  of  the  intentions  of  the  experiment. 

Within  TV-setting:  Big  versus  little  TV.  For  the  big  tv-monitor  setting  (n=l  1)  and  the  little  tv-setting  (n=15)  the  mean  change 
scores  for  mood  state  and  affective  responses  were  compared.  With  one  small  exception  the  bigger  video  seemed  to  have  more 
impact  than  the  smaller  video  both  for  the  positive  video  (clip  2)  and  the  negative  video  (clip  4),  see  Table  3 and  Figure  2 and  3 
(third  and  fourth  treatment  setting).  A T-test  (for  two  independent  samples)  showed  a significant  effect  on  mood  change  for  the 
positive  clip  (clip  2)  with  r(24,26)=2. 156,/?=0.041  (two-tailed). 

Table  3.  Video  setting:  Sample  sizes,  Means,  Standard  Deviations  and  Standard  Errors 
for  Mood  and  Affect  Change  Scores  by  treatment 


Change  scores 

Treatment 

n 

Mean 

St.  Deviation 

Std.  Error  Mean 

Mood  change  clip2 

tv/big  video 

11 

3.55 

1.44 

.43 

tv/small  video 

15 

.93 

3.81 

.98 

Mood  change  clip3 

tv/big  video 

11 

-5.82 

3.68 

l.Il 

tv/small  video 

15 

-3.07 

4.67 

1.21 

Mood  change  clip4 

tv/big  video 

11 

-2.00 

2.10 

.63 

tv/small  video 

15 

-2.60 

3.04 

.79 

Affect  change  clip  2 

tv/big  video 

11 

7.27 

5.39 

1.62 

tv/small  video 

15 

6.40 

9.04 

2.33 

Affect  change  clip3 

tv/big  video 

11 

-13.27 

4.86 

1.47 

tv/small  video 

15 

-9.73 

8.19 

2.11 

Affect  change  clip4 

tv/big  video 

11 

-8.82 

3.82 

1.15 

tv/small  video 

15 

-6.73 

3.99 

1.03 

Note: 

Statistics  for  Version  1 of  Clip  sequence:  Clip2=positive;  Clip3=instruction;  Clip4=negative 
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Version  1 : clip2=pos,  clip3=instr,  clip4=neg 


computer/small  video  TV/small  video 


treatment 


Figure  2:  Bar  Graph  of  Mean  Change  Scores  for  Mood  (Version  1 sequence  of  video  clips) 


Version  1 : clip2=pos,  clip3=instr,  clip4=neg 


computer/small  video  TV/small  video 

treatment 

Figure  3:  Bar  Graph  of  Mean  Change  Scores  for  Affect  (Version  I sequence  of  video  clips) 
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Within  Web-setting:  Big  versus  little  video  play-back  size.  In  the  Web-setting  the  bigger  video  play  back  size  (n=14)  and  the 
smaller  play  back  size  (n=13)  were  compared.  Without  exception  the  bigger  playback  size  always  produced  a larger  mean  score 
for  mood  change  and  affect  change  than  the  smaller  video  size  as  is  shown  in  Table  4 and  figure  2 and  3 (first  and  second 
treatment  setting).  A T-test  (for  two  independent  samples),  however,  showed  that  these  differences  were  not  significant.  When 
we  look  at  the  female  participants  only  within  the  Web-setting  (n=9  for  both  image  sizes),  we  did  found  significant  differences 
for  image  size  for  affect  change  by  the  second  clip,  the  positive  clip  (p=  033,  two-tailed)  and  for  the  third  clip,  the  slightly 
negative  clip  (p=.029,  two-tailed). 

Table  4.  Web  setting:  Sample  sizes,  Means,  Standard  Deviations  and  Standard  Errors  for 
Mood  and  Affect  Change  Scores  by  treatment 


Change  scores 

Treatment 

n 

Mean 

Std.  Deviation 

Std.  Error 
Mean 

Mood  change  clip2 

computer/big  video 

14 

2.93 

4.01 

1.07 

computer/small  video 

13 

1.31 

2.81 

.78 

Mood  change  cl  ip  3 

computer/big  video 

14 

-6.36 

5.00 

1.34 

computer/small  video 

13 

-5.23 

4.21 

1.17 

Mood  change  clip4 

computer/big  video 

14 

-3.57 

3.48 

.93 

computer/small  video 

13 

-2.46 

2.50 

.69 

Affect  change  clip2 

computer/big  video 

14 

9.57 

7.78 

2.08 

computer/small  video 

13 

3.92 

7.33 

2.03 

Affect  change  clip3 

computer/big  video 

14 

-15.21 

5.99 

1.60 

computer/small  video 

13 

-11.00 

5.82 

1.61 

Affect  change  clip4 

computer/big  video 

14 

-6.64 

6.27 

1.68 

computer/small  video 

13 

-6.15 

2.94 

.82 

Note: 

Statistics  for  Version  1 of  Clip  sequence:  Clip2=positive;  Clip3=instruction;  Clip4=negative 


Comparisons  of  media-settings:  Web  versus  television 

Although  comparisons  between  media  types  are  difficult  to  make,  because  different  media  and  viewing  context  variables  are 
at  stake,  we  wanted  to  explore  whether  media  setting  (media  type  plus  viewing  context)  might  have  effect  on  the  magnitude  of 
affective  responses  and  mood  changes.  This  could  be  a relevant  question  for  instructors  in  their  media  selection  process. 
Therefore  the  Web-settings  and  the  television  settings  were  compared.  First  the  smaller  image  sizes  and  then  the  larger  image 
sizes  in  both  settings  were  compared.  As  mentioned  in  section  ‘Design  and  procedure*,  we  controlled  for  viewing  angle  to 
exclude  differences  between  the  settings  on  this  variable.  T-Test  showed  that  no  significant  differences  between  the  media 
settings  were  found.  A comparison  of  the  small  video-playback  size  in  the  Web  setting  (n=9)  with  a big  television  monitor 
applied  in  a classroom-like  presentation  (n=10)  showed  no  significant  effects.  When  however  the  big  video  playback  size  (n=14) 
within  the  Web-setting  was  compared  with  the  small  television  setting  (n=15)  a significant  effect  was  found.  In  this  case  the 
Web-setting  produced  more  impact  on  mood  state  and  affective  responses  than  the  television  setting  (p=.049,  two-tailed). 

Discussion 

One  of  the  most  powerful  characteristics  of  video  media  is  their  potential  to  evoke  emotions.  This  is  one  of  the  reasons  why 
video  material  is  often  selected  for  educational  purposes.  This  study  raised  the  question  whether  affective  video  in  a WEB-based 
environment  is  experienced  differently  from  video  in  a traditional  context  and  whether  video  image  size  is  a relevant  factor  to 
consider  in  this  context.  The  key  question  is  in  short:  "Affective  video  in  a Web:  Does  it  work  ?". 

We  started  this  paper  with  the  expectations  that  affective  video  materials  might  indeed  work  when  implemented  in  a Web- 
based  learning  environment.  Media-setting  (Computer[Web]  versus  television),  affective  video  content  (positive  versus  negative) 
and  image  size  (big  versus  small)  were  the  three  key  factors  in  the  experiment.  Another  factor,  social  versus  individual  viewing 
situation,  was  introduced  to  see  whether  this  is  a relevant  aspect  of  the  media-setting  to  consider.  For  an  overview  of  the 
treatment  groups  we  refer  to  Table  1 . 

The  results  show  that  we  found  evidence  for  the  affective  potential  of  video  for  both  media  settings.  Only  in  the  individual 
television  setting  with  the  small  monitor  this  effect  was  not  significant  on  mood  changes  for  the  positive  video  clip  as  second 
clip.  Also  in  the  Web- environment  we  found  that  the  affective  video  materials  were  effective  in  evoking  affective  responses  and 
changes  in  mood.  Again  the  small  positive  clip  as  second  clip  was  the  exception  to  this  rule.  The  reason  why  the  positive  clip  as 
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second  clip  did  less  well  in  the  small  image  settings,  might  have  to  do  with  the  character  of  the  first  clip.  This  clip  (documentary 
about  birds)  was  also  experienced  as  (slightly)  positive,  which  reduced  the  room  for  further  positive  change  of  mood  or  affect. 
The  fact  that  in  some  cases  the  smaller  video’s  didn’t  work,  made  the  question  about  image  size  even  more  relevant.  In  general, 
however,  we  can  conclude  that  affective  video  materials  can  be  effective  when  embedded  in  a Web-based  environment. 

As  far  as  image  size  is  concerned,  in  almost  all  treatment  groups  the  larger  image  size  seemed  to  have  more  impact  than  the 
smaller  video  for  both  affective  clips.  This  ‘pattern’,  however,  was  only  significant  for  mood  change  for  the  positive  clip  (clip  2) 
in  the  television  setting.  When  we  looked  at  the  female  participants  within  the  Web-setting  we  did  found  significant  differences 
(although  two  tailed)  for  image  size  on  affect  change  by  the  affective  video  clips  (including  the  third  clip).  With  the  relatively 
small  sample  sizes  (on  average  n-12)  in  this  study  it  is,  however,  not  easy  to  find  significant  differences  between  the  treatment 
groups.  We  suggest  for  further  studies  the  use  of  larger  sample  sizes,  controlling  for  gender,  and  a broader  range  of  affective  clips 
to  explore  the  image -size  issue  more  in  depth.  It  is,  for  instance,  unknown  whether  the  impact  of  mood  or  affect  change  on 
performance  is  subject  to  threshold  levels.  Small  differences  in  the  amount  of  change  could  then  be  critical,  which  may  add 
importance  to  findings  such  as  in  this  study. 

Not  only  may  image  size  have  an  impact  on  affective  responses  and  mood  changes,  but  also  on  arousal  (another  aspect  of 
emotion)  and  attention.  A study  by  Reeves,  Lang,  Kim  and  Tatar  (1999)  about  the  effects  of  screen  size  and  message  content 
showed  that  image  size  can  increase  attention  and  arousal  for  audio-visual  messages.  Arousing  pictures,  like  sex  and  violence  for 
example,  produced  higher  levels  of  arousal  when  presented  on  a large  screen  than  presented  on  a medium  or  small  screen  (screen 
sizes  were  respectively  56  inch,  13-inch  and  2 inch  picture  heights).  Also  larger  images  were  remembered  better  the  smaller 
images.  It  would  be  interesting  to  explore  these  issues  within  a Web-environmental  context,  also  to  find  out  whether  Reeves, 
Lang,  Kim  and  Tatar  are  right  when  they  state  that  “The  inclusion  of  display  size  as  an  important  variable  in  media  research 
seems  critical  (p.  64).” 

One  of  our  purposes  was  to  explore  from  an  instructors  point  of  view  whether  a shift  from  traditional  classroom  or  individual 
use  of  video  to  an  individual  Web-based  learning  environment  would  have  an  impact  on  the  way  in  which  the  same  affective 
video  materials  are  experienced  by  students.  A comparison  between  the  two  media-settings  in  this  study  (television  versus  Web) 
was  made.  No  significant  effects  were  found  with  one  exception:  The  large  videos  within  a Web-environment  evoke  stronger 
affective  responses  and  mood  changes  than  the  small  television  monitor.  This  might  be  the  case  because  the  image  size  was 
perhaps  experienced  as  relatively  big  because  of  the  short  viewing  distance  to  the  monitor  in  the  computer  set  ting. 

This  study  could  not  reveal  effects  for  viewing  affective  video  materials  in  a group.  This  was  expected,  but  the  actual 
experimental  setting  was  probably  too  atypical  for  a social  viewing  session  because  of  the  procedure  and  the  questionnaires.  The 
session  was  primarily  experienced  as  a laboratory  setting.  Laughs  or  other  sounds  made  aloud  were  rare.  In  future  research  a 
more  realistic  viewing  situation  should  be  created. 

For  the  measurement  of  the  affective  responses  and  mood  states  we  used  Likert-scales  for  response.  A disadvantage  of  this 
type  of  measurement  is  that  it  is  subjective  and  that  it  asks  for  insight  in  one’s  feelings  and  mood.  We  would  recommend  the  use 
of  more  objective  instruments  like  physical  measurement  techniques,  if  available,  to  contribute  to  the  validity  of  this  type  of 
research. 

In  conclusion.  When  we  return  to  the  question:  "Affective  video  in  a Web:  Does  it  work  ?",  the  answer  is  that  in  most  cases  it 
does,  but  with  smaller  window  sizes  it  might  do  less  well.  Which  leads  for  the  time  being  to  a general  advice  for  applying  videos 
in  the  affective  domain:  Try  not  to  compromise  on  image  size  when  shooting  or  selecting  affective  video  materials  for  the  Web. 
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Abstract 

Using  the  Pathfinder  Scaling  Algorithm  (Pathfinder)  (Interlink,  1994),  the  researchers  conducted  a cognitive  task  analysis 
expertise  in  Human  Performance  Technology  (HPT).  The  study  investigated:  1)  the  extent  to  which  Pathfinder -derived  coherence 
scores  were  associated  with  other  measures  of  HPT  expertise;  2)  how  HPT  experts  organize  their  knowledge  of  the  discipline; 
and  5)  how  experts  organize  their  HPT  knowledge  differently  than  novices.  Findings  include:  1)  a significant  correlation  between 
coherence  and  the  number  of  HPT-related  books  participants  had  written;  2)  a Pathfinder -derived  concept  map  of  HPT;  and  5) 
expected  novice/expert  differences  in  Pathfinder  similarity  and  relatedness  scores. 

Most  representations  of  academic  disciplines  have  been  created  when  experts  depict  or  report  what  they  know.  Experts  draw 
diagrams,  establish  competencies,  and  write  textbooks.  Clearly,  this  approach  works;  new  practitioners  use  these  representations 
to  enter  and  gain  competency  in  the  discipline.  Some  practitioners  eventually  master  elements  of  the  discipline.  However,  there 
are  potential  problems  that  can  arise  when  practitioners  rely  on  expert  self-report.  Simply  stated,  experts  are  often  unable  to  state 
what  they  know  and  do  that  makes  them  experts.  Two  cognitive  processes  account  for  this  phenomenon.  First,  experts  may  be 
unable  to  explicitly  state  knowledge  they  have  learned  implicitly.  Implicit  learning  occurs  when  people  acquire  knowledge  about 
the  relationships  comprising  a complex  system,  without  necessarily  knowing  in  advance  what  the  variables  are.  Examples  include 
solving  differential  equations  underlying  a simulated  parking  task  (Broadbent,  Fitzgerald,  & Broadbent,  1986)  and  predicting 
where  a light  will  appear  on  a computer  screen  in  a serial  learning  task  (Willingham,  Nissen,  & Bullemer,  1989).  In  these 
situations,  people  have  encountered  a complex  task,  observed  the  variables  in  operation  in  an  unselective  manner,  and  attempted 
to  store  the  contingencies  among  them  (Berry,  1993).  Essentially,  people  implicitly  learn  these  complex  system  and  the 
relationships  comprising  it  by  “mucking  about.”  One  of  the  results  of  implicit  learning  is  that  the  resulting  knowledge  is 
relatively  inaccessible  to  free  recall  (Berry,  1993).  Experts  who  have  implicitly  constructed  domain-specific  knowledge  may  not 
be  able  to  articulate  what  they  know. 

Second,  experts  may  be  unable  to  articulate  cognitive  processes  that  have  reached  a level  of  automaticity.  According  to 
Anderson  (1993),  this  situation  exists  when  people  explicitly  learn  new  skills  or  knowledge  by  using  language  to  mediate  a 
cognitive  production,  such  “determine  the  cause  of  a performance  gap”  or  “select  an  appropriate  intervention  to  address  the  cause 
of  the  gap.”  With  successive  tuning,  the  production  fires  automatically,  without  invoking  any  mediating  language.  The  result  is 
automatic,  fluent  performance — without  the  ability  to  report  on  the  cognitive  aspects  of  the  performance  itself 

One  way  to  avoid  potential  problems  associated  with  expert  self-report  is  to  employ  cognitive  task  analysis  methods.  The 
Pathfinder  Scaling  Algorithm  is  one  such  method  (Schvaneveldt,  1990;  Interlink,  1994).  This  software  program  transforms 
participants’  pair-wise  ratings  of  related  concepts  into  a semantic  network — a concept  map  comprised  of  nodes  (the  concepts) 
and  links  that  depict  the  relationships  among  the  concepts. 

A Pathfinder-derived  concept  map  of  a discipline  could  supplement  existing  representations  created  using  expert  self-report. 
One  opportunity  for  using  Pathfinder  to  map  a discipline  lies  in  the  area  of  human  performance  technology  (HPT):  “...a  set  of 
methods  and  procedures,  and  a strategy  for  solving  problems,  for  realizing  opportunities  related  to  the  performance  of  people.  It 
can  be  applied  to  individuals,  small  groups,  and  large  organizations”  (ISPI,  2001).  As  no  cognitive  task  analysis  of  HPT  has  been 
conducted  to  date,  this  study  sought  answers  to  the  following  three  research  questions: 

1.  To  what  extent  are  Pathfinder-derived  coheraice  scores  associated  with  other  measures  of  HPT  expertise? 

2.  How  do  experts  in  the  field  organize  their  knowledge  of  HPT? 

3.  To  what  extent  do  experts  organize  their  HPT  knowledge  differently  than  novices? 
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Literature  Review 

This  section  defines  key  Pathfinder  concepts  and  addresses  the  literature  relevant  to  each  of  the  study’s  research  questions. 
In  addition  to  generating  concept  maps  that  represent  domain  knowledge  without  the  complications  associated  with  expert  self- 
report,  Pathfinder  generates  three  measures  employed  in  this  study:  coherence,  relatedness,  and  similarity.  Table  1 describes  these 
measures. 


Table  1 - 

Definition,  Uses,  and  Examples  of  Coherence,  Relatedness,  and  Similarity  Scores  (adapted  from  Villachica,  2000) 


Measure 

Definition 

Uses 

Examples 

Coherence 
(Theoretical 
Range  = -1.0  to 
+ 1.0) 

A Pearson  Product -Moment 
correlation  indicating  the 
internal  consistency  of 
ratings  within  an  individual’s 
or  group’s  set  of  concept 
ratings  (Interlink,  1996). 

A coherence  score  for  an 
individual’s  or  group’s  set  of 
concept  ratings  calculated. 
This  score  is  compared  to 
that  of  other  individuals. 

Student  coherence  scores 
obtained  prior  to  instruction  are 
compared  to  those  obtained  after 
instruction  to  determine  if 
learning  occurred.  . 

Relatedness 
(Theoretical 
Range  = -1.0  to 
+ 1.0) 

The  Pearson  Product- 
Moment  correlation  between 
sets  of  concept  ratings 
(Interlink,  1996). 

The  comparison  of  an 
individual’s  or  group’s  set  of 
concept  ratings  to  another’s. 

The  proximity  matrices  of 
students  are  correlated  with  a 
referent  matrix  containing  the 
averaged  responses  of  a panel  of 
experts. 

Similarity 
(Theoretical 
Range  = 0 to 
+ 1.0) 

The  proportion  of  shared 
links  in  two  concept  maps 
using  the  same  terms.  The 
ratio  of  shared 
neighborhoods  in  two 
Pathfinder  networks 
(Interlink,  1996). 

The  Pathfinder  networks  of 
an  individual  or  group  are 
compared  to  those  of  another. 

The  Pathfinder  networks  of 
students  are  compared  to  their 
instructor’s.  The  resulting 
similarity  scores  are  used  to 
predict  performance  on  a test. 

Coherence  and  Expertise 

Since  the  knowledge  structures  of  experts  are  more  organized  than  those  of  novices,  experts  in  a given  domain  should  possess 
relatively  higher  coherence  scores.  The  higher  the  coherence  score,  the  greater  the  internal  consistency  of  the  concept  ratings. 
Four  studies  have  investigated  the  relationship  between  coherence  and  expertise  in  different  domains.  However,  results  have  been 
uneven,  indicating  a need  for  additional  replication. 

Three  of  these  studies  found  a correlation  between  coherence  and  expertise.  In  a classroom  setting,  Housner,  Gomez,  and 
Griffey  (1993)  investigated  the  extent  to  which  coherence  scores  and  other  knowledge  structures  would  predict  performance  in  a 
physical  education  preservice  course.  They  found  a moderate  correlation  between  coherence  and  a simulated  teaching  activity  (r 
= .63).  Gaultieri,  Fowlkes,  and  Ricci  (1996)  investigated  the  effectiveness  of  training  that  eight  Navy  and  Air  Force  pilots 
received  over  a five-day  workshop.  This  training  consisted  of  pre-briefings,  simulated  missions,  and  debriefings.  The  pilots  were 
divided  into  two  teams  of  four  pilots  each,  with  each  participants  rating  concept  pairs  on  the  first,  third,  and  fifth  days.  Results  of 
a repeated  measures  analysis  of  variance  revealed  a statistically  significant  increase  in  participants’  coherence  scores  over  time 
(f2,14  = 1 1.55.  E.  < .001).  In  a similar  study.  Stout,  Salas,  and  Kraiger  (1997)  studied  12  naval  aviator  trainees  enrolled  in  a one- 
day,  complex  training  program  addressing  aviation  teamwork  and  communication  skills.  The  researchers  reported  that  course 
attendees  earned  higher  mean  coherence  scores  than  a control  group  who  received  no  training  (t  = 2.70,  p,  < .01,  Mtraincecs^  -63, 

Mrnntrni  ~ .26). 

One  study  did  not  find  a relationship  between  coherence  scores  and  domain  expertise.  Dorsey,  Campbell,  Forste,  and  Miles 
(1999)  used  Pathfinder  networks  to  create  concept  maps  that  evaluated  relationships  generated  by  88  computer  users  and 
compared  them  against  the  scores  of  four  subject  matter  ecperts.  Their  results  indicated  that  coherence  scores  were  not 
significantly  related  to  any  of  the  concept  map  scores. 

Within  the  domain  of  HPT,  coherence  scores  would  conceivably  be  positively  correlated  with  several  measures  of  expertise. 
Experts  in  HPT  are  members  of  a community  of  practice  that  has  its  roots  in  academe.  As  a result,  experts  can  be  expected  to 
present  at  conferences,  write  articles,  and  author  books.  They  can  be  expected  to  have  spent  at  least  10  years  developing  their 
expertise  (Ericsson,  1996). 

Organization  of  HPT  Domain  Knowledge 

The  organization  of  HPT  domain  knowledge  can  be  viewed  in  terms  of  existing  expert  representations  of  the  domain  and  the 
extent  to  which  a Path  finder- generated  concept  map  could  supplement  these  existing  models. 
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Existing  expert  representations  of  HPT.  Given  the  depth  and  breadth  of  HPT,  a variety  of  experts  have  authored  conceptual 
and  procedural  representations  of  the  discipline.  Each  representation  depicts  an  expert’s  organization  of  his  or  her  HPT  expertise. 
Conceptual  representations  place  HPT  within  theoretical  contexts  and  illustrate  the  relationships  among  its  components. 
Procedural  representations  illustrate  the  steps  comprising  an  HPT  process. 

Stolovitch  and  Keeps  (1999)  provide  both  conceptual  and  procedural  representations  of  HPT.  In  their  conceptual  model,  the 
authors  depict  external  and  organizational  environments  that  influence  internal  requirements  related  to  human  performance.  Once 
articulated,  these  requirements  trigger  behaviors  that  produce  accomplishments,  which  are  subjected  to  verification. 
Accomplishments  that  are  aligned  with  business  requirements  are  accepted;  those  that  are  not  aligned  are  subjected  to  subsequent 
alteration  in  behaviors,  which  change  the  organization’s  accomplishments.  Their  procedural  representation  of  HPT  consists  of  an 
iterative,  10-step  process  that  begins  with  the  identification  of  business  requirements  and  concludes  with  monitoring  and 
maintaining  performance  interventions.  At  this  point,  the  process  can  repeat  itself,  leading  to  the  identification  of  new 
requirements  and  subsequent  interventions. 

Addison  (2001)  provides  a conceptual  view  of  HPT  in  his  Performance  Consultant  HPT  Landscape.  This  representation  of 
the  discipline  depicts  the  interaction  of  two  conceptual  and  two  procedural  dimensions  that  comprise  the  landscape: 

1 . Levels  of  the  environment,  ranging  from  the  worker  to  society  (conceptual); 

2.  Principles  of  performance  technology  (conceptual); 

3.  Systematic  approach,  starting  with  need  and  ending  with  evaluate  (procedural);  and 

4.  System(s)  view  point,  beginning  with  conditions  and  ending  with  feedback  (procedural). 

The  International  Society  for  Performance  Improvement  has  created  its  own  procedural  representation  of  HPT.  This  process 
model  focuses  on  a systematic  combination  of  performance  analysis,  cause  analysis,  and  intervention  selection  (ISPI,  2001).  The 
majority  of  steps  comprising  the  HPT  model  contain  additional  conceptual  information  providing  additional  detail  about  a given 
step. 

Pathfinder  representations  of  expertise.  In  addition  to  the  conceptual  and  procedural  representations  used  to  depict  expert’s 
organization  of  HPT,  one  can  use  Pathfinder  networks  to  represent  this  domain-specific  expertise.  The  end  result  of  Pathfinder 
analysis  is  a concept  map  that  depicts  a semantic  network  representing  a domain,  such  as  HPT.  These  concept  maps  are  intended 
to  represent  the  knowledge  structures  that  humans  store  in  their  minds  (Jonassen,  Beissner,  & Yacci,  1993). 

Researchers  have  employed  Pathfinder-derived  concept  maps  to  study  the  nature  of  expertise  in  some  1 3 studies  and  nine 
different  domains,  ranging  from  aircraft  combat  to  programming  to  electronic  troubleshooting  to  medicine  (Villachica,  2000).  In 
a similar  vein.  Pathfinder-derived  concept  maps  could  depict  the  way  in  which  HPT  experts  organize  their  domain  knowledge, 
providing  another  representation  of  this  complex  discipline. 

Novice/Expert  Differences  in  the  Organization  of  HPT  Knowledge 

It  is  not  surprising  that  novices  and  experts  exhibit  different  levels  of  domain-specific  performance.  A well-established  line  of 
research  traces  the  sources  of  these  performance  differences  to  the  organization  of  cognitive  structures  in  memory.  De  Groot 
(1978)  and  Chase  and  Simon  (1973)  demonstrated  that  expertise  in  chess  is  partially  attributable  to  the  organization  of  memory. 
Chess  masters  possess  more  highly  organized  and  complex  structures  in  long-term  memory  than  chess  experts,  who  possess  more 
organized  and  complex  structures  than  non-experts.  These  differences  in  domain  knowledge  allow  chess  masters  to  employ  a 
larger  visual  scan  than  chess  novices  (Reingold,  Chamess,  Pomplun,  & Stampe,  2001).  Similarly,  Chi,  Feltovich,  and  Glaser 
(1981)  report  that  physicists  sort  physics  problems  differently  than  novices,  with  experts’  organizational  strategies  revealing  more 
sophisticated  cognitive  structures,  based  upon  their  knowledge  of  the  “deep  structure”  of  the  domain. 

Pathfinder-related  studies  that  address  novice/expert  differences  in  domain-specific  cognition  have  found  that  experts  tended 
to  exhibit  greater  degrees  of  intragroup  agreement  than  novices.  That  is,  the  relatedness,  similarity,  and  coherence  scores  of 
experts  tended  to  be  more  similar  than  those  of  novices.  For  example,  Schvaneveldt,  Durso,  Goldsmith,  Breen,  and  Cooke  (1985) 
conducted  a discriminant  analysis  using  Pathfinder  measures  that  successfully  predicted  novice  and  expert  performance  in  a 
fighter  pilot  task.  Schvaneveldt,  Durso,  and  Dearholt  (1989)  employed  Pathfinder-produced  concept  maps  to  study  differences  in 
the  ways  in  which  biology  graduate  students  (“experts”)  and  undergraduate  students  (“novices”)  organized  their  knowledge  of 
biology.  Schvaneveldt,  Beringer,  Lamonica,  Tucker,  and  Nance  (2000)  used  Pathfinder  to  demonstrate  differences  in  the 
priorities  that  novice  and  experienced  commercial  aircraft  pilots  assign  to  information  viewed  during  the  phases  of  a flight. 
Thompson  (1992)  employed  Pathfinder-based  measures  to  reveal  differences  in  the  organization  of  domain  knowledge  among 
expert,  non-expert,  and  novice  nurses. 

Given  the  consistency  of  these  findings,  one  should  expect  to  find  novice-expert  differences  in  the  organization  of  HPT 
knowledge.  That  is,  the  Pathfinder-related  measures  of  an  operationally  defined  set  of  experts  should  be  more  like  each  other  than 
they  are  like  those  of  a set  of  novices. 

Methodology 

Participants  and  Procedure 

ISPI  issued  invitations  to  participate  in  the  study  to  approximately  4,500  of  its  6,000  members  (75  percent).  ISPI  published 
the  invitations  on  two  occasions,  the  first  invitation  in  its  online  newsletter  ISPI  Quick  Read.  After  14  days,  only  38  people  had 
responded  to  this  invitation.  ISPI  then  emailed  a second  invitation  to  its  members  two  weeks  later.  With  this  invitation,  another 
103  persons  had  responded,  making  a total  of  141 . Of  those,  4 people  had  submitted  duplicate  data  sets,  which  were  subsequently 
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removed  from  the  study.  Of  the  137  people  who  had  completed  the  survey,  73  went  on  to  complete  all  435-concept  ratings.  Thus, 
the  overall  response  rates  to  the  study  were  2 and  1 percent,  respectively. 

The  researchers  drafted  an  initial  list  of  50  HPT-related  concepts  based  upon  a review  of  the  Handbook  of  HPT  (Stolovitch  & 
Keeps,  1999).  Acting  as  the  operationally  defined  referent  experts  for  the  study,  the  ISPI  2000-2001  Board  participated  in  a 
modified  Delphi  process  that  resulted  in  a final  list  of  30  HPT  concepts  that  would  be  employed  in  the  study.  ISPI  members  were 
invited  to  participate  in  the  study  via  two  methods.  One,  a general  announcement  was  published  in  the  ISPI  Quick  Read 
newsletter,  and  second,  individual  invitations  were  emailed.  The  second  section  of  both  the  ISPI  Quick  Read  article  and  the  email 
invitation  contained  an  informed  consent  form.  Subjects  who  agreed  to  participate  in  the  study  indicated  their  agreement  by 
clicking  on  a link  that  directed  them  to  a website  used  to  collect  demographic  data  and  concept  ratings.  In  order  to  maintain 
confidentiality,  all  data  were  aggregated  into  a single  computer  file  using  the  computer  to  assign  unique,  anonymous  participant 
identification  codes.  Qnce  subjects  had  entered  their  responses  into  an  online  dataset,  all  names  were  deleted  from  the  file, 
thereby  guaranteeing  the  anonymity  of  all  participants  and  protecting  subjects  against  unintentional  disclosure  outside  the 
experiment. 

The  website  consisted  of  the  several  pages,  which  participants  completed  in  order.  After  viewing  a welcome  page  that 
described  the  purpose  of  the  study,  participants  viewed  instructions  about  rating  HPT  concepts.  The  practice  component 
comprised  of  the  next  two  pages  of  the  web  site.  In  the  first  page,  participants  then  viewed  a list  of  practice  terms,  which 
appeared  next  to  radio  checkboxes.  Participants  could  click  next  to  any  term  they  did  not  know.  In  the  next  page,  participants 
completed  a set  of  practice  ratings.  The  survey  component  of  the  web  site  consisted  of  a single  web  page  that  participants 
completed  to  provide  information  about  themselves.  The  ratings  component  of  the  web  site  consisted  of  two  pages.  In  the  first 
page,  participants  viewed  the  complete  list  30  HPT  concepts;  with  the  option  to  check  any  terms  they  did  not  know.  In  the  next 
page,  participants  rated  up  to  345  randomly  presented  pairs  of  HPT  concepts,  checking  on  the  radio  buttons  comprising  a rating 
scale  to  assign  lower  numbers  (1-4)  to  unrelated  pairs  and  higher  numbers  (6-9)  to  related  pairs.  Participants  assigned  a “5”  to 
concept  pairs  they  did  not  know  or  could  not  rate.  The  last  pages  of  the  web  site  thanked  participants  and  allowed  them  to  view 
additional  information  about  the  HPT  Research  Group  conducting  the  study.  The  concept  ratings  participants  provided  were  then 
stored  as  proximity  matrices  for  subsequent  Pathfinder  analysis. 

Results 

Coherence  and  HPT  Expertise 

Table  2 summarizes  the  measures,  survey  items,  number  of  respondents,  means,  standard  deviations,  and  standard  errors  used 
in  the  regression  analysis  of  coherence  scores. 

Table  2 

Descriptive  Statistics  Used  in  the  Regression  of  Coherence  Scores  on  Measures  of  HPT  Expertise 


Measure 

Item 

Statistics 

Coherence  scores 

Not  applicable 

N = 

73 

M = 

0.324 

0.153 

Years  HPT  practitioner 

How  many  years  have  you  been  a 

N = 

137 

practitioner  of  human  performance 

M = 

11.161 

technology? 

8.239 

Number  of  juried 

How  many  JURIED  presentations  relating 

N = 

137 

presentations 

to  HPT  have  you  written  or  co-written? 

M = 

3.321 

SD  = 

11.758 

Number  of  non-juried 

How  many  NQN-JURIED  presentations 

N = 

137 

presentations 

relating  to  HPT  have  you  written  or  co- 

M  = 

4.409 

written? 

SD  = 

8.839 

Number  of  juried 

How  many  JURIED  articles  relating  to  HPT 

N = 

137 

articles 

have  you  written  or  co-written? 

M = 

0.744 

SD  = 

3.097 

Number  of  non-juried 

How  many  NQN-JURIED  articles  relating 

N = 

137 

articles 

to  HPT  have  you  written  or  co-written? 

M = 

2.007 

SD  = 

6.026 

Number  of  book 

How  many  book  chapters  relating  to  HPT 

N = 

137 

chapters 

have  you  written  or  co-written? 

M = 

0.774 

SD  = 

3.132 

Number  of  books 

How  many  books  relating  to  HPT  have  you 

N = 

137 

written  or  co-written? 

M = 

0.197 

SD  = 

1.028 

To  determine  the  relationship  between  Pathfinder  coherence  scores  and  other  measures  of  expertise,  the  researchers 
employed  a multiple  linear  regression  analyses  that  regressed  coherence  scores  on  the  number  of  years  as  an  ffPT  practitioner. 
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number  of  juried  presentations,  number  of  non-juried  presentations,  number  of  juried  articles,  number  of  non-juried  articles, 
number  of  book  chapters,  and  number  of  books  the  participant  had  either  written  or  co-written.  To  control  for  Type  I error,  alpha 
was  set  at  .05.  All  variables  were  entered  at  once.  The  multiple  linear  regression  produced  statistically  significant  results  (r  = 
.458,  E = .026),  accounting  for  21  percent  of  the  variance  in  coherence  scores.  As  depicted  in  Table  3,  the  only  statistically 
significant  independent  variable  contained  in  the  equation  was  the  number  of  books  the  participant  had  written  or  co-written.  The 
other  independent  variables  did  not  reach  levels  of  statistical  significance.  The  magnitude  of  this  correlation  is  weak,  although  it 
approaches  the  moderate  threshold  of  .5. 

Table  3 

Regression  Analysis  Summary  for  HPT  Expertise  Variables  Predicting  Coherence  Scores 


Variable 

Coherence  (n  = 73,  = . 0.210)* 

B 

SEB 

Years  HPT  practitioner 

0.0006 

0.002 

0.027 

Number  of  juried  presentations 

-0.0005 

0.007 

-0.012 

Number  of  non-juried  presentations 

-0.0049 

0.007 

-0.149 

Number  of  juried  articles 

0.0056 

0.029 

0.033 

Number  of  non-juried  articles 

0.0098 

0.013 

0.126 

Number  of  book  chapters 

-0.0025 

0.016 

-0.020 

Number  of  books 

0.2747 

0.067 

0.468*  1 

p<0.05 


It  should  be  noted,  however,  that  only  9 of  137  participants  who  responded  to  this  item  on  the  demographic  survey  had 
written  a book.  Stevens  (1992)  notes  that  statistical  procedures  based  upon  regression  are  mathematical  maximization  procedures 
may  capitalize  on  chance  and  artificially  increase  alpha,  especially  when  the  number  of  subjects  is  not  large  relative  to  the 
number  of  variables.  Thus,  the  results  obtained  in  this  study  may  not  replicate  in  another  sample. 

Organization  of  HPT  Expertise 

As  a measure  of  the  internal  consistency  of  a participant’s  concept  ratings,  coherence  scores  can  be  used  as  an  operational 
indicator  of  domain  expertise  (Interlink,  1994).  To  identify  the  participants  who  would  act  as  the  expert  HPT  practitioners  in  this 
study,  the  researchers  identified  all  participants  with  coherence  scores  greater  than  0.4  di  = 21,  M = .4976,  ® = .1309) 
(Schvaneveldt,  personal  communication,  June  23,  1998).  The  researchers  then  averaged  the  concept  ratings  of  these  operationally 
defined  experts  and  subjected  the  results  to  Pathfinder  analysis,  which  produced  the  following  concept  map.  Figure  1 represents 
the  organization  of  a statistically  derived,  “averaged  expert’s”  cognitive  organization  of  HPT. 

ISD 


Human  Capital 


^ Learning” 
Theory 


"Nwds 
. Analysis^ 


Instructional 
"Systems  Design 


pbb  'and  " ] 
JlaslLAn^sis; 


! Systematic  i 


Collaboration 


Individual  and 
Team  Workers 

/ 

Human 

^^ttal 


Resources 


Competencies 


[1 


Workplace 
Organization  1 


Motivation 


^Performance 
L^ers ^ 


HPT 


Information 


["systemic 


Requirements 

Analysis 


Business  Cases 


Cause 
Analysis  , 


" Perfomianoe ' 
LSupport^.  _ 

: interventions' 


Workplace  Organization 
Figure  1.  Expert  Concept  Map 


BEST  COPY  AVAILABLE 

441 

450 


The  key  concept  in  the  map  is  represented  by  ''results  which  is  linked  to  four  other  sets  of  concepts.  The  first  set  branch  of 
concepts  contains  the  terms  "learning  theory,  "ISD"  "needs  analysis"'  "job  and  task  analysis  " "systematic,'"  "measurement," 
"evaluation,"  and  "feedback,"  The  organization  of  these  concepts  indicates  that  experts  employ  ISD  and  its  related  components  to 
obtain  results.  The  branch  of  concepts  related  to  "results"  contains  the  terms  " goals ,"  change  management,"'  "motivation," 

"performance  barriers,"  "cause  analysis,"  "performance  support,"  and  "interventions,"  These  terms  form  the  basis  of  HPT 
theory  and  reflect  its  behavioral  roots.  The  organization  of  these  concepts  indicates  that  experts  employ  HPT  theory  to  obtain 
results.  A third  branch  of  concepts  related  to  "results"  contains  the  terms  "Return  on  Investment" , "business  case,"  and 
"requirements  analysis,"  Experts  use  requirements  analysis  to  build  business  cases  from  which  they  can  predict  a return  on 
investment  that  quantifies  obtained  results.  The  organization  of  these  items  indicates  that  experts  employ  business-related 
measures  to  obtain  results.  A fourth  branch  of  concepts  related  to  "results"  is  more  complex  and  consists  of  two  separate  subsets. 
The  first  subset  consists  of  the  terms  "outputs,"  "inputs,"  "conditions,  "work  organization,"  and  "systemic,"  Systemically 
accounting  for  workplace  organization,  experts  assess  the  conditions  of  performance,  their  inputs,  and  their  outputs  in 
determining  performance  results.  It  is  also  important  to  note  that  "workplace  organization"  is  also  linked  to  "business  case',' 
indicating  that  experts  use  the  context  of  the  workplace  to  create  such  cases.  The  second  subset  consists  of  the  terms 
"information,"  "resources,"  human  capital,  "competencies,"  "individual  and  team  workers,"  and  "collaboration,"  The 
organization  of  these  terms  indicates  that  collaboration  among  individual  and  team  workers  builds  human  capital,  which  can  be 
described  in  terms  of  competencies.  Human  capital  is  linked  to  resources.  Information  about  resources  allows  resources  to  be 
used  as  inputs  to  performance-which  ultimately  lead  to  outputs  and  results. 


Novice/Expert  Differences  in  the  Organization  of  HPT  Knowledge 

To  explore  differences  in  the  organization  of  HTP  knowledge  among  an  operationally  defined  set  of  novices  and  experts,  the 
researchers  employed  a median  split.  In  this  instance,  experts  were  operationally  defined  as  any  participant  possessing  a 
coherence  score  one  standard  deviation  above  the  mean  (n  = 8,  M = .1035,  SD  = .0553).  Novices  were  operationally  defined  as 
any  participant  possessing  a coherence  score  one  standard  deviation  below  the  mean  (n  = 8,  M,  = .1353,  SD  = . 0161).  Table  4 
summarizes  means  and  standard  deviations  associated  with  experts’  and  novices’  similarity  and  relatedness  scores. 


Tabled 


Numbers,  Means,  and  Standard  Deviations  for  Novice  and  Expert  Similarity  and  Relatedness  Scores 

Group  Measure  of  Knowledge  Organization 


Similarity  Scores 
N M SD 

Expert  8 0.1353  0.0553 

Novice  8 0.1035  0.0161 


Relatedness  Scores 
N M SD 

7 0.0659  0.0421 

8 0.0322  0.1122 


To  determine  if  the  similarity  and  relatedness  scores  of  experts  were  more  similar  to  each  other  than  they  were  to  novices,  the 
study  employed  a multivariate  analysis  of  covariance  (MANCOVA).  Novice/expert  designation  acted  as  the  independent 
variable.  Similarity  and  relatedness  scores  acted  as  the  dependent  variables.  As  coherence  scores  were  used  to  operationally 
define  groups  and  represented  a source  of  variation  beyond  the  control  of  the  researchers,  they  were  used  as  a covariate  in  the 
analysis.  To  control  for  Type  I error,  alpha  was  set  to  .05. 

The  results  of  the  MANCOVA  indicated  that  the  group  effect  was  statistically  significant  (Wilks’  Lambda  = .544,  f(l,  14)  = 
4.615,  £ = .035).  Eta-squared  (1 -Wilks’  Lambda)  revealed  that  approximately  46  percent  of  the  variance  in  the  linear  combination 
of  the  dependent  variables  was  associated  with  group  differences.  As  summarized  in  Table  5,  tests  of  between-subjects  effects 
revealed  that  group  differences  were  attributable  to  similarity  scores  alone. 

Table  5 

MANCOVA  Results  for  Similarity  and  Relatedness  Scores  as  a Function  of  Group  and  Coherence 


Source Dependent  Variable df  SS MS f 


Covariate 

Relatedness 

1 

0.0035 

0.0035 

.439 

(Coherence) 

Similarity 

1 

0.0065 

0.0065 

5.820* 

Group 

Relatedness 

1 

0.0065 

0.0065 

.816 

Similarity 

1 

0.0110 

0.0110 

9.803* 

Residual 

Relatedness 

12 

0.0952 

0.0079 

Similarity 

12 

0.0135 

0.0011 

Total 

Relatednes  s 

15 

.137 

Similarity 

15 

.249 
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*p<0,05 


Discussion 

Coherence  and  HPT  Expertise 

Although  the  number  of  years  participants  had  been  a HPT  practitioner  was  expected  to  weakly  correlate  with  participants’ 
coherence  scores,  it  did  not.  Although  the  sample  exceeded  Ericsson’s  (1996)  threshold  of  ten  years  to  reach  expert  status  in  a 
domain,  no  correlation  with  coherence  scores  or  any  other  measure  of  expertise  was  found.  Since  the  number  of  non-juried/juried 
presentations  and  publications  are  based  upon  recognition  in  the  field  and  the  results  of  peer  review,  it  was  thought  that  these 
indirect  measures  of  HPT  expertise  would  correlate  with  coherence  scores.  Although  these  measures  correlated  among 
themselves,  they  did  not  correlate  with  coherence;  book  production  alone  correlates  with  coherence. 

Three  possible  explanations  for  the  significant  relationship  between  coherence  scores  and  authoring  an  HPT-related  book 
include:  1)  desire  to  create  detailed  conceptual  structures  or  schemas,  2)  tolerance  for  tedium,  and  3)  low  sample  size.  The 
correlation  between  coherence  scores  and  writing  an  HPT-related  book  could  be  the  result  of  the  capacity  or  desire  of  an  author 
to  develop  highly  detailed  and  organized  cognitive  structures  or  schemas.  Books  are  representations  of  schemas,  divided  into 
chapters  and  sections.  Perhaps  book  authors  possess  such  schema,  making  it  more  likely  that  their  cognitive  maps  would  correlate 
with  coherence  scores. 

Another  conceivable  reason  for  the  correlation  between  book  authorship  and  coherence  scores  may  be  book  authors’ 
tolerance  for  tedious  activity.  Authoring  depends  on  patience,  self-discipline,  and  a large  devotion  of  time  and  energy.  Authoring 
also  requires  one  to  switch  attention  between  small  details  and  large-picture  views  frequently.  The  structure  of  this  study’s  pair- 
wise rating  activity  required  the  same  sort  of  mental  activity  for  an  extended  period  of  time  (between  30  minutes  and  one  hour). 
Perhaps  individuals  who  were  able  to  complete  the  ratings  and  stay  on  task  for  this  time  are  also  individuals  who  can  author 
books. 

Finally,  the  low  sample  size  employed  in  the  study  may  have  precluded  obtaining  significant  results.  In  addition  to  artificially 
increasing  alpha,  low  sample  size  may  have  lacked  adequate  statistical  power  to  detect  otherwise  significant  relationships 
between  coherence  scores  and  measures  of  HPT  expertise.  Replicating  this  study  with  a larger  sample  and  modifications  to  the 
website  to  decrease  experimental  mortality  (that  is,  the  number  of  participants  who  dropped  out  of  the  study  between  completing 
the  information  page  and  their  concepts)  could  produce  less  ambiguous  findings. 

Organization  of  HPT  Expertise 

The  expert  concept  map  is  similar  to  other  expert  representations  of  HPT,  suggesting  the  convergent  validity  to  this  finding. 
The  information  and  resource  components  of  the  human  capital  branch  of  the  concept  map  roughly  correspond  to  the 
organizational  environment  of  Stolovitch  and  Keeps’  (1999)  conceptual  representation  of  HPT,  The  business  case  branch 
corresponds  to  their  components  of  business  goals/objectives  and  internal  requirements.  Where  Stolovitch  and  Keeps  focus  on 
accomplishments  and  their  verification,  the  expert  concept  map  focuses  on  results.  Additional  similarities  can  be  found 
comparing  the  expert  concept  map  to  Stolovitch  and  Keep’s  procedural  representation  of  HPT,  The  business  case  branch  of  the 
concept  map  addresses  Stolovitch  and  Keeps’  steps  for  identifying  business  and  performance  requirements.  Likewise,  the  HPT 
branch  of  the  concept  map  addresses  the  steps  “define  performance  gaps,  specify  gap  factors,  and  select  interventions,” 

One  of  the  underlying  principles  of  the  HPT  landscape  model  (Addison,  2001)  is  a focus  on  results  and  outcomes.  The  expert 
concept  map  also  communicates  a focus  on  results,  which  is  the  key  concept  on  the  map.  Sections  related  to  ISD,  human  capital, 
business  cases,  and  HPT  are  linked  to  obtaining  results.  The  systems  and  business  case  portions  of  the  Addison  model  are  similar 
to  the  ISD  systems  and  business  case  branches  of  the  expert  concept  map.  The  predictive  validity  of  the  expert  concept  map  could 
be  determined  by  determining  the  extent  to  which  it  predicts  performance  in  HPT,  For  example,  concept  ratings  could  be 
obtained  from  previous  winners  of  ISPI  awards.  These  scores  could  be  compared  against  a control  group  who  did  not  win  such 
awards.  Subsequent  statistical  analysis  could  determine  the  extent  to  which  Pathfinder-derived  relatedness,  similarity,  and 
coherence  scores  predicted  the  award-winners.  In  addition  to  providing  a unique  perspective  of  HPT  as  a discipline,  practitioners 
and  researchers  could  employ  the  concept  map  in  different  ways.  For  practitioners,  the  expert  concept  map  can  assist  in  the 
creation  of  HPT  competencies.  The  different  branches  of  the  map  could  represent  the  major  HPT  competencies.  The  organization 
of  subordinate  and  related  competencies  could  also  be  derived  from  the  concept  map. 

Experienced  practitioners  could  use  the  concept  map  as  a “mind  tool”  to  introduce  new  practitioners  to  HPT,  Specifying  the 
nature  of  the  links  could  help  learners  construct  their  knowledge  of  the  discipline  (Jonassen,  2000),  The  similarity  measure  that 
Pathfinder  derives  from  the  concept  map  could  also  be  used  as  an  assessment  tool  for  practitioners  and  researchers  alike.  As  new 
practitioners  gain  experience  solving  increasingly  difficult  problems  in  the  domain,  their  similarity  scores  should  increase, 
indicating  that  their  mental  map  of  HPT  is  increasingly  like  those  of  experienced  practitioners.  Their  development  within  this 
community  of  practice  could  be  modeled  statistically,  with  different  stages  of  practitioner  HPT  development  lending  themselves 
to  different  learning  and  other  types  of  performance  supporting  interventions.  If  such  stages  and  interventions  could  be 
determined,  then  whole  programs  could  be  designed  to  help  novices  move  along  a continuum  of  HPT  expertise,  potentially 
improving  the  quality  of  ffPT  practice  while  decreasing  the  time  required  obtaining  expert-like  performance, 

Novice/Expert  Differences  in  the  Organization  of  HPT  Knowledge 

The  similarity  scores  of  experts  are  higher  than  those  of  novices,  indicating  that  expert  concept  maps  are  more  like  those  of 
other  experts  than  of  novices.  This  finding  replicates  the  results  of  other  studies  comparing  the  organization  of  expert  and  novice 
cognition.  Specifically,  experts  share  a greater  proportion  of  the  links  in  their  individual  concept  maps  with  those  of  the 
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“averaged  expert’s”  concept  map  (p.  8)  than  do  novices  (14  percent  versus  10  percent,  respectively).  It  should  be  noted,  however, 
that  the  small  sample  used  in  this  analysis  may  not  provide  replicable  results. 

Conclusion 

While  the  results  obtained  in  this  study  are  intriguing  and  suggest  potential  uses  of  the  expert  concept  map  and  related 
Pathfinder-based  measures  in  HPT,  these  results  are  based  upon  a small  sample.  Replication  with  a larger  sample  is  certainly 
warranted. 
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Abstract 

This  study  is  to  compare  the  effect  of  two  learning  strategies:  summarting  and  structured  questions  on  near  and  far  transfer 
tasks.  The  hypothesis  is  that  the  structured  questions  will  better  activate  metacognitive  and  critical  thinking  skills  than 
summarizing  so  that  it  will  better  facilitate  learning  transfer.  However,  the  results  do  not  show  any  significant  difference  between 
two  strategies  on  transfer  tasks.  Possible  reasons  that  the  students  failed  to  activate  metacognitive  skills  or  critical  thinking  skills 
are  discussed.  Recommendations  to  maximize  the  effect  of  structured  questions  on  transfer  tasks  are  also  discussed. 

Introduction 

Various  studies  (Sternberg  & Frensch,  1993;  Salomon  & Perkins,  1989;  Perkins  & Grotzer,  1997;  Kosonen  & Winne,  1995; 
Stolovitch  & Yapi,  1997)  have  been  conducted  in  the  field  of  educational  research  in  general,  and  instructional  systems  design  in 
particular,  on  how  to  facilitate  transfer  of  learning.  The  early  research  on  learning  transfer  focused  on  the  role  of  practice.  Most 
early  researchers  agreed  that  increased  amounts  of  practice  would  increase  transfer  (Thorndike,  1913;  Gagne  & Foster,  1949; 
Hovland,  1951).  More  recently,  one  major  interest  is  the  role  of  metacognitive  strategies  and  critical  thinking  in  learning  transfer. 
Most  of  the  research  focused  on  teaching  metacognitive  strategies  and  critical  thinking  skills.  However,  relatively  little  research 
has  focused  on  the  activation  of  metacognitive  strategies  and  critical  thinking  skills  that  ultimately  promotes  learning  transfer 
(Forster,  1996). 

This  study  explores  the  possible  way  to  activate  metacognitive  strategies  and  critical  thinking  skills  through  the  use  of 
reflective  activities,  like  summarizing  or  answering  structured  questions  after  reading.  Metacognitive  strategies  refer  to  the 
regulation  and  control  of  cognition.  Despite  differences,  most  researchers  would  agree  critical  thinking  involves  the  intentional 
application  of  rational,  higher  order  thinking  skills  that  include  analysis,  synthesis,  problem  recognition  and  problem  solving, 
inference,  and  evaluation  (Angelo,  1995).  Structured  questions  in  this  study  refer  to  a set  of  generic  questions  that  are  not 
associated  with  specific  content  of  any  articles.  Also,  this  research  studies  the  effect  of  metacognitive  and  critical  thinking  skill 
activation  on  near  and  far  transfer.  Near  transfer  is  the  application  of  learning  to  situations  similar  to  those  in  which  initial 
learning  has  taken  place.  In  contrast,  far  transfer  is  the  application  of  learning  to  situations  dissimilar  to  those  of  the  original 
learning  events. 

The  purpose  of  this  study  was  to  compare  the  relative  effectiveness  of  structured  questions  and  summarizing  on  near  and  far 
transfer  tasks.  More  specifically,  this  study  was  designed  to  address  the  following  questions: 

1 . Will  students  who  answer  specific  structured  questions  perform  better  on  near 

transfer  task  than  students  who  are  simply  asked  to  summarize  the  content? 

2.  Will  students  who  answer  structured  questions  perform  better  on  far  transfer 

task  than  those  who  only  write  a summary  of  an  article? 

It  was  expected  that  students  who  write  reading  reactions  based  on  structured  questions  would  outperform  significantly  those 
students  who  only  write  a summary  of  an  article.  This  was  because  the  structured  questions  in  this  study  are  higher  order 
cognitive  questions,  which  might  stimulate  the  development  of  students’  cognitive  strategies  beyond  simply  memorizing  content 
(Sanders,  1966).  In  other  words,  the  structured  questioning  format  requires  the  use  of  more  metacognitive  strategies  and  critical 
thinking  skills.  It  requires  not  only  analyzing  skills  but  synthesizing.  Judging,  and  evaluating  skills.  As  a result,  the  students  who 
answer  structured  questions  might  have  a better  retention  of  the  content  while  applying  what  they  have  read  than  those  who  only 
summarize  the  content.  With  the  better  retention,  the  structured  questions  group  might  do  better  on  near  transfer  task. 

It  was  further  expected  that  the  structured  questions  group  will  perform  significantly  better  on  far  transfer  task  than  the 
summary  group.  This  is  based  on  the  fact  that  structured  questions  in  this  study  might  activate  the  application  of  analyzing, 
synthesizing  and  evaluating  skills  that  are  part  of  metacognitive  strategies,  and  critical  thinking  skills.  This  is  a kind  of  skill 
activation  (Salomon,  1979).  According  to  some  researchers  (King,  1992;  Pressley,  Wood,  Woloshyn,  Martin,  King,  & Menke, 
1992),  generic  questions  (or  structured  questions  in  this  study)  are  very  effective  at  promoting  critical  thinking  skills  because  they 
will  induce  inference,  evaluation,  and  verification.  The  writing  of  an  EPSS  (Electronic  Performance  Support  Systems)  product 
evaluation  report,  as  a far  transfer  task,  also  involves  applying  synthesizing  and  evaluating  skills.  In  contrast,  although 
summarizing  may  also  involve  cognitive  strategy  activation,  most  people  will  focus  on  content  learning  when  they  summarize 
(Gamer,  1987). 
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Method 

Participants 

Sixty-six  undergraduate  students  (7  males,  59  females)  from  a class  of  101  students  at  Florida  State  University  participated  in 
the  study  as  a part  of  an  outside  class  assignment  and  a class  activity.  Although  101  students  participated  in  transfer  tasks  (as  a 
class  activity),  only  66  students  submitted  the  outside  class  reading  assignment.  So  the  final  result  was  based  on  the  66  students 
who  participated  in  both  activities  in  this  study.  Most  students  are  sophomores  or  juniors  who  major  in  or  plan  to  major  in 
elementary,  special,  and  early  childhood  education.  All  the  students  were  enrolled  in  a course  entitled  Introduction  to  Educational 
Technology.  Some  students  may  know  of  the  term  CBT  (Computer-Based  Training),  but  they  had  never  been  exposed  to  the  idea 
of  EPSS  (Electronic  Performance  Support  Systems),  the  content  topic  of  this  study.  The  students’  participation  in  this  study  was 
assigned  as  part  of  their  class  activities.  They  did  not  get  any  extra  credit  points  for  participating,  and  their  participation  in  this 
research  study  did  not  affect  their  grade  or  any  negative  consequences. 

Materials 

Two  articles  about  Electronic  Performance  Support  Systems  (EPSS)  were  used  for  the  reading  assignment.  EPSS  is  an 
interactive  computer-based  environment  that  provides  on-the-job  and  just-in-time  support  to  facilitate  task  performance  and 
product  development.  It  usually  consists  of  a library  and  information  support  system,  customized  tools,  learning  function,  and  an 
interactive  expert  system.  The  first  article  is  Leighton’s  (1996)  What  is  an  EPSS"}  It  introduces  the  background,  definitions,  goal, 
and  components  of  EPSS.  The  other  is  Sleight’s  (1993)  What  is  Electronic  Performance  Support  and  What  Isn't?  In  the  article* 
Sleight  identifies  major  characteristics  of  EPSS  and  compares  examples  and  non-examples  of  EPSS.  Both  articles  are  four  pages 
long  with  about  1500  words. 

The  EPSS  product  the  students  evaluated  for  the  far  transfer  task  is  the  Florida  Curriculum  Planning  Tool  (CPT)  version  2.0. 
CPT  was  designed  and  developed  by  the  Center  for  Performance  Technology  at  Florida  State  University  for  the  Florida 
Department  of  Education  (DOE).  The  purpose  of  CPT  is  to  facilitate  teachers'  planning  of  learning  activities  and  units  of 
instruction  that  are  in  compliance  with  Florida  Sunshine  State  Standards.  It  is  designed,  in  particular,  to  assist  teachers  in 
developing  learning  activities  for  any  subject,  grade  level,  strand,  theme,  standard,  benchmark  or  any  combination  of  the  above.  It 
also  allows  teachers  to  import  activities  developed  by  other  teachers,  and  export  activities  they  develop  to  share  with  others. 

Independent  Variable 

The  independent  variable  in  this  study  is  the  type  of  reading  assignment,  which  had  two  formats.  One  format  requires  the 
student  to  write  reading  reactions  based  on  the  given  structured  questions.  These  questions  include: 

• What  are  your  gut  reactions  to  the  article? 

• What  do  you  think  are  the  big  ideas  in  this  article? 

• What  are  the  implications  for  your  (future)  teaching  or  work? 

• What  are  the  implications  for  your  learning? 

• What  questions  does  it  leave  unanswered? 

• What  is  your  rating  of  the  usefulness  of  this  article?  Choose  one  of  the  following;  (poor,  fair,  good,  excellent) 

The  other  format  requires  students  to  write  a summary  of  the  article  in  a free  format. 

Dependent  Variables 

The  two  dependent  variables  are  near  transfer  and  far  transfer.  The  measurement  of  near  and  far  transfer  is  the  score  on  the 
writing  of  a list  of  EPSS  evaluation  criteria  (near  transfer)  and  the  score  on  the  CPT  evaluation  report  (far  transfer).  The  score  of 
the  EPSS  evaluation  criteria  is  determined  by  the  number  of  evaluation  standards  that  are  included  in  the  student’s  evaluation 
based  on  the  ideas  from  the  two  EPSS  articles.  The  total  possible  score  is  15,  representing  fifteen  major  points  mentioned  in  the 
two  articles  about  the  characteristics  of  an  EPSS.  The  score  on  the  CPT  evaluation  report  is  determined  by  the  following 
assessment  criteria.  These  criteria  are  adopted  and  modified  from  the  principle  of  authentic  writing  assessment  (Chapman,  1990; 
Hart,  1994;  Frederiksen  & Collins,  1989).  The  authentic  writing  assessment  examines  a student’s  writing  holistically.  One  major 
characteristic  of  this  method  is  that  the  students’  total  scores  consist  of  scores  representing  different  dimensions  which  include 
content,  organization,  style,  and  mechanics.  The  formal  writing  assessment  usually  emphasizes  writing  style,  mechanics  of 
spelling  and  syntax,  and  content.  However,  in  this  study,  the  focus  was  on  the  content.  This  is  because  the  content  in  the  final 
evaluation  report  should  reveal  how  well  they  have  learned  from  the  reading  assignment,  and  whether  any  transfer  has  occurred. 
Four  dimensions  are  used  to  score  the  evaluation  report.  They  are  listed  below. 

1.  Focus:  the  main  idea,  theme,  or  point  of  view  is  clear  and  consistently  maintained,  overall  recommendation  about  the 
product  included.  (0-3) 

2.  Support:  arguments  and  conclusions  are  adequately  supported  and  explained.  (0-3) 

3.  Organization:  the  logical  flow  of  ideas  is  clear  and  related.  (0-3) 

4.  Content:  Number  of  major  points  mentioned  in  the  EPSS  articles  included.  (0-6) 

On  the  basis  of  the  above  definitions,  more  detailed  scoring  principles  were  developed  for  each  dimension.  A three-point 
scale  was  used  for  the  first  three  dimensions  of  Focus,  Support,  and  Organization.  A six-point  scale  was  used  for  the  dimension 
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of  Content  because  the  content  was  the  key  factor  that  was  associated  with  the  transfer  of  learning  in  this  study.  Thus,  the  total 
possible  score  for  the  far  transfer  task  is  1 5 points. 

Procedure 

At  the  beginning  of  the  course,  the  students  were  told  about  the  study.  The  participants  were  randomly  assigned  to  one  of  two 
groups;  one  was  told  to  write  reading  reactions  based  on  the  structured  questioning  format,  the  other  was  told  to  write  summaries 
of  the  articles  in  a free  format.  The  experiment  began  in  the  third  week  of  class.  Both  groups  were  asked  to  read  two  articles  on 
the  topic  of  EPSS,  and  then  either  answer  structured  questions  or  write  a summary  as  an  outside  class  assignment.  They  were 
required  to  submit  their  reading  assignments  at  the  beginning  of  the  fourth  week  class.  During  the  fourth  week  of  class,  the 
students  were  required  to  do  two  tasks  in  class,  in  an  hour,  without  access  to  other  materials.  The  first  task  was  to  write  a list  of 
an  evaluation  criterion  for  evaluating  EPSS  products  based  on  what  they  had  read.  The  second  task  was  to  write  an  evaluation 
report  of  the  CPT.  The  students  finished  these  two  tasks  in  front  of  the  computer  in  the  computer  lab.  They  were  asked  to  browse 
the  CPT  first  and  write  down  their  evaluation  in  essay  format.  The  evaluation  criteria  and  report  were  collected  by  the  researcher 
after  the  class. 

The  students’  evaluation  reports  were  scored  by  the  researcher  and  another  doctoral  student  whose  major  is  measurement 
and  testing  in  the  College  of  Education.  Both  raters  have  experience  in  teaching  and  writing  assessment.  The  following  procedure 
was  used  in  scoring  the  students  evaluation  criteria  and  the  evaluation  report  of  CPT.  Before  starting  the  grading  process,  the 
raters  read  two  EPSS  articles  and  the  exercise  instructions  to  get  familiar  with  the  content  and  purpose  of  this  study,  which  was 
very  important  for  another  rater.  The  researcher  assigned  the  numbers  one  to  sixty-six  to  participants  for  scoring  purposes. 
Neither  rater  knew  the  student’s  treatment  group  when  grading  his  or  her  writing  so  as  to  avoid  any  bias  to  a specific  group. 
Based  on  the  scoring  guide  for  grading,  the  two  raters  tried  to  score  three  evaluation  reports  together  as  a test.  They  then 
discussed  the  differences  and  came  to  a consensus  for  further  grading.  Finally,  the  two  raters  finished  grading  all  sixty-six 
evaluation  reports  individually.  To  ensure  the  reliability  and  consistency  between  the  two  raters,  correlation  coefficients  for  both 
scores  were  calculated  using  the  statistical  software  package,  SPSS.  The  results  are  shown  in  Table  1.0.  Pearson’s  correlation 
coefficients  for  the  two  scores  are  .80  and  .73.  They  are  both  statistically  significant  at  the  .01  level. 

After  finishing  the  scoring,  the  researcher  averaged  the  two  raters’  given  scores  as  the  final  score  for  each  student.  Two 
scores  of  each  student  on  near  and  far  transfer  tasks  were  then  input  into  the  SPSS  data  worksheet  for  data  analysis. 

Table  1.0  Correlations  Between  Two  Raters  on  Transfer  Task  Scores 


Correlation 

Rater 

A 

B 

Evaluation  Criteria 

A 

1.00 

0.80 

B 

0.80 

1.00 

Evaluation  Report  of  CPT 

A 

1.00 

0.73 

B 

0.73 

1.00 

Note.  Correlation  is  significant  at  0.0 1 level. 

Results 

Relative  Effectiveness  on  Near  Transfer  Task 

The  score  of  each  student’s  performance  on  near  transfer  task  was  measured  by  the  number  of  key  points  included  in 
their  list  of  evaluation  criteria.  The  possible  total  score  is  15,  which  represents  15  key  points  mentioned  in  the  two  EPSS  articles. 
Table  2.0  presents  the  means,  standard  deviations,  and  ran^  of  scores  for  the  students’  EPSS  evaluation  criteria.  The  average 
score  of  the  structured  questions  group  = 3.35)  was  lower  than  the  average  score  of  the  summary  group  = 3.85),  but  the 
difference  was  not  statistically  significant,  F(l,  64)  = 2.257,  p.>  .05. 

Table  2. 0 Mean  Scores  of  Writing  a List  of  Evaluation  Criteria 


Measure 

Group 

N 

M 

SD 

Range 

Evaluation  Criteria 

Structured  Question 

34 

3.35 

1.00 

2.00-7.00 

Summary 

32 

3.86 

1.68 

1.00-9.00 

Total 

66 

3.60 

1.38 

1.00-9.00 

Note.  The  possible  total  score  for  evaluation  criteria  is  15  points. 
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Relative  Effectiveness  on  Far  Transfer  Task 

The  student’s  performance  on  the  far  transfer  task  was  determined  by  their  evaluation  report  of  Curriculum  Planning  Tool 
(CPT).  The  evaluation  report  was  in  essay  format.  The  scoring  criteria  included  four  dimensions,  focus  (3  points),  support  (3 
points),  organization  (3  points),  and  content  (6  points)  for  a total  possible  score  of  15  points.  The  means,  standard  deviations,  and 
range  of  scores  are  displayed  in  table  3.0.  Again,  the  summary  group  outperformed  the  structured  question  group.  The  average 
score  of  the  summary  group  (M  = 9.23)  is  a little  higher  than  the  average  score  of  the  structured  question  group  (M  = 8.94). 
However,  the  difference  is  not  statistically  significant.  The  result  of  a one-way  ANOVA  showed  F(l,  64)  = .19,  g.  = .664.  With 
£^>.05,  the  null  hypothesis  could  not  be  rejected. 


Table  3.0  Mean  Scores  of  the  Evaluation  Report  of  CPT 


Measure 

Group 

N 

M 

SD 

Range 

Evaluation  Criteria 

Structured  Question 

34 

8.94 

2.69 

3.00-14.50 

Summary 

32 

9.23 

2.77 

5.00-15.00 

Total 

66 

9.08 

2.71 

3.00-15.00 

Discussion 

The  discussion  will  focus  on  the  factors  that  might  have  affected  the  transfer  tasks  as  a whole,  since  the  possible  reasons  for  a 
lack  of  statistical  significance  should  be  similar  for  both  the  near  transfer  task  and  the  far  transfer  task. 

Although  there  were  no  previous  studies  that  directly  measured  the  effects  of  structured  questions  on  transfer  tasks,  many 
studies  (Applegate,  Quinn,  & Applegate,  1994;  Sanacore,  1984;  Andre  & Anderson,  1978;  Kosonen  & Winne,  1995;  Nisbett, 
1993;  Perins  & Grotzer,  1997)  did  suggest  that  metacognitive  skills  and  critical  thinking  skills  facilitate  better  learning.  The 
structured  questions  used  in  this  study  were  supposed  to  activate  metacognitive  skills  and  critical  thinking  skills.  Thus, 
theoretically,  the  forced  use  of  such  skills  as  analyzing,  synthesizing  and  evaluating  in  answering  structured  questions  should 
have  promoted  reading  comprehension  and  produced  better  understanding  in  applying  knowledge  and  skills  obtained  through 
reading  articles. 

Contrary  to  what  was  expected  by  theoretical  assumptions,  the  structured  questions  group  did  not  perform  any  better  than  the 
summary  group  on  both  near  transfer  task  and  far  transfer  task.  The  lack  of  a significant  effect  may  be  due  to  a number  of  factors: 
failure  to  activate  the  students ’metacognitive  skills  or  critical  thinking  skills,  the  lack  of  student  motivation  to  the  experiment, 
inadequate  practice  for  the  treatment,  and  the  unfamiliar  topic  area  with  a lot  of  technical  jargon  in  the  articles  used  for  the 
current  study. 

Failure  to  Activate  Metacognitive  Skills  or  Critical  Thinking  Skills 

There  are  several  possibilities  that  the  students  failed  to  activate  metacogn  itive  skills  or  critical  thinking  skills.  One 
possible  explanation  is  that  most  students’  answers  to  structured  questions  were  about  the  article  rather  than  the  reflective 
thinking  on  the  content  of  the  article.  The  purpose  of  using  structured  questions  in  the  current  study  was  to  force  the  student  to 
apply  critical  thinking  skills  such  as  analyzing,  synthesizing  and  evaluating  skills  to  reflect  on  the  implications  of  the  content  of 
the  articles  in  their  future  learning  or  teaching.  However,  while  answering  questions  like  “what  are  your  gut  reactions  to  this 
article?”,  “what  are  the  implications  for  your  future  teaching?  and  “what  are  the  implications  for  your  learning?”,  most  students 
failed  to  recall  details  of  the  content  of  EPSS  such  as  its  components  and  functions,  but  rather  focused  on  their  general  feelings. 
Those  general  feelings  were  emotional,  and  many  were  negative.  For  example,  several  students  expressed  their  concerns  that  the 
computer  would  replace  humans  because  of  EPSS,  which  was  not  what  they  wanted  to  happen.  Others  complained  there  were  too 
many  technical  terms  in  the  articles  that  were  difficult  for  them  to  understand  at  the  beginning. 

On  the  other  hand,  the  summary  group  at  least  applied  their  own  metacognitive  strategies  to  summarize  the  main  idea  and  its 
supporting  ideas.  The  group’s  sole  focus  on  the  content  might  have  helped  them  recall  the  content  of  articles  when  they  were 
required  to  write  the  list  of  evaluation  criteria  for  EPSS  product  evaluation. 

Another  possibility  of  the  failure  of  skill  activation  is  that  the  structured  questions  in  this  study  might  have  been  too  difficult 
to  these  students.  This  was  inferred  from  an  analysis  of  the  students’  answers  to  implication  questions.  Most  of  them  centered 
around  describing  the  content  of  the  articles  rather  than  the  implications  for  practice.  If  the  students  did  not  understand  what 
inferences  are,  or  simply  did  not  answer  the  question,  it  is  unlikely  the  structured  questions  had  the  desired  effect. 

Lack  of  Students’  Motivation  to  the  Experiment 

Participation  in  reading  assignment  phase  of  this  study  was  voluntary  though  the  participation  of  transfer  tasks  was  required. 
However,  the  participation  of  the  study  had  no  negative  effect  on  students’  grade.  The  students  did  not  get  reward  for  performing 
or  not  performing  on  the  tasks.  That  might  be  the  reason  why  only  around  one  third  of  101  students  submitted  their  reading 
assignments.  The  completion  of  the  reading  assignment  of  writing  summaries  or  answering  structured  questions  happened  outside 
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classes.  The  time  the  students  spent  finishing  the  assignment  and  the  efforts  they  made  were  unknown.  However,  by  reading  their 
answers  to  the  questions  and  summaries,  it  was  somewhat  obvious  that  many  students  did  not  put  much  effort  into  either 
answering  the  structured  questions  or  writing  summaries.  Most  students  answered  the  structured  questions  with  only  one  or  two 
sentences  or  even  left  one  question  unanswered.  Similarly,  the  summaries  of  most  students  were  short  with  one  paragraph  of 
about  130  words.  This  may  result  in  low  retention  of  the  content  and  poor  performance  on  near  transfer  task  among  students  of 
both  groups. 

Another  inference  is  that  the  students  did  not  spend  the  time  to  learn  or  perform,  since  participation  did  not  affect  their  grade. 
Perhaps  they  did  not  perceive  the  task  to  be  worth  a lot  of  effort.  This  low  effort  could  be  observed  in  the  time  spent  in 
completing  the  transfer  tasks.  The  transfer  tasks  were  designed  to  be  completed  in  about  40  to  50  minutes,  but  most  participants 
in  this  study  finished  the  tasks  in  30  minutes  or  less.  The  expected  finishing  time  was  based  on  the  average  time  the  students 
spent  in  two  pilot  tests.  The  first  pilot  test  was  conducted  in  the  same  course  during  the  previous  semester.  The  students  who 
attended  the  pilot  test  had  similar  backgrounds  and  the  average  time  they  spent  was  about  40  minutes,  ranging  from  30  to  50 
minutes.  In  another  pilot  test  done  in  a graduate  class,  the  average  time  the  students  spent  was  about  40  minutes. 

Inadequate  Practice  of  the  Treatment 

The  short  time  period  for  the  experiment  might  not  have  provided  the  structured  questions  group  with  enough  practice  on 
answering  structured  questions.  The  current  study  asked  students  to  read  two  articles  on  the  same  topic  within  a week.  The 
students  who  participated  in  this  study  were  not  familiar  with  the  structured  questions  format  for  reading.  Nor  did  they  get 
feedback  as  to  the  accuracy  of  their  responses.  Two  trials  may  not  be  adequate  for  them  to  make  use  of  this  format.  In  previous 
applications  of  this  format  as  used  in  Construe  (Lebow,  Wager,  Marks,  & Gilbert,  1996),  students  received  feedback  after  their 
first  application  which  led  to  better  responses  on  the  second  application.  As  a result,  perhaps,  the  treatment  was  not  optimized. 

Unfamiliar  Topic  Area  of  Reading  Material 

The  participants  in  this  study  had  no  prior  knowledge  of  the  topic  of  EPS S,  and  they  were  still  a bit  confused  after  reading  the 
articles.  The  two  EPSS  articles  used  in  this  study  are  conceptual  and  well  written  with  simple  and  very  concise  language. 
However,  perhaps  because  of  their  conciseness,  they  did  not  provide  examples  to  explain  where  the  EPSS  products  were  being 
used,  and  what  they  looked  like.  This  caused  problems  with  understanding  for  some  students. 

The  limited  prior  knowledge  of  EPSS  might  have  hindered  the  students’  comprehension  of  the  articles  and  weakened  their 
performance  on  the  transfer  tasks.  Participants  in  this  study  were  from  various  educational  majors.  The  course  was  one  of  the 
required  courses  for  educational  major  students  that  help  them  establish  a foundation  of  knowledge  in  educational  technology. 
Most  participants  were  novice  computer  users.  When  the  experiment  was  conducted,  it  was  in  the  early  weeks  of  the  course.  The 
students  were  just  beginning  to  learn  Microsoft  Word,  and  email.  They  had  not  studied  educational  software  evaluation  which 
was  a topic  to  be  discussed  later  in  the  semester,  and  is  related  to  EPSS  product  evaluation.  This  lack  of  basic  knowledge  about 
evaluation  may  have  affected  their  performance  on  the  transfer  task.  A large  number  of  previous  studies  (Hamilton,  1997; 
Machiels-Bongerts,  et  al.,  1995;  Wilson  & Cole,  1992;  Reigeluth,  1983)  have  indicated  that  prior  knowledge  facilitates 
information  processing  and  text  recall.  Without  related  prior  knowledge,  transfer  might  be  difficult  to  occur.  When  means  of  both 
summary  and  structured  questions  group  scores  were  examined,  it  is  evident  that  the  average  score  was  very  low.  For  the  first 
task,  the  total  average  score  was  only  3.60,  which  was  24.0%  of  the  possible  total  score.  The  total  average  score  of  9.08  for  the 
second  transfer  task  was  only  60.5%  of  the  total  points  which  would  also  be  considered  failing  performance.  One  assumption  of 
these  low  scores  is  that  the  acquisition  of  knowledge  never  took  place.  Without  the  retention  of  the  learning  content,  learning 
transfer  would  never  have  been  occurred. 

Suggestions  for  Future  Research 

While  the  hypotheses  were  not  supported,  the  current  study  provides  sufficient  evidence  that  further  research  should  be 
pursued  in  several  areas  for  a better  understanding  of  the  effect  of  structured  questions  on  transfer  tasks.  These  areas  include: 

• How  to  monitor  or  control  student’s  efforts  in  answering  structured  questions 

• How  to  provide  feedback  on  the  adequacy  of  answers  to  structured  questions 

• How  to  provide  adequate  practice  to  better  the  chance  of  transfer 

• How  to  assess  content  acquisition  before  determining  performance  on  transfer  tasks 

Monitor  or  Control  Student’s  Efforts  in  Answering  Structured  Questions 

This  study  did  not  record  the  time  the  students  spent  to  answer  the  structured  questions.  However,  the  students’  short  answers 
to  the  questions  revealed  that  many  of  the  students  might  not  engage  in  thinking  with  metacogiitive  skills  or  critical  thinking 
skills  that  were  expected  to  be  activated.  Without  the  activation  of  the  skills,  the  transfer  of  learning  barely  occurred.  To  assure 
the  occurrence  of  transfer,  future  research  may  include  directions  that  set  the  minimum  length  of  each  answer  to  the  questions, 
and  require  the  answer  to  focus  on  the  content.  Another  adjustment  for  the  research  can  be  to  ask  the  students  to  finish  the 
reading  assignment  in  the  class.  In  this  way,  both  structured  questions  group  and  summary  g'oup  will  have  the  same  amount  of 
time  to  finish  the  assignment.  The  time  each  student  spends  on  the  reading  can  also  be  recorded  and  analyzed. 
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Provide  Feedback  on  the  Adequacy  of  Answers  to  Structured  Questions 

One  recommendation  for  future  research  is  to  provide  feedback  or  a model  to  students  for  their  answers  to  structured 
questions.  The  feedback  should  be  directed  toward  getting  appropriate  and  accurate  answers  to  the  questions  related  to  key  ideas 
and  implications.  The  results  of  this  study  suggest  that  many  students  did  not  make  use  of  the  structured  questions  format  well. 
To  make  sure  the  students  answer  the  questions  more  adequately,  it  may  be  more  appropriate  and  effective  to  provide  each 
student  feedback  on  their  answers  after  the  first  reading,  and  show  them  some  good  examples  of  answers  as  models.  The 
feedback  may  result  in  better  application  of  the  structured  questions  format  and  a better  chance  of  activation  of  metacognitive 
skills  and  critical  thinking  skills. 

Extend  the  Duration  of  the  Experiment 

Tt  is  also  recommended  that  the  duration  of  the  experiment  be  extended.  The  results  of  this  study  indicate  that  students  might 
not  get  used  to  answering  the  structured  questions  after  reading  only  two  articles.  It  might  show  significant  difference  if  the 
students  are  required  to  read  two  more  articles  for  two  more  weeks  rather  than  to  read  two  articles  within  a week.  The  increased 
amount  of  practice  with  the  structured  questions  format  may  increase  the  possibility  of  activating  metacognitive  skills  and  critical 
thinking  skills  that  the  students  have  acquired  before.  The  skill  activation  may  in  turn  generate  better  transfer. 


Assess  Content  Acquisition  Before  Determining  Performance  on  Transfer  Tasks 

It  is  recommended  that  future  researchers  assess  content  acquisition  before  conducting  transfer  tasks.  A test  of  recall  may  be 
conducted  after  students  finish  reading  articles  to  examine  whether  students  acquire  knowledge  or  not. 

Future  research  may  also  select  another  topic  other  than  EPSS.  It  is  recommended  that  participants  not  be  familiar  with  the 
chosen  topic  but  are  interested  in  it  because  they  will  put  in  more  effort  and  participate  in  the  activities  more  actively  if  they 
perceive  something  as  relevant  and  interesting  (Keller,  1983).  It  is  also  recommended  that  the  transfer  task  be  designed  so  that  it 
can  be  assessed  properly,  if  not  easily. 

In  conclusion,  this  study  shows  that  under  the  stated  conditions  both  structured  questions  and  summarizing  strategies  are 
equally  effective.  However,  it  is  likely  that  different  conditions  would  lead  to  different  results.  That  is,  modified  conditions  in 
future  research  such  as  training  on  the  use  of  structured  questions,  a meaningful  and  relevant  task,  and  motivated  participants 
might  change  the  outcomes  of  this  study. 
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Abstract 

An  upper-level  class  of  secondary  education  majors  learned  how  to  moderate  online  asynchronous  computer  conferences 
and  each  student  was  assigned  a week  to  develop  a topic  and  lead  a discussion  on  an  issue  dealing  with  media  and  technology. 
This  paper  details  a structure  of  student  moderated  computer  conferences  that  can  be  used  to  instill  learner  autonomy,  a sense  of 
ownership  of  student  ideas,  and  take  advantage  of  the  reflective  nature  of  the  media 

Introduction 

Few  would  argie  that  interaction  is  an  important  part  of  any  class  regardless  of  whether  the  class  is  face-to-face,  delivered  at 
a distance  or  a combination  of  the  two.  Moore  defined  three  types  of  interaction  found  in  distance  learning:  learner  content 
interaction,  learner  instructor  interaction,  and  learner-learner  interaction.  He  contended  that  learner-learner  interaction  would 
challenge  future  thinking  and  practice  but  noted  that  its  use  would  vary  depending  upon  the  situation,  ages,  experiences,  and 
levels  of  autonomy  of  the  learners  (Moore,  1989).  Learner- learner  interaction  could  be  employed  with  or  without  the  intervention 
of  the  instructor.  Harasim  and  Yung  (1993)  in  discussing  a survey  of  176  teachers  and  learners  on  the  Internet  reported 
differences  found  in  computer-mediated  communication  when  compared  to  face-to-face  instruction.  They  discovered  that  when 
online,  learner- learner  group  interaction  became  more  detailed  and  deeper.  They  also  found  that  personal  communication 
between  learners  increased.  Online  discussion  groups  enable  students  to  develop  rapport  for  each  other  (Powers  & Mitchell, 
1997).  Ahem  (1995)  reported  that  computer  conferences  provide  opportunities  for  authentic  peer-interaction  wherein  students 
will  develop  an  awareness  of  authorship. 

This  case  study  explores  learner-learner  interaction  in  a structured  environment  using  aspects  of  trained  online  moderation  of 
discussions,  ownership  of  knowledge  and  learner  autonomy  in  course  mostly  delivered  face-to-face  with  online  asynchronous 
computer  conferencing  conducted  as  an  enhancement  to  the  class  serving  as  the  majority  of  the  class  homework. 

The  sample 

The  class  studied  was  comprised  of  20  undergraduate  Juniors  and  senior  in  various  fields  of  secondary  education.  The  course 
was  the  only  technology  requirement  in  their  program  of  study.  Since  this  was  a required  course,  the  student’s  level  of  computer 
experience  covered  a wide  range.  Some  students  were  quite  comfortable  with  computers  having  grown  up  with  them  in  their 
homes  and  schools.  Others,  especially  older  non-traditional  students,  had  very  little  experience  with  computers.  At  the  least 
experienced  level,  students  did  have  experience  with  using  word  processing  programs  to  write  papers.  Along  with  a wide  range 
of  computer  competencies,  there  was  Just  as  wide  a range  of  knowledge  regarding  issues  of  technology  and  education. 

The  structure 

All  students  underwent  a two-day  moderator  training.  The  training  was  based  on  a brief  practical  guide  to  moderating  online 
computer  conferences  covering  issues  including,  what  is  a computer  conference  moderator,  preparing  for  the  conference, 
welcome  messages,  and  common  problems  and  solutions  often  found  in  conferences.  The  guide  has  been  used  it  for  both  student 
training  and  faculty  development.  The  training  also  included  extensive  classroom  discussion  on  dealing  with  the  lack  of  non- 
verbal cues  and  finding  agenda  behind  the  words.  Students  were  also  given  clear  written  instructions  directions  covering  what 
was  expected  of  each  as  a conference  participant. 

The  course  syllabus  showed  that  computer  conferencing  was  a large  part  of  assessment.  Moderating  counted  for  10%  of 
assessment  and  participation  counted  for  an  additional  30%.  Contracting  for  participation  has  been  found  to  be  effective.  Often 
students  feel  reticent  to  participate  in  an  unfamiliar  activity  and  if  they  are  not  given  incentive,  they  will  choose  not  to  take  part  in 
the  conference  or  infrequently  post  message  wanting  in  both  quantity  and  quality  (Eastmond,  1995;  Hillman,  Willis,  & 
Gunawardena,  1994;  Murphy  et  al.,  1996).  Conferencing  groups  need  time  to  coalesce  and  if  participation  is  not  mandated,  this 
would  probably  not  happen.  A usual  case  is  that  a conferencing  system  is  ‘made  available*  but  no  more.  Invariably  it  does  not  get 
used. 

The  class  was  divided  into  two  conferencing  groups  of  ten  each.  These  groups  were  maintained  throughout  the  semester.  Ten 
is  a workable  number  since  assuming  full  participation,  a larger  group  would  be  hard  for  students  to  keep  track  of,  and  a smaller 
group  would  tend  to  make  the  participants  feel  as  if  they  always  need  to  be  ‘performing’  (Cifiientes,  Murphy,  Segur,  & Kodali, 
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1997).  The  instructor  modeled  the  first  week  of  conferencing.  Modeling  is  very  important  since  students  new  to  computer 
conferencing,  need  to  know  what  is  expected  of  them  with  as  much  specificity  as  possible.  Modeling  behavior  and  the  expected 
discourse  are  two  concepts  that  cannot  be  emphasized  too  strongly.  After  the  week  of  modeled  behavior,  the  instructor  did  not 
participate  in  the  conferences  further.  The  groups  knew  that  all  messages  were  being  read  by  the  instructor  since  they  were  often 
mentioned  in  class,  but  all  communication  regarding  conferencing  from  the  instructor  was  emailed  to  individual  moderators. 

Students  subscribed  to  the  Educause  listserv  (http://listserv.educause.edu).  This  provided  them  with  briefings  on  issues 
regarding  technology  and  education.  The  information  found  on  the  listserv  was  often  used  to  generate  topics  for  online 
discussion.  Each  student  moderated  cne  week  of  discussion  while  the  other  nine  students  were  participants.  No  topics  were 
assigned;  rather  each  was  negotiated  with  the  instructor.  The  topic  came  from  Educause  or  anything  of  interest  to  the  student  as 
long  as  it  dealt  with  a contentious  issue  of  technology  and  education. 

Welcome  messages 

Moderators  were  responsible  for  crafting  a ‘welcome  message’,  which  would  start  his  or  her  week  of  discussion.  This 
message  usually  contained  a four-part  structure: 

1.  A personable  introduction 

2.  Presentation  of  information 

3.  A minimum  of  four  open  ended  question 

4.  The  start  of  the  issue  oriented  discussion  with  an  invitation  for  participation 

This  welcome  message  was  emailed  to  the  instructor  on  Saturday  for  evaluation.  Suggestions  were  made  concerned  with  how 
to  q)proach  the  week  and  whether  or  not  the  topic  would  work.  From  these  suggestions,  the  message  was  rewritten  and 
resubmitted,  sometimes  a number  of  times,  until  it  was  approved  and  on  Sunday,  it  was  posted  on  the  BlackBoard  course 
management  system  in  a new  forum. 

Minimum  participant  responsibilities 

The  class  discussed  what  was  to  be  considered  a message  of  substance,  and  it  was  decided  that  a good  message  would  be  at 
least  a few  short  paragraphs  in  length,  but  more  important,  it  needed  to  advance  the  conversation.  A message  just  restating  a 
previously  made  point,  or  serving  no  other  purpose  than  agreement  or  disagreement  would  not  be  counted. 

The  responsibilities  required  of  a participant  were  also  quite  defined:  There  messages  a week  were  the  mandated  minimum. 

• The  first  message  must  be  in  response  to  the  welcome  message  within  two  days  after  its  posting. 

• The  second  message  must  be  in  response  to  a participant  response  within  next  two  days. 

• The  third  message  could  be  in  response  to  anything  previously  posted,  or  could  make  a new  point,  but  needed  to  be 
posted  within  the  next  two  days. 

• Further  messaging  was  encouraged  but  not  mandated. 

Responsibilities  of  the  moderator 

The  moderator  was  charged  with  keeping  things  going  while  using  proper  moderation  techniques  including: 

• Promoting  full  participation 

• Using  email  (cc.  to  the  instructor)  to  solve  problems  and  curtail  miscommunication 

• Extending  the  discourse 

• Setting  a tone  of  inclusion 

• Restating  and  ask  for  clarification 

• At  the  end  of  the  week,  weave  a summary  rich  in  quotation  detailing  the  major  points  of  the  week. 

• At  the  first  class  of  the  next  week,  present  the  welcome  message  and  summary  to  the  entire  class 
Since  the  two-conferencing  groups  were  segregated,  half  the  group  did  not  take  part  in  any  given  discussion. 

Findings 

The  conference  held  for  both  groups  the  final  week  of  the  course  was  moderated  by  the  instructor.  The  discussion  was  based 
on  student  impressions  of  moderating  and  participating  in  online  discussions.  What  worked  and  what  did  not?  Was  it  fun?  Was  it 
too  much  work?  Students  were  asked  to  pose  their  own  questions  as  well  to  the  group.  Some  themes  that  emerged  and  seemed  to 
strike  a chord  with  students  were,  that  although  moderating  was  a time  consuming,  many  enjoyed  their  moderator  week  more 
than  those  when  they  were  participants.  Although  it  was  generally  agreed  upon  that  moderating  was  more  work  than  being  a 
group  member,  many  really  enjoyed  the  sense  of  ownership  of  an  engaging  discussion  dealing  with  a topic  of  their  own. 

Nearly  half  of  the  participants  posted  more  (often  many  more)  message  than  the  required  three.  All  the  students,  except  one 
who  couldn’t  see  how  it  could  fit  in  a beginning  math  course,  wrote  that  if  the  equipment  were  available  when  they  teach,  that 
they  would  use  asynchronous  conferencing  in  their  classes. 

Previous  computer  experience  was  not  correlated  to  success  using  computer  conferencing.  This  may  be  due  to  the  fairly 
simple  system  provided  by  BlackBoard,  but  it  may  also  be  related  to  a number  of  comments  say  ing  that  the  experience  is  closer 
to  writing  or  talking  than  it  is  to  running  computer  programs.  The  class,  without  prompting,  set  a tone  of  supportiveness  and 
politeness  in  both  groups.  This  enabled  a deeper  and  more  reflective  discourse  than  found  in  discussions  where  the  moderator 
was  found  to  be  terse  and  confrontational.  The  degree  of  respect,  friendliness,  and  support  found  in  the  messages  just  ‘happened’. 
A space  of  comfort  and  acceptance  was  created  in  both  groups. 
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Conclusion 

Online  asynchronous  conferencing  is  a useful  enhancement  to  face-to-face  classes.  It  enables  discussions  on  topics  that 
would  be  able  to  be  dealt  with  in  as  much  depth  given  the  constraints  of  class  time.  Allowing  students  to  own  their  words  proved 
quite  effective  and  allowed  engaged  and  vibrant  discussions  that  might  have  taken  on  a different  dimension  if  filtered  through  a 
figure  of  authority.  It  promotes  critical  thinking,  as  well  as  writing.  The  inherent  asynchronicity  allows  time  for  reflection  and 
results  in  remarkably  complex  and  insightful  discourse. 
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Abstract 

One  of  the  important  characteristics  of  the  Web-based  courses  is  that  they  utilize  different  Internet  tools  to  close  the  gap 
between  the  instructor  and  the  learner.  Although  many  different  Internet  tools  can  be  used  today  to  close  this  gap,  there  are  a 
number  of  problems  with  the  existing  tools.  In  this  paper,  we  introduce  ICQ  Active  List,  which  has  the  characteristics  of  many 
Internet  tools  that  can  be  used  in  place  of  all  those  tools  to  eliminate  the  problems.  We  also  touch  to  the  issue  of  creating  an 
online  community,  which  is  very  important  in  Web-based  courses,  and  describe  the  principles  we  followed  to  create  such  a 
community.  Most  of  the  principles  and  the  experience  can  be  used  to  create  learning  communities  in  Web-based  courses. 


Introduction 

One  of  the  most  important  characteristics  of  Web-Based  Instruction  (WBI)  is  to  utilize  different  Internet  communication 
tools.  Since  almost  all  of  the  interaction  vital  for  WBI  takes  place  on  these  tools,  effective  use  of  them  is  directly  related  to  the 
success  of  the  courses.  There  a number  of  Internet  communication  tools  that  can  close  the  gap  between  the  instructors  and 
students  of  WBI.  These  tools  are  course  web  sites,  electronic  mail  (e-mail),  electronic  mailing  lists  (or  listservs),  threaded 
discussion  groups  (or  Usenet  newsgroups),  Internet  Relay  Chat  (IRC  or  chat).  Instant  Messaging  (IM),  audio  conferencing  (or 
Internet  telephony),  and  video  conferencing  (Harrison,  1997;  Madjidi  et.  al.,  1999;  Pattis  on,  1999). 


Internet  Communication  Tools 

A course  web  site  is  a combination  of  web  pages  that  can  hold  such  course  related  information  as  course  syllabus,  course 
notes,  course  lectures,  course  resources,  web  links  to  other  useful  web  sites  on  the  Internet,  and  etc.  Students  access  course  web 
sites  through  web  browsers  such  as  Internet  Explorer  or  Netscape  Navigator. 

E-mail  is  the  most  popular  Internet  communication  tool.  An  e-mail  client,  such  as  Microsoft  Outlook  Express,  Microsoft 
Outlook,  or  Netscape  Messenger,  is  enough  to  use  this  basic  communication  technology.  When  someone  sends  you  an  e-mail,  it 
is  kept  on  an  e-mail  server.  When  you  would  like  to  check  for  your  e-mail  messages,  you  use  the  e-mail  client  software  to 
connect  to  the  e-mail  server  and  retrieve  your  e-mail  messages.  For  WBI,  e-mail  can  serve  such  functions  as  the  exchange  of 
ideas,  questions,  and  data.  In  addition,  any  kind  of  relevant  course  materials  such  as  assignments  can  be  attached  to  an  e-mail 
message  for  easy  submission. 

An  electronic  mailing  list  makes  mass  distribution  of  an  e-mail  message  an  easy  task.  Instead  of  sending  the  same  e-mail 
message  to  many  users  one  by  one,  the  message  is  sent  to  a list.  Then,  every  member  of  the  e-mail  list  receives  the  same  message. 
An  e-mail  list  is  a suitable  tool  for  WBI  instructors  for  sending  common  interest  items  to  students  such  as  course  announcements, 
reminders,  additional  web  links  to  other  useful  web  sites,  and  etc. 

A treaded  discussion  group  is  a subject-related  discussion  in  which  participants  post  their  messages  in  a central  place.  This 
central  place  could  be  a Bulletin  Board  System  (BBS),  Usenet  newsgroup,  or  web  forum.  In  this  central  place,  users  can  read 
messages  and  respond  to  others’  messages.  Each  new  topic  becomes  a separate  thread  in  the  discussion.  When  someone  reacts  to 
someone  else’s  message,  it  appears  under  the  original  one.  In  WBI,  this  structure  can  be  used  by  students  for  topical  discussion, 
assignment  submission,  social  interaction,  and  collaborative  working. 

Chat  provides  a text -based  conversation  among  multiple  users.  The  text  messages  are  seen  immediately  by  all  users.  The 
conversation  usually  takes  place  in  virtual  chat  rooms  which  reside  on  chat  server  software.  Users  connect  to  these  chat  servers 
by  using  a client  chat  software  such  as  mIRC.  Chat  rooms  can  be  used  in  WBI  for  such  purposes  as  virtual  office  hour,  real  time 
discussion,  real  time  collaboration,  social  interaction,  and  etc. 

An  instant  messaging  tool  is  a convenient  way  to  see  when  Internet  friends  are  online  and  to  message  them  in  real  time.  It  is 
faster  than  e-mail  because  the  technology  behind  IM  allows  the  message  to  be  pushed  to  the  receiver  in  a fraction  of  time,  which 
can  be  said  almost  instantly.  AOL  Instant  Messenger,  MSN  Messenger  Service,  and  Yahoo!  Messenger  are  examples  of  IM  client 
software  that  can  be  used  between  Internet  users.  IM  tools  can  be  utilized  in  WBI  for  such  instructional  activities  as  virtual  office 
hour,  real  time  discussion,  real  time  collaboration,  social  interaction,  and  etc. 

Audio  conferencing  allows  two  way  voice  communication  via  an  Internet  connection.  With  the  help  of  a technology  called 
Voice  over  Internet  Protocol  (VoIP),  communication  can  be  carried  out  between  two  different  computers,  between  two  different 
physical  phones,  or  between  a computer  and  a physical  phone.  In  all  of  these  methods,  voice  is  converted  to  IP  packets  which 
travel  through  the  Internet  networks.  Since  the  use  of  Internet  is  cheaper  than  the  use  of  phone  lines,  audio  conferencing  is  an 
economical  solution  to  get  WBI  users  together  in  an  audio  conference.  In  WBI,  audio  conferencing  can  be  used  for  such  purposes 
as  virtual  office  hour,  real  time  discussion,  real  time  collaboration,  social  interaction,  and  etc. 

Video  conferencing  requires  the  attachment  of  a small  camcorder  to  the  computer.  With  the  help  of  this  device  a stream  of 
audio  and  video  can  be  transmitted  over  the  Internet  to  the  other  users.  The  advantage  of  video  conferencing  over  chat  is  the 
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presence  of  live  audio  and  picture  of  the  participants  of  the  communication.  Video  conferencing  tools  can  be  used  in  WBI  for 
such  purposes  as  virtual  office  hour,  real  time  discussion,  real  time  collaboration,  social  interaction,  and  etc. 

Instructors  and  students  have  used  these  communication  tools  in  practical  WBI  settings  for  various  purposes.  Branon  and 
Essex  (2001)  found  the  reasons  for  using  synchronous  communication  tools  in  web-based  instruction  environments  as: 

• Holding  virtual  office  hours, 

• Team  decision-making, 

• Brain-storming, 

• Community  building,  and 

• Dealing  with  technical  issues. 

They  also  stated  the  reasons  for  using  asynchronous  communication  tools  in  such  environments  as: 

• Encouraging  in-depth,  more  thoughtful  discussion, 

• Communicating  with  temporally  diverse  students, 

• Holding  ongoing  discussions  where  archiving  is  required,  and 

• Allowing  all  students  to  respond  to  a topic. 

Their  survey  indicated  that  35%  of  the  distance  educators  planned  to  increase  their  use  of  synchronous  tools,  while  65% 
planned  to  increase  the  use  of  asynchronous  tools.  Just  5%  of  them  planned  to  decrease  the  use  of  synchronous  tools  and  only  3% 
planned  to  decrease  the  use  of  asynchronous  tools.  These  numbers  show  that  there  will  be  a growing  demand  in  the  utilization  of 
synchronous  and  asynchronous  tools  in  WBI  environments. 

Classification  of  Internet  Communication  Tools 

Since  there  are  a number  of  WBI  tools  and  technologies  available,  one  of  the  most  important  challenge  for  the  designers  and 
instructors  of  distance  courses  is  to  identify  the  right  tools  for  online  learning  (Levin,  1997),  Dennison  (2000)  uses  the  properties 
of  the  tools  to  classify  them  into  a taxonomy  that  shares  one  or  more  of  the  general  properties.  A taxonomy  is  an  ordered 
hierarchical  list.  The  parts  of  his  taxonomy  are: 

• Number  of  people  communicating:  The  communication  can  be  individual-to-individual,  individual-to-group,  group-to- 
group,  or  group -to-individual, 

• Real-time  communication:  The  tools  can  be  divided  as  asynchronous  and  asynchronous  tools. 

• Permanency  of  communication:  The  tools  can  be  divided  based  upon  whether  messages  on  them  are  permanent  or  non- 
permanent, 

• Automatic  delivery  of  communication:  The  tools  can  be  divided  as  to  if  the  tool  automatically  delivers  the  message 
(pushing)  or  if  the  user  must  request  for  the  message  (pulling). 

While  the  four  properties  above  result  in  a possibility  of  32  combinations,  some  of  the  combinations  cannot  have  realistic 
tools.  Dennison  also  stated  that  at  the  time  of  his  writing,  there  were  no  tools  for  some  of  the  combinations  such  as  Individual-to- 
group,  Permanent,  and  Push  type.  Example  Internet  communication  tools  for  some  of  the  categories  are  depicted  in  table  1. 

Table  1.  Taxonomy  of  Internet  Communication  Tools,  Adapted  from  Dennison  (2000), 


Combination 

Internet  Tools,  Dennison 
(2000) 

Individual-to-individual,  Synchronous,  Permanent,  Pull 

CU-SeeMe 

Individual-to-individual,  Synchronous,  Permanent,  Push 

Talk 

Individual-to-individual,  Asynchronous,  Non-permanent,  Pull 

Voice  mail 

Individual-to-individual,  Asynchronous,  Non-permanent,  Push 

E-mail 

Individual-to-individual,  Asynchronous,  Permanent,  Push 

E-mail 

Individual-to-group,  Synchronous,  Permanent,  Pull 

IRC 

Individual-to-group,  Asynchronous,  Non-permanent,  Pull 

Listservs 

WWW 

Usenet 

Individual-to-group,  Asynchronous,  Non-permanent,  Push 

E-mail 

Individual-to-group,  Asynchronous,  Permanent,  Pull 

Usenet 

Individual-to-group,  Asynchronous,  Permanent,  Push 

E-mail 

Individual-to-group,  Permanent,  Push 

No  tools  in  2000 

Problems  with  the  Internet  Communication  Tools 

In  the  literature,  the  current  problems  with  tools  that  support  WBI  are  found  as  (Harrison,  1997): 

• Finding  appropriate  software  for  students  and  instructors, 

• Access  control  to  keep  outsiders  from  disrupting  class  communication, 

• Training  students  and  instructors  in  the  use  of  various  software  packages,  and 

• Cost  of  acquisition  and  use  of  appropriate  tools. 

This  paper  describes  ICQ  Active  List  (ICQ  AL),  an  Internet  tool  which  can  support  a web-based  distance  education 
community  by  eliminating  all  of  the  problems  stated  above.  It  also  discusses  its  strengths  and  weaknesses  and  presents  strategies 
for  creating  a community  for  a WBI  course. 
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ICQ  and  ICQ  Active  List 

New  forms  of  web  tools  are  becoming  available  everyday.  ICQ  (I  Seek  You)  is  one  of  them,  which  is  an  integrated  set  of 
tools  that  informs  who  is  on-line  at  any  time  and  enables  to  contact  them  at  will.  Although  it  debuted  as  an  IM  tool  with  limited 
capabilities,  users  of  ICQ  can  chat,  send  instant  messages,  files  and  URL’s  to  others,  or  they  just  socialize  with  their  Internet 
friends  while  connected  to  the  Internet.  All  these  functions  are  embedded  into  one  easy  to  use  client  software. 

Version  99b  of  ICQ  client  comes  with  a new  feature,  the  ICQ  Active  List  (ICQ  AL).  Users  can  create  or  join  ICQ 
communities  based  on  a common  interest  and  easily  access  them  from  their  ICQ  client  software.  Once  users  join  an  AL,  they  can 
broadcast  messages  to  all  of  the  members  of  the  list,  receive  events  from  other  list  members,  chat  in  a virtual  room,  post  and  view 
messages  in  a threaded  discussion  group,  e-mail  each  other  and  view  information. 

Users  can  also  become  an  AL  owner,  which  gives  them  total  control  over  the  AL.  They  can  delegate  authority  to  any  member 
of  their  list.  In  order  to  run  an  AL,  Active  List  server  software  needs  to  be  run.  As  long  as  the  AL  server  software  is  running,  the 
AL  will  be  accessible  by  all  ICQ  clients.  Figure  1 , figure  2,  and  figure  3 show  the  interfaces  of  ICQ  AL  server,  ICQ  AL  manager 
and  ICQ  client  respectively.  ICQ  AL  and  ICQ  client  as  an  Internet  tool  for  some  of  the  categories  of  Dennison’s  taxonomy  is 
shown  in  table  2. 
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Table  2.  ICQ  Active  List  as  an  Internet  Tool  for  the  Different  Categories  of  Dennison  *s  Taxonomy. 


Combination 

Internet  Tools 

Individual-to-individual,  Synchronous,  Permanent,  Pull 

E-mail,  chat,  threaded 
discussion,  instant  message, 
audio  conferencing,  and  video 
conferencing  functions  in  ICQ 
AL  and  ICQ  client 

Individual-to-individual,  Synchronous,  Permanent,  Push 

Individual-to-individual,  Asynchronous,  Non-permanent,  Pull 

Individual-to-individual,  Asynchronous,  Non-permanent,  Push 

Individual-to-individual,  Asynchronous,  Permanent,  Push 

Individual-to- group.  Synchronous,  Permanent,  Pull 

Individual-to- group.  Asynchronous,  Non-permanent,  Pull 

Individual-to- group.  Asynchronous,  Non-permanent,  Push 

Individual-to- group.  Asynchronous,  Permanent,  Pull 

Individual-to- group.  Asynchronous,  Permanent,  Push 

Individual-to- group.  Permanent,  Push 

Characteristics  of  the  ICQ  Active  List 

AL  eliminates  most  of  the  problems  of  Internet  communication  tools  depicted  before.  The  most  obvious  advantage  is  that  it  is 
an  appropriate  tool  for  online  communities  and  web-based  courses.  It  includes  many  functions  including  email,  threaded 
discussion,  chat,  instant  messaging,  audio  conferencing,  and  video  conferencing.  It  even  includes  a very  useful  feature  called 
‘Message  of  the  Day’  (MOTD)  through  which  you  can  make  last  minute  or  daily  announcements.  You  can  add  a button  and  an 
Internet  address  (URL)  as  part  of  this  message.  As  soon  as  the  users  are  connected  to  the  AL,  the  message  is  displayed  to  the 
user.  User  may  click  the  button  in  the  MOTD  at  will  and  visit  the  Web  page  associated  with  the  message.  AL  integrates  all  of 
these  functions  into  one  easy  to  use  interface;  therefore  users  perceive  all  of  the  functions  as  a whole.  They  do  not  switch  from 
application  to  application  to  perform  different  tasks. 

AL  works  on  all  Windows  platforms  including  Window  NT,  Windows  2000,  Windows  98,  and  Windows  95.  According  to 
information  presented  at  the  counter.com  web  site,  these  4 windows  platforms  cover  93%  of  all  users  in  October  2001.  Therefore, 
most  of  the  Internet  users  are  able  to  run  AL  on  their  operating  systems. 

AL  provides  a detailed  security  policy  in  which  the  administrator  of  the  AL  can  define  security  at  different  levels.  The 
administrator  might  authorize  the  members  of  the  AL  or  members  can  auto  join  to  the  AL.  When  authorization  is  required,  all 
potential  participants  must  ask  for  authorization  before  joining  to  the  AL  community.  The  administrator  might  or  might  not 
publish  AL  information  and  online/offline  status  on  ICQ  AL  directory  listing.  When  it  is  not  published,  Internet  users  will  not 
able  to  see  your  AL  in  the  AL  directory.  In  case  of  Internet  security  attacks,  the  administrator  may  ban  specific  IP  addresses, 
therefore  not  allowing  access  to  the  AL  from  those  IP  addresses.  The  banning  may  be  done  at  the  user  level  also.  When  you  ban  a 
user,  he/she  will  not  be  able  to  access  to  the  AL.  To  make  handling  security  easier  for  different  people  in  the  same  AL,  the 
administrator  may  create  groups  in  the  AL  and  assign  rights  to  these  groups.  These  rights  can  be  assigned  in  terms  of 
broadcasting,  chatting,  forums,  and  seeing  information  about  the  AL. 

Both  ICQ  and  ICQ  AL  come  with  detailed  help  files.  However,  help  function  needs  to  be  installed  separately  for  ICQ  client. 
AL  help  includes  such  topics  as  introduction  to  AL  server,  registering  an  AL,  components  of  the  AL,  operating  the  AL,  and 
configuration.  In  addition  to  the  help  included  with  the  ICQ  client,  ICQ  Web  site  includes  an  extensive  online  help  site  at  the 
following  internet  address: 

• http://www.icq.com/support/ 

AL  provides  detailed  statistics  for  each  of  the  members.  It  provides  details  on  when  the  user  joined  the  community,  when  the 
user  last  logged  on,  how  many  times  the  user  logged  on  to  the  AL,  how  many  messages  were  posted  by  the  user  to  the  threaded 
discussion  group,  and  how  many  messages  were  broadcasted  by  the  user  to  the  AL  group  (Sending  a message  to  the  AL  is  called 
broadcasting).  You  can  also  get  such  server  statistics  as  the  total  logon  times  since  the  server  was  created,  the  maximum  number 
of  users  who  got  online  at  the  same  time  in  the  AL,  total  number  of  threaded  discussion  group  messages,  the  maximum  number 
of  users  who  used  the  chat,  and  the  number  of  users  in  the  chat  in  real  time.  Additional  statistics  included  are  total  members  of 
the  AL,  number  of  connected  members,  guests,  administrators,  and  online  users  in  real  time.  You  can  make  an  extensive  analysis 
of  the  online  community  with  the  help  of  these  data. 

In  any  AL,  administrative  privileges  can  be  shared  by  the  owner  of  the  AL  with  other  members.  This  provides  an  opportunity 
to  share  the  tasks  related  to  the  administration  of  the  AL.  Remote  administrators  can  manage  the  AL  server  through  their  ICQ 
client,  but  before  doing  that  the  owner  of  the  list  should  add  these  people  to  the  administrators  group. 

You  do  not  need  to  plan  a budget  for  the  use  of  this  software,  because  both  the  AL  server  software  and  ICQ  client  are 
provided  to  the  Internet  community  free  of  charge.  The  AL  server  software  can  be  downloaded  from: 

• ftp://ftp.icq.eom/pub/ICQ_Win95_98_NT4/ICQ2000a/ActiveList.exe 

AL  supports  individualized  messaging.  Users  of  the  AL  can  send  a message  only  and  only  to  the  administrator  of  the  group 
by  using  the  ‘Message  Administrators’  function.  In  this  way  they  can  ask  questions  about  a specific  issue  that  needs  not  to  be 
known  by  the  rest  of  the  online  community. 

On  the  negative  side  is  the  fact  that  the  AL  server  needs  a continuous  Internet  connection  to  be  active.  The  AL  exists  as  long 
as  the  AL  server  software  is  run  and  connected  to  the  Internet.  If  this  does  not  happen,  users  will  not  be  able  to  connect  to  the  AL 
and  the  AL  status  will  be  offline  to  them. 
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Another  problem  might  be  in  the  support  area.  Since  the  software  is  offered  free  of  charge,  you  can  not  contact  the  producer 
and  request  for  a specific  support.  However,  the  help  functions  provided  cover  many  of  the  issues,  so  this  might  not  be  a big 
problem. 

It  is  not  clear  that  the  producer  of  the  ICQ  AL  will  continue  to  release  new‘ versions  or  the  ICQ  AL  will  be  supported  in  the 
new  releases  of  the  ICQ  client.  However,  the  authors  have  operated  an  AL  server  for  more  than  two  years  without  an  interruption. 
Besides,  even  if  a new  version  of  ICQ  client  is  released  which  does  not  support  AL,  users  may  still  use  the  old  version  of  the  ICQ 
client  which  supports  AL.  It  was  experienced  that  the  oldest  versions  of  the  ICQ  client  software  can  still  be  run.  The  producer 
does  not  force  the  users  to  install  the  newest  version  of  the  ICQ  client. 

In  terms  of  technical  difficulties,  there  is  a 400  member  limit  in  the  AL  server.  When  this  limit  is  arrived,  no  other  people  can 
join  to  the  AL.  The  administrator  should  delete  the  non-active  members  to  make  list  open  to  the  new  members.  Most  of  the  time, 
400  member  limit  will  be  enough  for  most  of  the  communities,  especially  if  you  are  using  AL  for  a Web-based  course. 

There  is  a policy  set  by  the  producer  of  the  ICQ  with  the  use  of  ICQ  client.  This  policy  states  that  ICQ  service  is  not  for  use 
by  children  under  13  years  of  age.  According  to  this  policy,  if  it  comes  to  company's  attention  through  reliable  means  that  a 
registered  user  is  a child  under  13  years  of  age,  ICQ  will  cancel  that  user's  account.  Unfortunately,  this  may  limit  the  creation  of 
online  communities,  whose  users  are  under  the  age  of  13. 

The  Context  of  ICQ  Active  List  in  This  Study 

Kim  (1998)  proposed  a model  for  creating  online  communities.  Her  model  contains  9 principles,  which  she  calls  ‘9  Timeless 
Design  Principles’  for  creating  an  online  community.  She  built  this  model  based  on  her  experience  with  designing  online 
communities  for  various  clients.  The  principles  in  the  model  are  helpful  for  addressing  a number  of  social  issues  that  everybody 
encounters  when  they  create  and  administer  an  online  community.  The  principals  in  this  model  are: 

1 . Define  the  purpose  of  the  online  community 

2.  Create  distinct  gathering  places 

3.  Create  member  profiles 

4.  Promote  effective  leadership 

5.  Define  a clear  code  of  conduct 

6.  Organize  events 

7.  Provide  a range  of  roles  such  as  visitors,  new  members,  regular  members,  leaders,  and  etc. 

8.  Facilitate  sub-groups 

9.  Integrate  the  online  environment  with  the  real  world 

Below  is  the  description  of  how  we  applied  these  principles  to  create  a sense  of  community. 

1.  Defining  the  Purpose  of  The  Online  Community 

The  purpose  of  an  online  community  can  be  as  general  as  bringing  users  together  for  a general  discussion  on  a topic  of 
interest.  It  can  also  be  as  specific  as  answering  a question.  In  our  case,  we  chose  ‘music’  as  a general  topic.  Members  of  the 
community  have  used  the  AL  to  discuss  different  genres,  artists  and  music  albums,  to  make  comments  on  these,  to  share  new 
services  and  Web  sites  related  to  the  topic,  and  to  socialize. 

2.  Creating  Distinct  Gathering  Places 

As  soon  as  you  run  the  ICQ  AL  server  and  connect  it  to  the  Internet,  you  have  a virtual  place  on  the  Internet  for  your 
community  to  get  together.  Within  the  ICQ  AL  server  itself,  chat  room  functions  as  an  extra  unique  place  to  gather.  There  is  an 
indicator  on  the  interface  of  the  ICQ  client,  which  shows  the  number  of  the  people  in  the  ICQ  AL  chat  room.  If  members  see  any 
number  here,  they  know  that  there  are  people  in  the  chat  room,  and  they  can  go  there  to  get  together.  Even  if  there  is  nobody  in 
the  chat  room,  they  may  go  to  the  chat  room  and  wait  for  someone. 

3.  Creating  Member  Profiles 

Every  Internet  user  needs  to  create  an  account  to  use  the  ICQ  client.  During  the  creation  of  this  account,  the  user  might 
provide  a wide  range  of  personal  information  including  name,  nick  name,  e-mail  addresses,  postal  address,  gender,  age,  Internet 
homepage,  occupation,  position,  interests,  affiliations,  phone  numbers  and  a picture.  Providing  some  or  all  of  this  information  is 
up  to  the  users.  They  may  update  their  profile  anytime  by  using  their  ICQ  client.  Since  this  profile  information  is  provided  by  the 
members,  administrators  do  not  need  to  enter  this  information  for  each  new  member  of  the  community.  ICQ  AL  software 
associates  the  personal  information  provided  by  the  members  with  the  user  accounts  in  ICQ  AL  server.  Therefore,  the 
administrators  and  other  members  can  easily  get  information  about  the  members  of  the  community  by  using  this  feature. 

4.  Promoting  Effective  Leadership 

During  our  experiences,  it  turned  out  that  creating  a community  is  just  one  side  of  the  coin.  After  the  creation,  you  need  to 
identify  strategies  that  will  foster  communication  in  the  online  community.  Otherwise,  the  communication  will  not  go  beyond 
simple  greeting  messages.  You  would  be  even  lucky  if  someone  would  respond  to  those  greetings.  Whatever  tool  you  use,  either 
chat,  threaded  discussion,  or  instant  messaging,  someone  or  a group  of  members  need  to  take  the  initiative  to  make  the 
communication  ongoing.  Kim  (1998)  examples  this  by  using  the  land  metaphor.  You  can  purchase  a piece  of  land,  but  it  will  not 
automatically  turn  into  a nice  garden.  You  need  to  know  your  needs,  ‘sow  the  seeds  at  the  right  time,  manage  the  growth,  defend 
against  attacks,  and  be  prepared  to  improvise’. 

5.  Defining  a Clear  Code  of  Conduct 

Within  the  ICQ  AL  server,  the  properties  of  the  AL  server  may  be  described  with  a short  description  and  long  description. 
Short  description  is  useful  for  informing  people  in  the  AL  directory.  Long  description  may  be  used  to  define  the  code  of  the 
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conduct  for  the  community.  In  this  area,  we  have  defined  rules  to  follow  for  the  members  of  the  community.  As  part  of  these 
rules,  we  defined  actions  for  inappropriate  behavior.  For  example,  if  a member  has  a bad  mouth  in  the  community,  he  is  banned 
and  he  can  not  access  to  the  community  for  a period  of  time.  New  rules  had  to  be  defined  as  the  community  grew  and  the  number 
of  interactions  increased.  For  example,  we  recommended  members  not  to  accept  any  type  of  files  from  members  that  they  did  not 
know.  We  also  made  the  code  of  conduct  available  on  a Web  site  for  the  community. 

6.  Organizing  Events 

We  organized  several  events  in  our  online  community.  A backgammon  tournament  was  one  of  them.  The  administrators 
made  an  analysis  of  available  tools  for  this  purpose.  Most  of  the  time,  new  tools  are  required  to  organize  such  events.  Therefore, 
emphasize  should  be  put  on  those  tools  that  the  members  of  the  community  will  be  able  to  use  with  minimum  effort  and 
resources.  An  innovative  3D  world  might  sound  good  at  first,  but  you  need  to  make  sure  that  every  member  of  the  community 
will  have  access  to  such  a tool.  It  was  observed  that  this  kind  of  activities  made  the  members  who  participated  in  the  activities 
closer  to  each  other,  just  like  in  the  real  world.  This  kind  of  members  interacted  among  themselves  more  often  and  less  formally. 

7.  Providing  a Range  of  Roles 

Some  of  the  roles  are  already  defined  when  you  run  your  AL.  These  roles  are  administrator  roles,  new  member  roles,  and 
guest  roles.  We  defined  regular  member  roles  in  addition  to  this.  Administrators  are  responsible  for  the  administration  of  the  AL, 
like  running  the  AL,  adding  and  removing  the  members,  maintaining  the  member  database,  and  etc.  By  default,  every  new 
member  of  the  community  is  assigned  to  new  member  role.  This  makes  other  members  differentiate  between  new  members  and 
regular  members.  Guest  role  was  disabled  since  we  wanted  every  participant  of  the  community  to  be  a member  of  the 
community.  Regular  member  role  was  given  to  those  members  who  regularly  logged  on,  broadcasted  messages,  posted  messages 
in  threaded  discussion  group,  and  participated  in  the  chat  room.  Up  to  4 icons  can  be  used  to  differentiate  the  members  in 
different  roles.  The  function  of  these  icons  is  very  important.  By  looking  at  the  icon  of  a member,  another  member  can  easily 
recognize  the  role  of  the  other  member  just  like  identifying  social  status  of  a person  by  looking  at  him  in  the  real  world. 

8.  Facilitating  Sub-Groups 

We  experienced  that  after  a while,  some  of  the  members  would  like  to  focus  on  specific  sub-topics  of  the  main  topic.  For 
example,  in  the  threaded  discussion  group,  different  threads  have  emerged  in  time.  These  threads  included  such  sub-topics  as  the 
properties  of  a quality  guitar,  exchange  of  music  hardware,  comments  on  a particular  song,  and  etc.  We  experienced  that  threaded 
discussion  tool  serves  a good  function  towards  this  principle. 

9.  Integrating  the  Online  Environment  with  the  Real  World 

Since  most  of  the  members  of  our  community  are  scattered  around  the  world,  it  is  almost  impossible  to  gt  together 
physically  with  the  members  of  the  community.  However,  audio  and  video  conferencing  technologies  may  help  closing  the  gap 
between  the  virtual  world  and  the  physical  world.  In  today’s  computers  nearly  every  computer  comes  with  a sound  card.  When 
you  have  a sound  card  in  your  computer,  you  can  use  a software  called  NetMeeting  to  make  voice  conversation  between  2 
different  computers  connected  to  the  Internet.  ICQ  client  successfully  integrated  NetMeeting  into  its  own  structure.  A member 
may  just  select  another  member  in  the  list  and  initiate  a voice  conversation.  Then,  NetMeeting  starts  and  the  connection  is 
established  between  the  remote  computers.  In  our  community,  we  have  made  many  successful  voice  conversations  between 
different  locations  as  far  as  10000  miles  from  each  other.  Most  of  the  time  the  quality  and  pace  of  the  conversation  was  not  very 
different  from  a phone  conversation.  NetMeeting  also  supports  video  conferencing.  However,  almost  none  of  the  members  had  a 
camcorder  required  to  conduct  a video  conferencing. 

External  Evaluation 

We  have  examined  some  external  evaluation  data  related  to  the  ICQ  AL.  Download.com  is  a Web  site  that  distributes 
software.  On  this  site.  Users  may  also  submit  their  opinions  about  various  software  they  use  by  rating  the  features,  usability,  and 
stability  of  the  software  between  bad  and  excellent.  They  may  also  add  their  additional  comments.  As  of  November  2001, 69%  of 
the  users  recommended  ICQ  AL  to  be  used  by  others  (109  votes).  Features,  usability,  and  stability  were  all  rated  as  4,  which 
meant  good  in  the  range. 

Some  of  the  users  made  useful  qualitative  comments.  One  of  the  users,  who  was  positive,  identified  the  software  as  very  nice. 
He  gave  some  technical  information  on  his  usage  and  indicated  that  he  ran  several  AL  servers  flawlessly  on  a computer  with  a 
Pentium  1 33  MHz  processor  and  32  MB  RAM. 

Another  user  made  a very  interesting  comment.  He  indicated  that  his  AL  server  was  running  smoothly  all  the  time  with  no 
problems.  He  commented  that: 

The  downfall  of  ICQ  [AL]  is  its  best  feature.  It  has  tons  of  options.  You  just  have  to  be  smart  enough  to  use  them.  If  you 
are  afraid  of  your  programs  or  you  don't  play  with  them,  then  you  will  never  get  it  right. 

Discussion  and  Conclusions 

We  have  used  the  principles  identified  by  Kim  (1998)  successfully  to  create  an  online  community  on  the  Internet.  This  study 
is  still  in  progress  and  every  new  day  might  result  in  a different  experience.  If  you  can  successflilly  follow  the  9 principles  in  the 
creation  of  your  online  community,  most  of  the  time  members  of  your  community  will  swing  between  the  virtual  world  and  the 
actual  world  just  like  a pendulum  swinging  between  two  different  points  on  a continuum. 

ICQ  Active  List  is  a suitable  and  efficient  tool  in  creating  an  online  community.  And  you  can  use  this  tool  to  create  learning 
communities  for  your  Web-based  courses.  However,  just  like  any  other  tool,  you  need  to  use  components  of  this  tool  at  the  right 
time,  right  place,  and  with  appropriate  strategies.  Providing  gathering  places,  defining  the  rules,  organizing  events,  providing 
roles,  and  facilitating  sub-groups  are  useful  strategies  that  can  give  your  online  community  a character.  However,  they  are  not 
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enough  to  complete  the  recipe.  If  you  top  off  these  with  effective  leadership  then  you  will  have  an  effective  and  functional 
community.  Our  experience  showed  us  that,  most  of  the  people  in  online  communities  are  still  not  used  to  the  freedom  Internet 
has  offered.  They  still  need  leaders  who  will  guide  them. 
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Abstract 

In  this  study  instructor  and  student  problems  related  to  the  Web-Based  Instruction  (WBI)  were  identified  after  examining 
their  experiences.  Interviewing  technique  was  used  to  collect  data.  Semi-structured  interview  questions  were  asked  in  order  to 
identify  problems  related  to  the  design,  technology,  support,  management,  student-centered  learning,  communication,  and  time 
from  the  perspectives  of  students  and  instructors.  Instructors ' problems  were  found  in  the  area  of  support,  communication  with 
students,  providing  feedback,  and  handling  the  number  of  students.  Students’  problems  were  found  in  the  area  of  learning  new 
tools,  communication  with  classmates  and  instructors,  and  social  aspects  of  learning.  Results  indicated  that  meeting  at  least  once 
face-to-face  before  the  course  and  providing  environment  for  an  effective  communication  were  important  parameters  for 
formalizing  the  interaction  between  students  and  instructors  and  also  among  the  students.  In  addition  to  this,  it  is  also  found  that 
fostering  dialog,  participation,  and  interaction,  formative  assessment  of  assignments,  increasing  students  ’ motivation,  giving 
importance  to  student  differences  in  learning,  using  active  learning  techniques,  and  designing  effective  web  sites  will  increase  the 
success  of  the  WBI. 

Introduction 

Using  the  Internet  for  different  purposes  has  entered  a revolution  to  provide  better  communication  among  people  during  last 
two  decades.  Especially,  after  the  development  of  the  hyperlink  on  the  World  Wide  Web  (WWW),  the  Internet  has  offered  more 
user-friendly  environments  (Starr,  1997).  Researchers  indicated  that  WWW  is  not  only  a communication  medium  for  e-mail  and 
document  distribution  but  it  is  also  a place  to  learn  (Lightfoot,  1999;  Mioduser,  Nachmias,  Lahav,  & Oren,  2000).  With  the 
combination  of  the  specifically  designed  software  and  pedagogical  knowledge,  WWW  can  provide  an  educational  environment 
that  maintains  the  knowledge  building  approach  to  learning.  These  understanding,  technological  developments  in  communication 
and  WWW  have  been  used  as  new  opportunities  for  delivering  instruction  online.  Thus,  distance  learning  has  emerged  as  an 
approach  to  education  in  the  last  few  years  (Yellen,  1998).  Especially  for  graduate  students,  universities  have  started  to  offer  their 
courses  online  (Barnard,  1997;  Duchastel,  1997;  Kearsley,  Lynch,  & Wizer,  1995).  These  practices  in  new  educational  agenda 
initiated  some  questions  on  researchers’  mind.  They  started  to  identify  how  effectively  online  teaching  has  been  delivered. 
According  to  Duchastel  (1997),  WWW  have  been  used  mostly  to  support  the  traditional  model  of  university  instruction.  This 
approach  resulted  in  the  loss  of  potential  of  Web  tools  in  teaching.  To  use  the  Web  effectively  in  instruction,  instruction  should 
be  transferred  to  Web  totally  and  given  by  using  web-based  teaching  approaches  and  technologies.  Moreover,  Duchastel  (1997) 
also  argued  that  models  for  transferring  instruction  to  Web  have  not  been  determined  definitely. 

In  addition  to  ways  of  delivering  instruction  online,  researchers  have  also  started  to  investigate  the  characteristics  of  online 
teaching,  which  affect  the  success  of  distance  teaching  (Mclsaac,  Murphy,  Games,  & Igoe,  1989;  Pisik,  1997;  & Trentin,  1997). 
Mclsaac,  et  al.  (1989)  reviewed  62  articles  and  found  that  researchers  have  carried  out  their  studies  mainly  on  two  broad 
categories;  instruction  and  administration.  Under  the  instruction  category,  researchers  studied  learning,  attitude  and  dropout. 
Under  the  administration  category,  cost-effectiveness  and  courseware  design  were  investigated.  According  to  these  researchers 
these  two  main  categories  and  subcategories  are  the  main  characteristics  of  online  teaching.  It  is  emphasized  that  these 
characteristics  should  be  considered  carefully  during  design  and  delivery  of  the  online  courses. 

Berge  (1997)  explored  how  teachers  design  and  deliver  online  instruction  for  adults  in  a post  secondary  school.  It  was  found 
that  student -centered  learning,  slf-reflection,  discussion,  collaboration,  authentic  learning,  and  online  discussion  are  the  most 
important  characteristics  of  the  online  teaching.  These  parameters  determine  the  success  of  the  online  teaching.  According  to 
Sheffield  (1997)  learner  characteristics  and  the  instructional  strategies  such  as  selecting  and  sequencing  events  and  contents  are 
the  most  important  parameters  in  online  teaching.  Since  the  learners’  characteristics  impact  the  choice  of  instructional  strategies 
and  the  subject  matter  of  the  course,  instructors  need  to  consider  the  diversity  among  the  students.  As  opposed  to  traditional 
classroom,  online  classrooms  combine  the  students  whose  cultural  backgrounds  are  very  different  from  each  other.  Teaching 
multicultural  group  is  an  inevitable  aspect  of  the  online  instruction.  Even  though  the  parameters  that  affect  the  success  of  online 
teaching  have  been  investigated  from  different  perspectives,  teaching  through  online  courses  created  different  problems  for 
students  and  instructors.  Lui  and  Thompson  (1999)  investigated  how  teaching  simultaneously  the  same  course  in  both  a distance 
and  a traditional  educational  format  affects  instructors  and  what  the  differences  are  between  the  two  teaching  styles.  It  was  found 
that  even  though  the  instructor  prepared  the  same  teaching  materials  for  distance  and  traditional  classes,  preparation  of  distance 
course  materials  were  more  time  consuming  than  traditional  one.  Moreover,  time  management,  monitoring  students,  and  e-mail 
communication  also  took  most  of  the  instructor’s  time  to  provide  effective  teaching  to  distance  students.  In  addition  to 
instructors’  problems,  researchers  have  investigated  the  students’  perspectives  in  comparing  traditional  teaching  and  online 
teaching.  Results  indicated  that  there  are  some  advantages  and  disadvantages  of  learning  online.  In  terms  of  advantages,  students 
indicated  that  they  have  wait  time  before  answering  the  questions  or  responding  to  the  situations.  In  traditional  classroom  they 
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were  expected  to  give  answer  right  after  the  questions  are  asked.  In  addition,  they  believed  that  with  online  teaching  learning 
goes  deeper  and  broader.  There  is  no  time  constraint.  They  can  study  24  hours  a day.  In  terms  of  disadvantages,  they  found  that 
the  text -based  communication  creates  a sense  of  isolation,  misunderstanding,  suspicion,  and  a lack  of  credibility  (Herman,  1999 
& Mory,  Gambill,  & Browning,  1998).  In  addition  to  these  problems,  Wulf,  & Schinzel  (1998)  investigated  the  effectiveness  of 
videoconferencing,  telelecture,  and  teleleaming.  It  was  found  that  videoconferencing  is  not  sufficient  in  quality  transmission, 
telelecture  reduces  the  students’  attention  and  interactivity,  and  teleleaming  requires  new  effective  instructional  tools.  Ozden  & 
Cagiltay  (2000)  found  that  lecturing  technique  should  not  be  preferred  as  an  instructional  method  in  WBI. 

In  this  study,  we  explored  WBI  problems  from  two  different  approaches  that  are  different  from  the  previous  studies.  First,  in 
the  literature,  it  is  observed  that  researchers  investigated  instructors’  and  students’  problems  separately.  Those  problems  mostly 
related  to  one  type  of  Web-based  course  and  participants  explained  their  problems  related  to  that  course.  In  other  words,  in  each 
study,  researchers  concentrated  on  a single  Web-based  course  and  problems  associated  with  that  course  from  the  students’  or 
instructors’  perspectives.  With  this  study,  we  aimed  to  identify  the  problems  associated  with  different  Web-based  courses  and 
from  both  students’  and  instructors’  perspectives  in  order  to  provide  a more  detailed  and  two  sided  information.  Besides,  we 
identified  that  problems  related  with  support,  learning  new  tools,  feedback,  class  discussions,  and  group  projects  have  not  been 
identified  in  detail.  We  considered  that  these  are  very  important  characteristics  of  WBI  and  should  be  investigated  thoroughly. 

Methodology 

Participants 

In  this  study,  two  instructors  and  two  graduate  students  from  a large  mid-western  university  were  interviewed.  Participants 
from  both  the  instructor  and  the  student  groups  were  selected  with  purposeful  sampling.  Since  the  nature  of  this  study  is 
qualitative,  participants  were  selected  according  to  their  previous  experience  with  the  WBI  to  get  deep  and  broad  understanding 
about  their  problems  related  to  WBI.  It  was  required  for  the  instructor  participants  to  teach  at  least  one  Web-based  course  and  it 
was  required  for  the  student  participants  to  take  at  least  one  Web-based  course.  Instructors  and  students  were  the  members  of  the 
same  university,  but  none  of  the  participants  had  a relationship  through  the  courses. 

Both  of  the  students  were  graduate  students.  They  were  doctoral  students  in  science  education  department.  They  held  full- 
time jobs  as  an  Assistant  Instructor  (AI)  at  the  same  university.  Both  of  them  were  comfortable  with  using  the  computers  and  the 
Internet  both  in  their  courses  and  in  their  daily  lives. 

Instructor!  was  a non-native  speaker  of  English  and  he  was  an  AI  in  the  Informatics  department.  He  taught  a Web-based 
course  three  years  ago.  The  students  took  this  course  within  another  country  while  the  instructor  was  in  US. 

Instructor2  was  a native  speaker  of  English  and  she  was  a faculty  in  an  Instructional  Systems  Technology  (1ST)  department. 
She  had  already  finished  the  first  nine  weeks  of  her  Web-based  course  when  the  interview  was  conducted.  The  students  who  took 
this  course  were  the  online  master  students  and  they  were  seeking  for  a master  degree  through  a distance  master  program  in  an 
1ST  department. 

Web-Based  Courses 

Student  1 took  a Web-based  course  as  an  elective,  which  was  given  in  a semester.  The  main  objective  of  the  course  was  to 
teach  the  ways  of  using  the  Internet  in  K-12  schools.  There  was  one  instructor  teaching  the  course. 

Student2  took  a Web-based  course  as  an  elective  for  her  major  in  science  education  department.  The  course  was  offered  in 
six  weeks  during  the  summer  session.  The  purpose  of  the  course  was  to  teach  incorporating  technology  into  the  instruction.  There 
was  one  instructor  teaching  the  course. 

Instructor!  taught  a course  in  which  the  purpose  was  to  teach  courseware  design  for  computer  mediated  learning.  The  course 
was  at  the  graduate  level  and  10  students  took  the  course.  The  course  was  offered  in  an  instructional  technology  department. 
There  was  another  instructor  for  the  course. 

Instructor2  taught  2 courses.  One  of  the  courses  aimed  to  provide  an  introduction  to  the  field  and  profession  of  instructional 
technology.  The  other  course  provided  information  on  the  instructional  design  process.  Both  courses  were  at  the  graduate  level 
and  they  were  part  of  an  online  master’s  program.  The  courses  were  offered  in  an  instructional  technology  department.  There 
were  1 instructor  and  10  students  in  the  first  course,  2 instructors  and  18  students  in  the  other  course. 

Data  Collection 

Semi-structured  interview  questions  were  used  to  collect  interview  data.  Interviews  were  conducted  over  a one  month  period. 
Each  interview  session  was  treated  as  an  individually  constructed  discourse  between  the  researchers  and  the  participant.  Both 
open  ended  and  probing  questions  were  used  to  get  deep  information  for  categories  determined  before  the  interviews.  Main 
categories  included  problems  related  to  design,  technology,  support,  management,  student -centered  learning,  communication,  and 
time.  In  addition  to  these  categories  four  open-ended  questions  were  asked  independently  to  compare  classroom  teaching  and 
online  teaching.  The  questions  for  the  instructors  were: 

• Which  one  is  difficult  classroom  teaching  or  online  teaching? 

• Do  you  feel  like  facilitator  or  instructor? 

And  the  questions  for  the  students  were: 

• Which  one  is  difficult,  classroom  learning  or  online  learning? 

• Do  you  like  student -centered  learning? 
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Data  Analysis 

The  study  called  for  an  in-depth  understanding  of  the  experiences  of  participants  involved  in  WBL  Instructors’  and  students’ 
data  were  analyzed  separately  from  each  other.  All  data  were  transcribed  from  audiotapes  for  analysis.  Then,  researchers 
struggled  to  understand  the  context,  discourse,  and  meaning  behind  the  participant  responses  to  determine  the  main  areas  about 
which  respondents  have  problems  with  WBI,  It  was  observed  that  participants  did  not  have  problems  about  some  of  the  pre- 
determined categories.  New  categories  for  instructors  were  determined  as  support,  communication  with  students,  providing 
feedback,  and  handling  the  number  of  students.  New  categories  for  students  were  determined  as  learning  new  tools, 
communication  with  classmates  and  instructors,  and  social  aspects  of  learning  from  students’  perspectives.  To  increase  the 
credibility  of  the  study,  participants’  responses  were  coded  by  both  researchers  separately.  Later,  common  themes  accepted  as  the 
new  emergent  categories.  In  addition,  the  data  obtained  from  the  literature  was  used  for  detailed  interpretation  of  the  results. 

Findings 

Interview  results  with  students  and  instructors  will  be  presented  under  miscellaneous  categories.  First,  instructors’  problems 
while  teaching  Web-based  courses  will  be  presented.  Then  students’  problems  will  be  explained  in  detail.  Finally,  we  will 
provide  instructors’  and  students’  preference  in  terms  of  classroom  instruction  or  WBI. 

1.  Instructors’  Problems  with  WBI 

Instructors’  problems  and  difficulties  will  be  summarized  in  the  following  categories:  Support,  communication  with  students, 
providing  feedback,  and  number  of  students. 

1.1.  Support 

Feeling  lonely  and  finding  support  when  they  are  in  need  are  the  biggest  problems  for  the  instructors.  Since  they  do  not  know 
what  kind  of  problems  they  will  have  during  the  course,  it  is  difficult  for  them  to  be  prepared  against  these  problems  in  advance. 
When  they  have  problems,  instructors  need  two  types  of  support:  technological  support  and  design  support  for  the  revision  of  the 
course  web  site.  If  they  do  not  have  both  kind  of  support  on  time,  they  spend  most  of  their  time  to  overcome  these  difficulties 
instead  of  preparing  themselves  for  future  instructions.  Instructor2  indicated  that  she  was  not  able  to  have  a Macintosh  computer 
(which  she  was  used  to  using)  and  required  software  at  the  begi  nning.  Because  of  the  lack  of  technological  support,  she  had  a lot 
of  communication  problems  with  the  students.  She  had  to  provide  feedback  much  later  than  her  planned  time  frame.  She 
explained  the  design  problems  as  the  following: 

Instructor!: ...  The  other  big  and  huge  problem  was  the  sound  in  PowerPoint.  ...So,  you  can  go  through  the  whole  week 
then  look  how  to  fix  it.  That  has  been  very  time  consuming  for  me.  Probably  more  time  consuming  than  any  other  thing 
that  I have  done  for  the  course.  I have  been  doing  the  redesign  of  the  web  pages  and  then  going  through  and  editing  the 
lessons,  [and]  the  visual  parts  of  the  lesson. 

Learning  new  software  also  creates  problems  for  instructors  if  they  learn  them  while  giving  Web-based  courses.  They 
need  support  from  the  people  who  know  these  software. 

Researcher:  How  about  learning  new  software? 

Instructor!:  Yes,  because  I had  to  learn  Dreamweaver  and  I did  not  have  the  application,  so  I needed  to  track  down 
getting  the  application  from  one  of  my  friends.  Even  though  the  documentation  was  written,  it  was  brand  new  program 
for  me.  Thankfully  one  of  my  friends  came  in,  and  spent  about  15  minutes  with  me  kind  of  showing  the  basics  to  me. 


1.2.  Communication  with  students 

Having  students  involved  in  class  discussions  is  really  important  for  instructors  to  grasp  an  idea  about  student  improvement, 
student  learning,  and  student  problems.  However,  both  of  the  instructors  indicated  that  having  students  involved  in  class 
discussions  is  not  an  easy  task.  Since  most  of  the  students  prefer  to  study  alone,  they  hardly  respond  to  class  discussions.  In 
addition  to  the  lack  of  participation  in  class  discussions,  most  of  them  also  do  not  send  any  response  to  their  instructors. 

Instructor!:  Students  are  very  intentional  about  their  own  learning.  They  are  doing  it  and  I think  they  would  like  to  see 
more  of  it  because  they  are  kind  of  doing  it  on  their  own.  ...In  fact,  there  has  not  been  very  much  interaction  between 
them  even  though  there  has  been  a kind  of  mechanism  for  them  to  have  some  discussion.  They  do  not  respond  to  each 
other  a lot.  They  are  supposed  to  put  these  smaller  type  assignments  up  and  comment  on  each  others  ’.  They  are  pretty 
good  about  putting  up  their  own  work,  but  only  ! or  3 students  have  consistently  been  commenting  back  to  each  other. 
And,  many  students  never  made  any  comment  to  anybody  else.  So  there  is  not  a lot  of  interaction  going  on  in  the  course. 

Face  to  face  communication  is  also  necessary  for  WBI.  Both  of  the  instructors  think  that  knowing  students  personally  is 
important  to  get  feedback.  Talking  to  person  without  knowing  puts  distance  between  the  two  people.  Both  of  the  instructors  felt 
this  distance  between  them  and  their  students.  Instructor!  indicated  that  he  lost  one  or  two  of  the  students  during  instruction.  He 
believed  that  if  he  did  a face-to-face  talk  with  these  students,  he  would  be  more  helpful  for  them  to  solve  their  problems  and 
make  them  believe  not  to  drop  the  course.  Instructor2  indicated  the  importance  of  the  face-to-face  interaction  as: 

Instructor!:  I think  one  thing  in  distance  environment  that  is  dijferent  from  our  typical  face-toface  courses  is  the 
informal  interaction  that  often  takes  place  either  in  the  classroom  or  as  they  walk  by  in  the  hall.  ...So,  it  is  like  there  are 
some  interactions  fin  web-based  courses]  that  are  missing. 
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1.3.  Providing  Feedback 

Instructors  provide  feedback  to  students’  questions  and  assignments.  Students  ask  their  questions  basically  by  sending  e- 
mails.  Any  given  day,  instructors  have  lots  of  e-mails  related  to  students’  problems.  Responding  to  each  e-mail  message  on  time 
is  really  time  consuming  for  instructors  and  sometimes  it  is  frustrating.  Instructors  feel  pressure  to  check  their  e-mails  very  often 
because  they  believe  the  importance  of  timely  feedback.  Instructor!  indicated  that  students  were  expecting  an  immediate 
feedback  from  their  instructor.  Otherwise,  the  students  believe  that  the  instructor  does  not  give  importance  to  their  e-mails.  This 
feeling  affects  the  students’  performance.  To  prevent  any  negative  student  opinions,  instructor2  tries  to  check  her  e-mail  every 
hour  and  try  to  respond  to  them  immediately.  If  the  questions  need  detailed  answers  and  the  instructor  does  not  have  enough  time 
to  write  long  e-mails,  she  prefers  to  send  a short  e-mail  to  confirm  the  reception  of  the  message  and  responds  in  detail  later.  Since 
all  of  the  students  in  instructor!  ’s  class  were  taking  the  course  from  a different  country,  he  changed  his  daily  and  nightly  schedule 
according  to  students’  time.  He  did  not  sleep  until  4 am  to  provide  feedback. 

Grading  students’  assignments  and  providing  feedback  are  also  difficult  duties  for  the  course  instructors.  Instructor2  explains 
her  opinions  about  this  issue  as: 

Researcher:  Are  you  having  any  feedback  problems? 

Instructor!:  Yes,  just  getting  it  done.  I have  a nightmare  last  night.  I feel  terrible  about  that.  Because  I feel  timely 
feedback  is  important.  So,  not  to  have  the  feedback  within  reasonable  time,  like  a week  turnaround,  bothers  me  a great 
deal.  I literally  had  nightmares  all  last  night  thinking.  I kept  up  waking  up  to  see  if  I have  got  the  feedback.  So  it  bothers 
me  a lot  and  it  bothers  them  a lot.  They  keep  asking  "when  you  are  gonna  get  things  graded”.  So,  it  is  definitely  an  issue 
in  [Distance  Education]  class. 

Instructor2  also  indicated  that  she  did  not  feel  comfortable  even  though  she  graded  and  returned  student  assignments  on  time 
because  she  felt  that  some  of  the  students  were  suffering  from  silence.  She  expected  the  students  would  reflect  their  ideas  about 
their  grades.  Some  of  the  students  did  so.  But  some  of  the  students  preferred  to  remain  silent.  She  really  wanted  to  get  response 
from  each  student  to  make  sure  that  they  were  happy  with  their  grades. 

Both  of  the  instructors  agreed  that  the  hours  spent  for  a three-credit  course  is  more  than  a face-to-face  course.  Instructor2 
emphasized  the  hard  work  she  put  with  the  following  phrase: 

Instructor!:  I really  do  work  16  hours  a day  and  7 days  a week  and  every  day  since  school  started. 

1.4.  Number  of  Students 

Instructors  argued  that  number  of  students  is  important  to  provide  effective  instruction  to  them.  According  to  the  instructors, 
10  to  15  students  is  ideal  class  size  for  WBI.  Instructor2  hired  an  additional  graduate  assistant  to  grade  students’  assignments  on 
time.  She  felt  that  otherwise  she  would  not  be  able  to  finish  grading  students’  assignments. 

2.  Students’  Problems  with  WBI 

Interview  results  indicated  that  students  have  wide  range  of  difficulties  while  taking  Web-based  courses.  In  this  paper, 
students’  problems  and  difficulties  are  summarized  under  the  following  categories:  learning  new  tools-novelty  effect, 
communication  with  classmates  and  instructors  (class  discussions,  group  projects,  and  feedback),  and  isolated  learning. 

2.1.  Learning  New  Tools:  Novelty  Effect 

Both  of  the  students  indicated  that  they  had  to  learn  new  tools  such  as  software,  uploading  files,  chat  communication  through 
Oncourse  or  Sitescape,  and  etc.  These  new  tools  were  not  easy  for  them  to  learn  at  first  when  they  were  introduced  to  the  online 
environment.  Learning  a new  way  of  communication  has  created  chaos  at  first,  but  in  time  students  got  used  to  it.  However, 
learning  to  use  new  software  to  finish  projects  was  really  painful  for  students.  It  takes  time  and  extra  effort  to  learn  new  software. 
Students  believe  that  during  this  process,  face-to-face  instruction  is  necessary  to  increase  their  effectiveness  of  doing  their 
project.  One  of  the  students  indicated  that  software  learning  was  not  part  of  the  instruction.  Instructor  of  the  course  expected  from 
students  to  learn  different  software  to  do  their  assignments.  However,  learning  new  software  and  application  of  this  knowledge  to 
the  assignments  are  not  easy  for  a student  if  he/she  is  taking  the  course  online.  At  first,  students  feel  lonely  and  confused  a little 
bit  in  finding  resources  or  people  who  can  help  on  that.  Sending  e-mail  to  the  instructors  to  learn  the  new  software  is  not  the  best 
solution  at  this  point.  The  following  part  of  the  interview  depicts  the  second  student’s  problems  with  learning  new  software: 
Researcher:  What  about  software? 

Student!:  Yes,  so  I had  to  learn  a lot  of  programs  with  that.  Flash...  I had  to  do  some  video  editing.  Dreamweaver...  so  I 
had  learned  them  on  my  own.  I got  some  help  at  the  technology  lab. 

Researcher:  You  told  that  you  had  to  learn  Flash  and  you  got  help  from  a center.  How  was  that  help?  Was  it  related  to 
the  course?  Was  it  part  of  the  course? 

Student!:  It  was  external  You  are  bringing  up  a good  point;  it  would  be  more  helpful  if  there  was  a help  center  in  our 
faculty.  They  just  expected  us  we  had  to  learn  this  stuff.  Not  a student,  but  someone  else  told  me  about  technology 
center...  I still  feel  guilty  going  there.  I felt  bad  asking  them  questions,  because  I really  thought  my  course  should  have 
been  setup  to  help  the  students.  I would  give  my  ideas. 

One  of  the  students  also  indicated  that  trying  to  learn  new  software  is  scary  at  first,  even  though  accomplishing  this  task  was 
not  difficult. 

Researcher:  What  about  learning  new  software? 
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Student!:  I had  to  learn  how  to  use  notepad  to  do  my  html  I had  not  used  notepad  before.  So,  I think  it  was  assumed  that 
everybody  had  used  notepad  before,  and  I was  like  what  do  I do?  So  I had  to  figure  that  out  on  my  own  and  it  turned  to 
be  really  easy  but  it  was  a little  scary  at  first,  like  just  trying  to  use  something  for  the  first  time. 

2.2.  Communication  with  Classmates  and  Instructors 

Class  discussions,  group  projects,  and  feedback  are  the  main  way  of  communication  for  students  to  share  their  ideas  with  their 
classmates  and  instructors.  In  this  part,  problems  with  class  discussions,  group  projects,  and  feedback  will  be  given  in  three 
different  categories. 

2.2.1  Class  Discussions 

During  the  class  discussion,  since  most  of  the  students  actively  participating  in  the  discussion  ask  their  questions  at  the  same 
time,  students  have  difficulty  in  following  the  flow  of  the  discussion.  In  time,  they  get  bored  and  quit  writing  their  responses  and 
asking  questions.  It  is  very  difficult  for  students  and  instructors  to  create  an  environment  helpful  for  their  learning  and  teaching. 

Student!: ...  And  then  when  we  used  the  live  chat,  which  happened  through  Oncourse,  somebody  would  post  a comment 
and  all  of  us  would  respond  to  it.  But  it  took  so  much  time  to  response  to  a post,  in  the  meanwhile  another  response 
would  come  up,  we  could  lose  the  chain  of  conservation  because  everybody  was  seemed  to  be  engaged  with  a bunch  of 
different  conversations,  it  did  not  flow.  Like,  you  say  something.  I say  something,  and  he  says  something.  There  were  all 
postings  out  of  sequence.  So,  that  was  frustrating.  fVe  even  talked  about  in  the  discussion  like  ”woo  I cannot  track  of 
this  "you  know,  too  many  people  are  saying  too  many  things  at  once. 

In  addition  to  this  chaos,  the  students  also  indicated  that  time  issue  was  problematic  for  the  communication.  Generally,  since 
there  is  no  meeting  time  for  WBI  like  in  face-to-face  instruction,  students  study  at  their  own  pace  and  respond  to  the  activities 
according  to  their  available  time.  Some  of  the  students  prefer  to  study  during  night  and  some  of  them  prefer  to  study  during  the 
day.  These  working  schedules  create  difficulty  in  finding  common  time  for  class  discussions.  Due  to  these  problems,  some  of  the 
students  prefer  not  to  attend  to  class  discussions.  Interestingly,  in  this  study,  both  of  the  students  indicated  that  they  did  not  have 
a computer  at  home.  They  have  used  the  university’s  computers.  For  some  class  discussions,  one  of  the  students  needed  to  go  to 
school  without  considering  the  time. 

Student!:  Several  students  could  not  make  the  live  chat  sessions.  We  tried  to  schedule  the  live  chat  and  it  was  very 
difficult  to  get  a good  time.  We  ended  up  making  it  very  late:  actually  I had  to  go  up  to  the  school  at  night,  sit  around  and 
do  this  thing. 

Because  of  having  difficulties  in  following  the  chat  and  finding  a suitable  time  for  the  chat  sessions,  students  think  that  class 
discussions  are  not  helpful  for  their  learning. 

Student!:  There  really  was  not  a lot  of  discussion.  It  almost  seemed  like  because  the  discussion  was  so  awkward  and 
unnatural  compared  to  just  face-to-face  interaction,  I almost  fell  a desire  to  just  do  the  course  independently,  like  do  it 
myself 

2.2.2.  Group  Projects 

The  main  problem  in  doing  group  projects  is  to  communicate  effectively  with  the  group  members.  Since  the  main 
communication  method  is  e-mail,  it  is  difficult  to  respond  to  the  e-mail  messages  immediately  after  they  are  received.  Delay  in  e- 
mail  response  time  makes  the  partners  frustrated  and  they  perceive  it  as  the  waste  of  time.  Students  believe  that  face-to-face 
communication  during  group  projects  helps  them  to  find  quick  solutions  to  their  problems  and  ask  questions  and  have  answers 
directly  without  a delay. 

Student!:  ...  lam  always  checking  and  replying.  That’s  [e-mail]  like  a primary  form  of  communication  for  you.  And  for  a 
lot  of  other  classmates,  it  is  not.  It  was  frustrating  for  me  to  send  an  e-mail  and  not  getting  immediate  response  or  at 
least  getting  a response  that  day.  To  not  hear  from  the  people  for  two  days  after  I send  them  e-mail  when  I am  supposed 
to  working  with  them  on  something.  So,  I kind  of  felt  like  waitingggg.  Hellooo.  Like,  answer  my  e-mail.  So  that  was 
frustrating. 

Since  students  do  not  want  to  take  other  persons’  time,  they  try  to  check  their  e-mail  more  often  than  they  do  usually. 
Students  feel  pressure  on  themselves  to  check  their  e-mail  whenever  they  find  time  and  a computer.  Because  of  this  pressure, 
even  though  they  do  not  want  to  think  anything  about  the  course,  they  force  themselves  to  check  their  e-mail  and  respond  to  them 
accordingly.  They  feel  like  they  have  to  work  on  their  group  members’  schedule  and  not  their  own  schedule.  In  time,  the  course 
becomes  a part  of  their  daily  life. 

Student!: ...  Just  like  in  meeting,  you  have  that  time  you  know  you  can  ask  these  questions,  and  talk  about  things.  When 
you  do  not  have  these  meetings  the  question  is  always  going  on.  I do  not  like  that  as  much...  And  you  know  some 
weekends,  I do  not  have  to  think  about  the  course  but  I get  e-mails  from  people  and  you  cannot  always  work  the  same 
schedule  with  someone  else. 

2.2.3.  Feedback 

Students  usually  take  their  feedback  from  instructors  and  classmates  during  courses.  For  WBI,  students  feel  like  responses 
and  feedback  from  classmates  are  not  as  informative  for  them.  Students  have  difficulty  in  giving  credibility  to  classmates’ 
opinions.  Student  1 indicated  difficulty  in  trusting  people  if  she  does  not  know  them.  Because  of  this  reason,  she  gave  importance 
to  instructors’  feedback  rather  than  classmates’  feedback.  In  addition  to  that,  student2  emphasized  that  most  of  the  students  did 
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not  have  enough  time  to  spend  for  other  persons’  problems.  Instead,  they  are  spending  their  time  to  find  solutions  to  their  own 
problems.  This  belief  also  decreases  the  effectiveness  of  the  communication  among  the  students.  Students  perceive  the  course 
instructor  as  the  only  resource  to  get  information  and  to  find  answers  to  their  questions.  However,  thinking  like  that  is  not  the 
solution  for  students’  problems.  Students  do  not  feel  free  to  ask  questions  to  the  instructors  at  any  time.  Instructors  post  a time 
limit  for  students  on  when  they  should  expect  a response.  Student2  felt  herself  guilty  when  she  sent  an  e-mail  outside  of  this  time 
limit  because  she  thought  that  she  was  taking  instructor’s  time  and  disturbing  him. 

In  face-to-face  communication  it  is  easy  for  instructors  to  follow  their  students*  learning  and  provide  feedback  when  it  is 
necessary.  In  WBI,  instructors  can  do  so  only  if  students  send  e-mails  or  responds  to  classroom  discussion.  Most  of  the  time,  this 
situation  creates  problems  for  doing  assignments.  Students  want  instructor  feedback  when  they  work  on  their  assignments  or 
projects.  For  WBI,  students  are  expected  to  send  their  final  version  of  projects.  Both  studentl  and  student2  felt  that  they  did  not 
get  any  feedback  for  their  studies  during  their  work.  Also,  their  instmctors  did  not  return  their  final  projects  with  the  feedback. 
They  thought  that  this  way  of  grading  and  submission  was  not  helpful  for  their  learning.  Since  they  did  not  know  their  strengths 
and  weaknesses,  they  were  not  sure  whether  they  were  successful  or  not  on  achieving  course  objectives.  This  situation  is  more 
problematic  for  students  who  are  not  comfortable  with  asking  questions  to  instructors  to  get  feedback  for  their  assignments. 
Sending  everything  on  due  date  and  having  nothing  but  the  grade  makes  students  frustrated. 

Student2: .. .because you  could  not  ask  clarification  questions.  We  were  not  able  to  ask  [like  in  traditional  class]  for  her 
to  say  ''oh  you  make  sure  you  do  this  part  of  the  assignment.  ” It  was  a huge  project  that  was  hard  to  imagine  the  entire 
time  that  was  involved.  It  would  have  been  nice  if  it  had  little  checkpoints  on  the  web  like  "oh  you  need  to  turn  this  part” 
instead  of  it  was  like  everything  was  due  at  the  end. 

Studentl:  I would  like  to  get  more  feedback  along  the  way  I was  doing  my  project.  Like  creating  my  web  page.  I would 
like  to  be  able  to  submit  it,  get  some  feedback  then  go  back  and  change  it.  Improve  it  and  be  graded  on  it.  Whereas  we 
submitted  it,  we  read  everybody ’s  comment  and  got  a grade.  A big  part  of  the  grade  was  coming  from  this  project. 

2.3.  Isolated  Learning 

Doing  collaborative  work  or  group  projects  in  WBI  is  difficult  for  students.  Because  of  the  difficulty  in  finding  time  for 
group  work  and  giving  importance  to  just  instructors*  feedback  make  students  isolated  during  the  course  taking.  The  primary 
reason  for  students  to  take  WBI  is  the  time  flexibility.  They  do  not  have  to  be  in  the  classroom  at  certain  times.  So,  they  can  take 
these  courses  when  they  work.  This  is  another  reason  for  the  isolation  of  students  from  collaborative  work. 

Studentl:  I like  to  work  with  groups  sometimes.  It  depends  on  if  your  schedules  can  match  up  like  in  face-to-face  classes. 
With  this  one,  I am  glad  that  we  had  individual  assignments  versus  group  assignments.  I think  group  assignments  would 
be  way  too  difficult  to  coordinate  because  people  are  online  at  different  times.  I would  do  my  things  during  the  week  and 
then  not  online  during  the  weekend.  So,  there  are  a lot  of  people  doing  it  (classroom  teachers)  at  the  weekend.  So,  we 
always  missed  each  other. 

In  face-to-face  communication  students  have  a chance  to  get  to  know  each  other  and  to  develop  trust  for  their  classmates. 
Studentl  indicated  that  since  she  did  not  know  the  people,  it  was  hard  for  her  to  develop  trust  for  them.  Students  do  not  care 
about  other  students*  learning,  ideas,  and  difficulties.  Their  purpose  is  to  finish  the  course  at  their  own  pace.  They  do  not  want 
anyone  to  disturb  their  comfort.  Student2  said  that  she  had  a difficulty  in  learning  different  software  such  as  Flash.  However,  she 
did  not  request  for  help  from  anybody  in  her  class.  She  thought  that  everybody  should  have  spent  their  time  for  their  assignments. 
They  did  not  have  time  to  teach  Flash  to  any  other  classmate.  In  her  opinion,  it  was  the  instructor’s  responsibility  to  help  students 
with  learning  new  software.  When  she  asked  for  help  to  this  problem  from  her  instructor,  the  instructor  sent  an  informative  web 
site.  However,  she  had  difficulty  in  finding  answers  to  her  questions.  Finally,  she  decided  to  find  an  external  support  for  her 
problems.  She  further  indicated  that  too  much  student-centered  learning  makes  students  isolated.  There  should  be  more 
instruction  and  guidance  provided  by  instructors.  Interestingly,  student2  also  felt  that  her  instructor  was  isolated  and  her  teaching 
style  was  also  an  isolated  type. 

Researcher:  Do  you  think  it  is  the  duty  of  the  instructor  to  find  solutions  to  your  problems? 

Student2:  Yes,  what  was  she  doing?  I do  not  know  what  she  did  all  semester.  It  bugged  me.  I did  not  see  her  taking  time 
out  to  organize  anything.  It  is  obvious  to  me  that  there  should  have  been  help  sessions. 

According  to  students,  instructors  are  not  aware  of  students*  problems.  Sometimes  as  the  result  of  the  student  e-mails, 
instructors  are  informed  about  the  problems.  In  most  cases  instructors  send  information  such  as  a useful  web  site  or  a detailed  e- 
mail  message  to  help  students.  However,  students  indicate  that  some  of  the  problems  are  difficult  to  solve  with  these  information. 
Students  still  have  questions  in  their  minds  and  reading  information  through  web  sites  or  through  e-mail  still  do  not  answer  their 
questions.  They  need  face-to-face  interaction  with  instructors  not  only  for  asking  questions  but  also  for  discussing  the  ideas 
related  to  their  problems  and  for  getting  immediate  help. 

3.  Responses  to  Open  Ended  Questions 

At  the  end  of  each  interview  session,  two  questions  were  asked  to  identify  instructor  and  student  preferences  for  classroom 
instruction  and  WBI. 
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3.1.  Which  One  is  Difficult  Classroom  Teaching  or  Online  Teaching? 

There  are  some  advantages  and  disadvantages  for  classroom  and  Web-based  teaching.  Because  of  this,  instructors  had 
difficulty  in  choosing  just  one  type. 

Instructor2,.. because  it  (WBI)  is  new  and  because  it  is  unfamiliar  both  to  instructors  and  learners,  it  is  harder  and  more 
time  consuming.  So  if  someone  says  I could  teach  either  way,  I would  probably  choose  the  classroom,  because  I know 
how  it  is  done,  the  students  know  how  it  is  done.  I can  do  it  more  quickly  and  easily.  But  that  isn ’t  to  say  that  I do  not 
think  there  is  good  place  for  online  learning.  I actually  think  that  there  is  and  I think  it  allows  us  just  to  do  things  and 
have  opportunities  that  we  could  not  have  otherwise.  But  I think  face-to-face  is  definitely  easier.  It  is  easier  to  prepare 
for;  it  is  easier  to  deal  with  every  way.  The  other  is  just  more  time  consuming  [for  the  instructor]. 

3.2.  Which  One  is  Difficult  Classroom  Learning  or  Online  Learning? 

Even  though  student  1 had  positive  and  negative  ideas  about  both,  student2  definitely  preferred  the  classroom  teaching. 

Studentl:  I think  it  depends  on  the  subject  matter.  Our  class  dealt  with  the  Internet  itself.  It  was  appropriate  in  a way.  But 
I could  imagine  if  we  were  doing  something  like  teaching  methods  class,  then  it  would  have  been  ridiculous  to  try  to  do 
that  online.  I think  for  some  courses  you  have  to  interact  face-to-face  with  other  people  and  actively  do  versus  doing  on 
your  own  and  submitting. 

Student2: 1 think  learning  online  is  more  difficult.  If  you  are  in  the  classroom,  teacher  is  talking  so  you  know  what 
information  is  important  or  what  you  need  to  know,  whereas  on  the  web  it  is  just  like  there  is  so  much  stuff  that 
sometimes  it  is  hard  to  know  what  is  the  most  important  thing.  Are  they  all  important? 

3.3.  Do  You  Feel  Like  a Facilitator  or  Instructor? 

Both  of  the  instructors  felt  themselves  as  a facilitator. 

3.4.  Do  You  Like  Student-Centered  Learning? 

Student2  indicated  that  student  centered  learning  isolated  students  and  make  learning  more  individual.  Studentl  liked  the 
individual  learning  since  it  provides  her  to  arrange  her  daily  schedule  according  to  other  tasks. 

Students2:  It  was  too  student  centered.  I think  that  can  be  bad,  really  bad  because  it  makes  you  isolated,  you  need  to  have 
a little  bit  more  instruction,  or  guidance. 

Conclusions  and  Recommendations 

In  general,  both  students  and  instructors  have  problems  on  similar  issues  such  as  communication,  feedback,  and  support.  It  is 
clear  that  solving  problems  of  instructors  might  be  helpful  in  solving  students’  problems  or  vice  versa.  For  any  Web-based  class, 
face-to-face  interaction  should  be  held  at  least  once  at  the  beginning  of  the  course  to  help  instructors  and  students  to  get  to  know 
each  other.  Since  the  technology  is  developing  very  fast  and  making  the  hardware  and  software  cheaper,  Web-based  video 
conferencing  can  be  an  option  for  those  who  cannot  attend  face-to-face  interaction. 

For  instructors,  technological  support  should  be  provided  on  time  and  when  it  is  needed.  Computers  and  software  are  the  only 
way  to  communicate  with  students.  If  instructors  have  difficulty  in  obtaining  this  support  on  time,  their  communication  with 
students  can  be  disastrous.  Students  can  easily  drop  the  course  just  because  of  this. 

Application  of  different  instructional  methods  can  be  helpful  for  increasing  students’  motivation  and  interest  toward  WBI.  In 
addition  to  this,  spending  time  for  preparing  instructional  materials  by  considering  differences  in  learning  style  and  diversity  in 
class  can  make  students  think  that  their  instructors  are  spending  time  for  teaching  the  course.  Since  the  time  spent  for  solving 
course  problems  by  instructors  is  invisible  to  students,  they  think  that  their  instructors  are  doing  nothing  to  help  them  in  their 
learning.  In  face-to-face  teaching,  instructors  give  importance  to  apply  different  teaching  methods  according  to  students’ 
difficulties.  Application  of  the  same  strategy  in  WBI  would  be  helpful  for  both  instructors  and  students. 

Formative  assessment  is  another  missing  part  in  WBI.  According  to  Buchanan  (1999),  feedback  given  as  in  the  form  of 
continuous  formative  assessment  might  help  students  to  monitor  and  evaluate  their  progress.  At  the  same  time,  this  approach  can 
help  instructors  to  see  the  students’  difficulties  and  provide  immediate  feedback  when  it  is  necessary.  Another  benefit  of  the 
formative  assessment  might  be  increasing  the  communication  between  instructor  and  students. 

In  face-to-face  classroom,  instructors  have  advantages  in  understanding  students’  difficulties  while  observing  them  in 
classroom.  In  online  teaching,  there  is  no  chance  for  instructors  to  do  this.  However,  instructors  can  do  this  observation  by 
sending  e-mails  to  students  to  learn  their  difficulties  instead  of  waiting  e-mails  from  them.  Some  of  the  students  can  prefer  to  stay 
silence  in  WBI  but  instructors’  e-mails  can  make  them  feel  that  their  instructors  are  taking  care  of  their  progress  and  difficulties. 
This  approach  can  also  be  helpful  in  preventing  student  dropout. 

To  foster  dialog,  participation,  and  interaction,  instructors  need  to  find  activities  to  increase  students’  involvement  in  group 
working  and  discussions. 
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Abstract 

As  the  popularity  and  accessibility  of  the  World  Wide  Web  (Web)  increases  for  shopping,  effective  navigation  is  becoming 
more  and  more  critical  to  the  success  of  E-commerce,  Even  though  many  educational  technologists  and  Web  designers  have 
spent  their  energy  developing  effective  navigation  tools,  it  remains  difficult  to  develop  Web  systems  that  can  help  customers  find 
products  or  services  that  thy  wan  to  purchase  without  experiencing  disorientation  problems  and  cognitive  overload.  Many  E- 
commerce  Web  sites  are  beginning  to  employ  a pull-down  menu  since  it  provides  the  most  versatile  navigation  mechanism. 
Although  the  pull-down  menu  design  has  been  used  in  other  computer  applications  it  is  relatively  new  on  the  Web.  This  study 
analyzed  the  navigation  effect  of  the  pull-down  menu  design  with  three  hierarchical  information  structures  (constant,  increasing, 
and  decreasing  types).  The  navigation  effect  was  measured  by  two  information  searching  strategies  (searching  and  browsing) 
and  three  users  attitudinal  measures  (appeal,  perceived  usability,  and  perceived  disorientation).  Three  Cyber -shopping  malls 
were  developed  with  the  pull-down  menu  design  and  three  information  structures.  Fifty-eight  undergraduate  and  graduate  at 
mid-western  university  participated  voluntarily  in  this  study. 

The  findings  provide  useful  information  for  designing  a pull-down  menu  design  and  information  structure  for  effective 
navigation.  The  results  of  this  study  show  that  there  exist  better  combinations  of  pull-down  menu  design  and  information 
structures  in  terms  of  the  efficacy  of  browsing,  the  overall  appeal  of  the  site,  the  perceived  usability  and  the  users  perceived 
disorientation.  The  overall  results  showed  that  decreasing  information  structure  produced  more  effective  browsing  speed, 
appeal,  perceived  usability,  and  disorientation  than  increasing  information  structure.  This  study  demonstrated  that  the 
information  structure  that  has  more  links  on  the  upper  levels  induced  more  effective  browsing  by  providing  more  links.  Finally  it 
is  recommended  that  Web  designers  or  Web  researchers  should  consider  the  information  structure  in  order  to  analyze  the 
navigation  effect  of  menu  design. 

Introduction 

The  popularity  and  accessibility  of  the  Web  have  been  increasing  dramatically  and  changing  the  fundamental  way  to 
purchase  products  and  services  (Yoo  & Kim,  2000).  As  both  the  amount  and  availability  of  products  and  services  on  the  Web 
increase,  effective  navigation  is  becoming  more  and  more  critical  to  the  success  of  E-commerce.  However,  navigatinga  Web  site 
is  often  not  an  easy  task,  especially  for  novices  (Berg,  1997;  Dieberger,  1997;  King,  1996;  Sand,  1996).  The  potentially  complex 
linking  system  and  information  structure  awaiting  Web  customers  can  cause  disorientation,  increase  cognitive  loads  (Collis, 
1991),  and  lead  to  users  getting  lost  in  cyberspace  (Neilsen,  1990).  Hammond  and  Allinson  (1989)  contend  that  users  may 
encounter  a number  of  common  problems:  ‘They  may  have  difficulty  gaining  an  overview,  finding  specific  information,  or  using 
the  interface  tools;  they  may  wander  without  an  orienting  goal  or  strategy,  or  may  even  get  lost”(p.  69). 

Web  customers  navigate  through  an  enormous  body  of  information  by  following  a likely  path  from  one  page  to  another  until 
finding  the  product  they  want  to  buy.  As  the  amount  of  products  and  services  have  been  increasing  dramatically,  the  way  to 
organize,  present,  and  access  the  products  has  become  a crucial  issue  in  order  to  support  customers’  effective  navigation  because 
the  complexity  of  navigation  has  increased  correspondingly  (Berg,  1997;  Chen,  Mathe,  & Wolfe,  1998;  Newfield,  Sethi,  & Ryall, 
1998;  Pitkow  & Recker,  1994).  Recent  usability  tests  show  similar  evidences.  Users  still  get  lost  very  easily  on  the  Web  and  it  is 
still  a dilemma  in  designing  a Web  site  where  users  can  find  information  fast  and  easily  (Nielsen,  1997,  1999;  Kim,  1995) 

Such  problems  have  prompted  research  on  the  manner  in  which  users  interact  with  the  Web.  Many  researchers  have  studied 
the  relationships  between  the  user  interface  and  information  structure  for  effective  navigation  and  information  searching.  Since 
the  main  purpose  of  the  E-commerce  Web  site  is  to  access  products  effectively,  how  the  information  is  structured  on  the  Web  site 
and  how  the  link  mechanism  is  presented  on  the  menu  of  the  Web  site  can  significantly  influence  the  success  of  navigation 
effects  (Berg,  1997;  Bra,  1988;  Halasz,  1988;  Hardman,  Bulterman,  & Rossum,  1994;  Shneiderman,  1998;  Shneiderman  & 
Kearsley,  1989). 

E-commerce  Web  sites  have  been  applied  many  menu  designs  for  better  navigation.  Numerous  Web  sites  began  to  employ  a 
pull-down  menu  since  it  provides  the  most  versatile  path  mechanism  for  fast  navigation.  The  navigation  effect  of  a pull-down 
menu,  however,  can  be  different  depending  on  how  information  structure  is  organized.  The  information  structure  is  invisible  to 
users  when  they  enter  the  Web  site.  It  is  the  menu  design  that  provides  users  with  the  linking  mechanism.  They  come  to 
understand  the  structure  of  the  Web  site  only  after  interacting  with  menu.  What  users  perceive  from  menu  design  may  also  differ 
depending  on  how  information  is  organized.  Therefore,  studies  investigating  the  effect  of  a menu  design  on  the  Web  should  be 
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analyzed  with  information  structure.  In  other  words,  the  ideal  study  should  be  conducted  in  a situation  which  employs  the  menu 
design  and  information  structure  together  to  allow  an  analysis  of  how  these  two  factors  have  an  influence  on  navigation  or 
information  searching  performance. 

It  must  be  emphasized  that  the  research  on  navigation  or  information  searching  should  consider  those  two  factors  in  the  same 
situation.  This  study  concentrates  on  a pull-down  menu  design  and  information  structure  in  the  same  line  in  order  to  analyze  how 
the  pull-down  menu  design  affects  information  searching  performance  depending  on  different  information  structures. 

The  Navigational  Structure  of  the  Web 

Navigation  can  be  defined  simply  as  accessing  information  on  the  Web  (Gay,  1991).  Navigation,  however,  means  more  than 
a process  of  simply  accessing  information.  Cunliffe,  Taylor,  and  Tudhope  (1997)  defined  navigation  as  “...high  interactivity  in  a 
structured  environment  with  the  destination  seldom  pre-determined.  Navigation  is  often  a compromise  between  user  and  system 
responsibility;  an  incremental  process  with  the  user  making  choices  from  directions  and  feedback  provided  by  the  system...”  (p. 
99). 

A Web  site  is  the  networked  body  of  chunks  of  information  with  links  (Homey,  1993;  Jonassen,  1996).  The  basic  technique 
for  navigating  a Web  site  is  selecting  the  paths  provided  by  links  (Rosenfeld  & Morville,  1998).  Navigation  in  the  Web  demands 
users  engage  in  relatively  easy  activities  (typing  a URL,  using  search  engines,  and  moving  mouse  and  clicking  links)  for 
navigation.  Information  structure  is  how  the  information  is  organized  on  the  Web  and  it  provides  the  primary  ways  users  can 
navigate  through  links  (Rosenfeld  & Morville,  1998).  Since  users  can  only  create  paths  based  on  the  links  provided,  the  logical 
navigation  underlying  design  of  those  links  influences  the  paths  that  users  can  take.  Therefore,  the  structuring  of  the  information 
plays  a fundamental  role  in  navigation  (Hardman  et  al.,  1994). 

When  users  enter  the  Web  site,  it  is  not  easy  to  understand  the  entire  information  structure  of  it.  Users  come  to  understand  its 
structure  as  they  navigate  through  the  paths  provided  by  the  menu  design.  The  fundamental  function  of  the  menu  is  to  display 
links  on  the  screen  so  that  a user  can  navigate  through  a Web  site  (Schwartz  & Norman,  1986).  A series  of  interactions  among  a 
user,  information  structure,  and  menu  allow  a user  to  construct  a cognitive  structure  for  navigation. 

Users  exchange  the  information  and  control  with  a given  system  based  on  two  main  components:  menu  design,  which 
controls  the  communication  with  users  through  linking  mechanisms,  and  the  information  structure,  which  relates  to  how  to 
incorporate  the  original  structure  of  the  content  into  the  structure  of  a Web  site  (Chang,  1995;  Jul  & Furnas,  1997;  Marchionini, 
1995;  Oliveira,  Goncalves,  & Medeiros,  1999).  Although  the  information  structure  is  invisible  to  users,  they  come  to  understand 
it  when  they  interact  with  menus  that  reflect  the  information  structures.  Clearly  there  are  a number  of  ways  menus  and 
information  structures  vary.  For  instance,  the  same  information  can  be  structured  using  a different  structure  style.  There  also  can 
be  different  ways  to  design  the  menu  to  present  the  same  information  structure.  In  that  point,  it  is  possible  that  the  information 
searching  performance  can  be  different  in  situations  where  users  interact  with  the  same  menu  style  that  presents  a different 
information  structure. 

In  the  research  on  menu  design,  a simple  and  constant  information  structure  has  been  employed  to  analyze  the  effect  of  menu 
design.  Most  studies  have  investigated  only  constant,  symmetric  hierarchies  in  which  the  menus  at  all  levels  have  the  same 
number  of  items.  However,  most  real-world  menus  are  not  constant  but  vary  in  the  number  of  alternatives  at  each  level  due  to  the 
nature  of  the  database  (Norman  & Chin,  1988).  For  these  reasons,  this  research  employed  three  information  structures  in  order  to 
examine  the  navigation  effect  of  a pull-down  menu. 

Materials  and  Methods 

Pull-down  Menu 

The  pull-down  menu  design  is  relatively  new  on  the  Web  even  though  it  has  been  used  in  many  other  computer  applications. 
Pull-down  menus  appear  over  objects  in  the  interface  instead  of  in  a static  menu  area,  and  they  allow  users  to  access  directly  the 
Web  page  they  want.  The  advantage  of  this  menu  style  is  that  it  provides  the  most  versatile  path  mechanism  for  navigation.  Users 
can  Jump  to  any  page  by  moving  the  mouse  and  clicking  without  through  passing  intermediate  pages.  Many  Web  sites  for 
commerce  have  been  adapting  this  menu  design  since  users  can  find  information  fast. 
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Figure  4.  Pull-down  Menu 
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Information  Structure 

Literature  in  hypertext  and  the  Web  shows  that  the  hierarchical  information  structure  is  the  most  popular  and  appropriate 
structure  (Morris  & Hinrichs;  Rosenfeld  & Morville,  1 998).  In  this  study,  the  hierarchical  information  structure  on  the  Web  site 
was  organized  in  three  ways:  constant,  increasing,  and  decreasing  structure.  Figure  2 shows  three  structures  for  presenting  256 
items  of  cyber-shopping  merchandise  items  with  a depth  of  four  levels. 


Constant  Structure 


Decreasing  Structure  Increasing  Structure 

Fi^re  5.  Three  Information  Structures 


Constant  Hierarchical  Structure 

The  constant  structure  serves  as  a baseline  of  comparison  since  it  has  been  used  most  frequently  in  past  research.  There  are 
four  links  at  each  level  in  constant  hierarchical  structure  (4  x 4 x 4 x 4). 

Decreasing  Hierarchical  Structure 

The  decreasing  hierarchical  structure  gives  a large  number  of  choices  at  the  top  of  the  menu  and  narrows  the  range  of  choice 
at  the  bottom.  In  this  structure,  there  are  eight  links  at  first  and  second  level  and  two  links  at  third  and  forth  level  (8  x 8 x 2 x 2). 
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Increasing  Hierarchical  Structure 

The  increasing  structure  gives  a small  number  of  links  to  the  user  at  the  top  level  of  pages  and  increases  the  number  of  links 
at  the  bottom.  There  are  two  links  at  first  and  second  levels  and  eight  links  at  third  and  fourth  levels  (2  x 2 x 8 x 8). 

Table  1 shows  a summary  of  three  Web  site  variants  comparing  the  number  of  pages  and  links  for  each  experiment.  Each 
information  structure  was  organized  with  a total  of  256  pieces  of  information  in  a depth  of  four  levels.  However,  the  number  of 
Web  pages  at  each  level  in  each  information  structure  is  different.  For  example,  the  experimental  Web  sites  where  constant 
information  structure  (4  x 4 x 4 x4)  was  or^nized  with  a pull-down  menu  designs  have  a total  85  numbers  of  Web  pages.  In  the 
experimental  Web  sites  2,  each  Web  site  was  consisted  with  a total  number  of  201  pages  while  experimental  Web  site  3 has  only 
39  pages  in  total. 

Table  1.  Comparison  of  Number  of  Pages  and  Links  in  Experimental  Websites 


Information 

Number  of  Pages 

Number  of  Links 

weDSii 

es 

Level 

Level 

S tructurc 

1 

2 

3 

4 

Total 

1 

2 

3 

4 

1 4x4x4x4=256 

1 

4 

16 

64 

85 

84 

84 

84 

84 

2 

8x8x2x2=256 

1 

8 

64 

128 

201 

290 

290 

290 

290 

3 

2x2x8x8=256 

1 

2 

4 

32 

39 

38 

38 

38 

38 

The  most  important  fact  in  experimental  systems  is  that  the  number  of  links  is  different  from  each  other  since  it  is  determined 
by  the  shapes  of  information  structure.  For  instance,  the  number  of  links  at  each  page  is  84  with  constant  structure,  290  with 
decreasing  one,  and  38  with  increasing  one  when  users  open  the  all  pull-down  menus.  The  main  focus  of  this  study  is  how  these 
differences  caused  by  combination  of  different  information  structures  and  a pull-down  menu  affect  users’  information  searching 
and  other  perception  on  appeal,  usability,  and  disorientation. 

Selection  of  Searching  Task 

In  order  to  measure  the  influence  of  three  different  information  structures  on  the  Web  site,  this  study  included  two  types  of 
tasks:  searching  and  browsing  (Canter,  Rivers,  & Storrs,  1985).  Each  five  searching  and  browsing  tasks  were  included  in  this 
study.  The  examples  of  each  task  were  as  follows: 

- Searching  task:  You  want  to  buy  Epson  Color  200  printer  in  this  Web  shopping  mall.  Please  find  the  price  of  this  printer  in 
this  site. 

- Browsing  task:  Your  father  likes  music  very  much  and  you  want  to  buy  a birthday  gift  for  your  father.  Please  select  the 
music  item  that  will  make  your  father  happy. 

Participants 

Total  60  undergraduate  and  graduate  students  participated  at  mid-western  university  in  this  study  voluntarily.  We  excluded 
two  subjects  from  final  analysis  because  one  subject  had  a serious  sight  problem  due  to  her  age  and  another  subject  missed  or 
misunderstood  several  tasks  due  to  language  problem.  The  age  of  the  former  subject  was  54years  old,  and  the  latter  subject  was 
an  international  student  who  was  taking  an  intensive  English  program  first  level.  Therefore,  the  actual  number  of  subjects  for  the 
final  analysis  was  58.  Subjects  ranged  in  age  from  20  to  49  years.  They  were  diverse  in  terms  of  their  computer  and  Internet 
related  abilities. 

Procedures 

This  experiment  consisted  of  three  sessions.  During  the  first  session,  a participant  was  asked  to  fill  out  the  background 
information  form.  It  took  approximately  five  minutes  to  fill  out  this  form.  After  completing  the  questionnaire,  a participant  was 
assigned  to  one  of  three  treatments  randomly  and  was  asked  to  find  the  answers  of  10  tasks.  Each  task  was  given  to  the  subject 
one  at  a time. 

The  subject  was  told  to  tell  the  researcher  “start”  before  he/she  started  each  searching  task  and  to  tell  the  researcher  “the  price 
of  the  item”  after  he/she  found  the  answer.  During  information  seeking  tasks,  the  researcher  measured  the  time  for  each  task.  This 
procedure  continued  until  the  subject  finished  all  10  tasks.  It  took  approximately  10  to  15  minutes  for  a subject  to  finish  all  tasks. 
After  completing  the  test  session,  the  participant  was  asked  to  complete  an  attitude  questionnaire.  This  took  about  5 minutes. 

Results 

Information  seeking  performance,  as  mentioned  earlier,  was  divided  into  two  task  types:  searching  task  and  browsing. 
ANOVA  statistics  showed  the  result  of  effect  of  searching  task  on  time  to  spend  finding  answers  among  three  structure  designs 
(see  Table  2).  There  was  no  significant  difference  among  three  structure  designs,  F (2,  55)  = .35,  p > .05. 
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Table  2.  An  AN  OVA  Summary  Table  With  Group  Means  and  Standard  Deviations  for  Searching  Task  by  Structure  Design 


Structure 

N 

M 

SD 

4 x4  x4  x4 

19 

47.22 

20.08 

8x8x2  X 2 

20 

43.75 

22.85 

2 X 2 X 8 X 8 

19 

50.74 

32.86 

Source 

df 

SS 

MS 

F 

Structure 

2 

476.93 

238.47 

.35 

Error 

55 

37171.69 

675.85 

Total 

57 

37648.62 

Tables  3 showed  that  there  was  significant  difference  among  three  structure  designs  in  terms  of  browsing  task,  F (2,  55)  = 
3.86, 2.  < .05.  Tukey’s  HSD  post-hoc  comparisons  were  used  to  determine  significant  differences  between  means  at£  < .05.  Post 
hoc  comparisons  results  revealed  that  there  was  a significant  difference  between  increasing  information  structure  (2  x 2 x 8 x 8) 
and  decreasing  information  structure  (8x8  x 2 x 2).  The  amount  of  browsing  time  of  increasing  information  structure  (M  = 
56.55,  ^ = 1 5.49)  was  longer  than  that  of  decreasing  structure  (M  = 42.05,  SD  = 8.95),  £ < .05. 


Table  3.  An  ANOVA  Summary  Table  With  Group  Means  and  Standard  Deviations  for  Browsing  Task  by  Structure  Design 


Structure 

N 

M 

SD 

4 x4  x4  x4 

19 

47.32 

22.04 

8x8x2  x2 

20 

42.05 

8.95 

2 x2  X 8 X 8 

19 

56.55 

15.49 

Source 

df 

SS 

MS 

F 

Structure 

2 

2087.76 

1043.88 

3.86* 

Error 

55 

14878.72 

270.52 

Total 

57 

16966.47 

*£<.05. 


The  following  results,  on  the  other  hand,  showed  that  there  was  significant  difference  among  three  structure  designs  with 
respect  to  three  participants’  perceptions:  the  degree  of  perceived  appeal,  usability,  and  disorientation. 

As  shown  in  Table  4,  there  was  statistically  significant  difference  in  the  degree  of  users’  perceived  appeal  in  terms  of  three 
structure  designs,  F (2,  55)  = 4.60,  £ < .05. 


Table  4.  An  ANOVA  Summary  Table  With  Group  Means  and  Standard  Deviations  for  Users ' Perceived  Appeal  by  Structure 
Design 


Structure 

N 

M 

SD 

4 x4  x4  x4 

19 

17.21 

4.66 

8 X 8 X 2 X 2 

20 

20.65 

4.09 

2 X 2 X 8 X 8 

19 

15.63 

6.80 

Source 

df 

SS 

MS 

F 

Structure 

2 

258.03 

129.01 

4.60* 

Error 

55 

1544.13 

28.08 

Total 

57 

1802.16 

*£<.05. 


Tukey’s  HSD  post-hoc  analyses  also  were  conducted  to  examine  differences  in  users’  perceived  appeal  among  three  structure 
designs.  There  was  significant  differences  between  decreasing  information  structure  (8  x 8 x 2 x 2)  and  increasing  information 
structure  (2  x 2 x 8 x 8).  That  is,  the  degree  of  users’  perceived  appeal  for  decreasing  information  structure  ^ = 20.65,  SD  = 
4.09)  was  higher  than  that  of  increasing  information  structure  (M  = 1 5.63,  SD  = 6.80),  £<  .05. 

Table  5 showed  that  there  was  statistically  significant  difference  in  the  degree  of  usability  in  terms  of  three  structure  designs, 
F (2,  55)=  5.61, £<.05. 
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Table  5.  An  ANOVA  Summary  Table  With  Group  Means  and  Standard  Deviations  for  Usability  by  Structure  Design 


Structure 

N 

M 

SD 

4 x4x4  x4 

19 

15,68 

1,92 

8 x8 x2  x2 

20 

17,30 

2,56 

2x2x8  x8 

19 

14,00 

4,28 

Source 

df 

SS 

MS 

F 

Structure 

2 

106,11 

53,05 

5,61* 

Error 

55 

520,31 

9,46 

Total 

57 

626.42 

Post-hoc  analyses  showed  that  there  was  significant  differences  between  decreasing  information  structure  (8  x 8 x 2 x 2)  and 
increasing  information  structure  (2  x 2 x 8 x 8).  The  degree  of  usability  for  decreasing  information  structure  (M  = 17,30,  SD  = 
2.56)  was  higher  than  that  of  increasing  information  structure  (M  = 1 4.00,  SD  = 4,28),  p < ,0 1 , 

Finally,  as  shown  in  Table  6,  there  was  also  statistically  significant  difference  in  the  degree  of  disorientation  in  terms  of  three 
structure  designs,  F (2,  55)  = 4.35,  p < .05. 


Table  6,  An  ANOVA  Summary  Table  With  Group  Means  and  Standard  Deviations  for  Disorientation  by  Structure  Design 


Structure 

N 

M 

SD 

4 x4x4  x4 

19 

10.68 

3,75 

8 x8x2  x2 

20 

8.40 

3,47 

2x2x8  x8 

19 

12.47 

5,51 

Source 

df 

MS 

F 

Structure 

2 

162.84 

81.42 

4.35* 

Error 

55 

1029.64 

18.72 

Total 

57 

1192.48 

*a<.05. 


Tukey’s  HSD  post-hoc  comparisons  results  revealed  that  there  were  significant  differences  between  decreasing  information 
structure  (8  x 8 x 2 x 2)  and  increasing  information  structure  (2x2  x 8 x 8).  The  degree  of  disorientation  for  decreasing 
information  structure  (M  = 8.40,  ® = 3.47)  was  lower  than  that  of  increasing  information  structure  (M  = 1 2,47,  SD  = 5.5 1 ),  g.  < 
.05. 

Discussions  and  Conclusions 

The  important  outcome  of  the  study  was  that  the  information  structure  had  an  influence  on  the  navigation  performance  of  the 
pull-down  menu  on  the  World  Wide  Web.  Results  showed  that  the  combination  of  decreasing  information  structure  and  a pull- 
down menu  resulted  in  faster  browsing  performance,  high  er  appeal,  higher  usability,  and  lower  disorientation,  while  performance 
and  attitudinal  assessments  were  worse  when  a pull-down  menu  was  combined  with  increasing  information  structure. 

As  reviewed  in  the  previous  section,  each  information  structure  was  organized  with  a total  of  256  pieces  of  information  in  a 
depth  of  four  levels.  However,  the  number  of  Web  pages  and  links  at  each  level  varies  in  each  case.  The  decreasing  information 
structure  has  more  links  than  the  increasing  information  because  a decreasing  information  structure  consists  of  more  information 
on  the  upper  level  and  less  information  on  the  lower  level,  while  an  increasing  information  structure  is  the  reverse. 

Customers  who  employ  a browsing  strategy  to  purchase  a product  or  service  need  more  information  not  only  for  narrowing 
down  their  purchasing  ideas  but  also  for  navigating  the  E-commerce  system.  It  is  important  for  customers  who  browsing  products 
to  provide  more  links  or  selections  because  they  can  see  more  products  without  additional  navigation  activities.  This  may  help 
customers  decide  the  product  to  buy.  The  result  of  users’  attitudes  supports  this  explanation.  Users  perceived  a Web  site  more 
appealing,  more  usable,  and  less  disorientating  under  a decreasing  information  structure  than  an  increasing  information  structure. 
Therefore,  the  differences  in  the  number  of  links  provided  by  information  structure  may  have  influenced  the  navigation  effect  of 
a pull-down  menu. 

This  study  failed  to  show  significant  differences  on  searching  performance.  There  are  three  possible  explanations  for  no 
significant  differences  on  searching  speed  being  found.  First,  the  information  structures  may  be  not  complex  enough  to  detect  the 
differences  of  the  speed  on  searching  performance.  All  three  information  structures  were  organized  with  256  information  in  four 
level  depths  with  different  shapes.  Even  though  each  structure  has  a different  number  of  links  and  of  possibility  of  error  selecting 
the  right  path,  the  differences  among  the  three  structures  may  not  be  complex  enough  to  reveal  the  interaction  with  different 
menu  designs  on  searching  performance.  Second,  the  level  of  task  difficulty  of  directed  browsing  may  have  been  too  low  to 
reveal  differences  among  three  information  structures.  The  tasks  for  directed  browsing  were  to  find  simple  information  in  this 
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study.  Lai’s  (1994)  study  showed  that  there  was  no  difference  on  the  participants’  searchingperformance  when  the  tasks  were  at 
a low  level  of  difficulty,  regardless  of  the  experimental  treatments.  This  study’s  results  are  consistent  with  her  findings. 

There  are  several  limitations  to  the  current  study.  First,  this  study  employed  only  three  information  structures  for  the 
expaiment.  However,  there  are  more  diverse  information  structures  in  the  real  Web  sites.  Therefore,  further  studies  should  be 
conducted  with  more  diverse  information  structures  in  order  to  investigate  the  information  searching  performance.  Second,  the 
information  structure  used  in  this  study  was  the  hierarchical  structure  with  four  pages  at  each  four  levels.  The  information 
structures  in  the  real  Web  sites  are  not  constant  but  varied.  The  research  should  expand  to  more  different  types  of  information 
structure  (e.g.  linear,  matrix,  network)  and  different  shapes  of  hierarchical  structures  with  different  level  of  depths.  These 
differences  in  information  structure  may  result  different  in  effects  of  menu  design.  Third,  the  population  of  participants  was 
limited  to  undergraduate  and  graduate  students  in  a mid-western  university.  Most  of  them  had  the  basic  skills  in  computer  and 
Web.  Further  research  should  expand  the  population  of  participants. 
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Preface 


For  the  twenty-  fourth  year,  the  Research  and  Theory  Division  of  the  Association  for  Educational 
Communications  and  Technology  (AECT)  is  sponsoring  the  publication  of  these  Proceedings. 
This  is  Volume  #2  of  the  Annual  Proceedings  of  Selected  Papers  On  the  Practice  of 

Educational  Communications  and  Technology  Presented  at  The  National  Convention  of  the 
Association  for  Educational  Communications  and  Technology  held  in  Atlanta,  GA.  A limited 
quantity  of  these  Proceedings  were  printed  and  sold.  It  is  also  available  on  microfiche  through 
the  Educational  Resources  Clearinghouse  (ERIC)  system. 

This  volume  contains  papers  primarily  dealing  with  instruction  and  training  issues.  Papers 
dealing  with  research  and  development  are  contained  in  the  companion  volume  (24'*’  Annual, 
Volume  #1),  which  also  contains  over  60  papers. 

REFEREEING  PROCESS;  Papers  selected  for  presentation  at  the  AECT  Convention  and 
included  in  these  Proceedings  were  subjected  to  a reviewing  process.  All  references  to  authorship 
were  removed  from  proposals  before  they  were  submitted  to  referees  for  review.  Approximately 
fifty  percent  of  the  manuscripts  submitted  for  consideration  were  selected  for  presentation  at  the 
convention  and  for  publication  in  these  Proceedings.  The  papers  contained  in  this  document 
represent  some  of  the  most  current  thinking  in  educational  communications  and  technology. 


M.  R.  Simonson 
Editor 
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Abstract 

The  purpose  of  this  study  was  to  examine  problems  with  the  current  computer-based  electronic 
communication  systems  and  to  initially  test  and  revise  a new  and  different  paradigm  for  e-collaboration  — 
Relate@IU,  Understanding  the  concept  of  sending  links  to  resources,  rather  than  sending  the  resource 
itself  is  at  the  core  of  how  Relate@IU  differs  from  the  traditional  communications  models.  Instructional 
Systems  Technology  faculty  and  students  were  invited  to  participate  in  focus  group  sessions,  where  they 
were  presented  with  an  initial  prototype  of  Relate@IU. 

Our  findings  from  the  focus  group  sessions  indicated  that  the  current  electronic  communication  systems 
are  meeting  user  needs,  while  generating  new  needs  that  currently  are  not  being  met.  This  report  reveals 
these  newly  generated  needs  stemming  from  the  problems  with  current  electronic  communication  systems. 
Relate@IU  is  intended  to  solve  many  of  these  emerging  problems. 

Following  the  initial  testing  of  the  Relate@IU  prototype,  we  developed  an  updated  computerized  version  of 
Relate@IU,  named  Share@IU.  IVe  are  currently  developing  PHP  scripts  which  will  be  tested  on  a small 
scale. 

Introduction 

Technology  utilization  in  higher  education  has  been  witness  to  a dramatic  increase  in  recent  years.  Two 
main  areas  of  increase  have  been  in  electronic  mail  (Email),  and  Web  technologies.  “Three-fifths  (59.3  percent)  of 
all  college  courses  now  utilize  electronic  mail,  up  from  54.0  percent  in  1999,  44.0  percent  in  1998  and  20.1  percent 
in  1995.  Similarly,  two-fifths  (42.7  percent)  of  college  courses  now  use  Web  resources  as  a component  of  the 
syllabus,  up  from  10.9  in  1995,33.1  percent  in  1998  and  38.9  percent  in  1999.  Almost  a third  (30.7  percent)  of  all 
college  courses  have  a Web  page,  compared  to  28.1  percent  in  1999,  22.5  percent  in  1998  and  9.2  percent  in  1999” 
(The  Campus  Computing  Project). 

Well-known  problems  exist  with  e-mail  and  asynchronous  conferencing  systems.  Many  people  get  flooded 
with  e-mail  messages  they  do  not  want  in  their  inboxes  (“e-mail  assault”  or  “Spam”).  Negotiation  in  advance 
between  a sender  and  a recipient  to  agree  to  communicate  is  not  allowed  under  the  current  e-mail  paradigm;  e-mail 
filters  are  often  marginally  effective.  Furthermore,  if  one  sends  a large  attachment  to  a group,  then  a copy  of  the 
attachment  is  delivered  to  each  recipient’s  inbox  and  may  cause  inboxquota  overflows  as  well  as  network 
congestion. 

While  we  have  seen  a dramatic  increase  in  the  utilization  of  communication  technologies,  there  has  been  a 
distinct  absence  of  new  communication  paradigms  to  match  the  current  needs  of  users.  Current  literature  suggests 
that  “...communication  may  be  a more  important  use  and  determinant  of  participants’  commitment  to  the  Internet 
than  is  information  acquisition  and  entertainment”  (Kraut,  Mukhopadhyay,  Szczpula,  Kiesler,  and  Scherlis,  1998). 
Problems  still  reside  with  the  current  electronic  communication  systems  and  a better  paradigm  needs  to  be 
considered. 

Literature  Review 

Computer-Based  Communication  Systems 
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Computer-based  communication  systems  require  three  basic  components:  hardware,  software,  and 
telecommunication  lines  (Harasim,  Hiltz,  Teles,  & Turoff,  1995).  Hardware  components  include  a personal 
computer  or  workstation  and  an  Internet  connection.  A Second  component  is  the  software  used  for  group  interaction 
(e.g.,  bulletin  boards,  electronic  mail,  and  computer  conferencing  system).  Lastly,  telecommunication  lines  are 
needed  to  link  the  computers  to  groups  of  people  to  communicate  and  learn  together. 

Problems  with  the  Current  Computer-Based  Communication  Systems 

E-mail.  An  E-mail  system  is  an  electronic  data  transfer  tool  to  exchange  messages  over  networks.  Most  E- 
mail  systems  have  capabilities  for  attaching  files  to  messages  to  facilitate  the  exchange  of  large  amounts  of 
information  (Harasim  et  al.,  1995).  The  distributive  nature  of  E-mail  has  lead  to  some  of  the  current  problems  being 
experienced  by  users.  One  such  example  of  this  is  the  phenomenon  of  attachments.  Graphical  E-mail  applications 
such  as  Microsoft  Outlook  and  Eudora  allow  for  documents  to  be  attached  to  an  E-mail  message  with  ease.  This 
attached  document,  however,  has  the  potential  to  spread  information,  or  a virus,  with  extreme  speed.  Also,  the  file 
size  of  attached  documents  is  large  relative  to  a normal  E-mail  message.  This  large  size  can  lead  to  an  over-quota 
status  of  the  recipients  E-mail  account.  When  such  accounts  are  over-quota,  other  electronic  messages  cannot  be 
delivered.  There  are  numerous  other  examples  of  the  problems  users  face  every  day  with  E-mail  which  will  not  be 
discussed  here. 

Computer  collaboration  systems.  According  to  Harasim  et  al  (1995),  computer  collaboration  systems  allow 
joint  collaboration  in  a method  other  than  exchanging  isolated  pieces  of  information  (i.e.  Email  attachments);  It 
provides  groups  with  specific  spaces  inside  the  software  that  can  be  tailored  to  their  needs.  Each  group 
communication  space  has  access  privileges  set  by  the  person  who  creates  the  space.  Each  collaboration  space 
provides  lists  that  allow  participants  to  tell  who  has  accessed  what  material.  Changes  on  earlier  contributions  and 
notifications  of  changes  are  also  possible.  Contrary  to  such  strengths,  however,  computer  collaboration  systems 
have  some  weaknesses.  The  most  common  difficulty  is  technical  problems:  people  do  not  know  how  to  use 
collaboration  systems.  They  are  lost  in  a system,  fail  to  edit  online,  and  have  difficulties  uploading  and  downloading 
documents  (Harasim  et  al.,  1995).  These  problems  relate  to  the  specifics  of  the  collaboration  system  or  the  interface 
between  the  hardware  and  software  - there  are  various  types  and  versions  of  collaboration  systems.  Another 
problem  is  communication  anxiety,  which  comes  from  lack  of  immediate  or  appropriate  responses  within  a 
reasonable  time.  This  is  especially  true  for  asynchronous  environments  where  immediate  interactions  are  not 
possible.  People  also  have  difficulties  managing  increased  information  flows.  With  networking  and  increased  access 
to  education  and  information,  the  key  challenge  becomes  learning  how  to  manage  the  information  overflows. 

Transformation  to  Peer-to-Peer  Computing 

Peer-to-Peer  computing  provides  individual  users  with  direct  connections  to  desktop  computers  for 
communicating.  This  is  in  contrast  to  the  more  familiar  centralized  model  of  computing  used  for  broadcasting 
information  and  electronic  commerce.  In  the  following  passage,  Lewis  (2000)  describes  the  transformation  from  a 
centralized  model  of  computing  to  a peer-to-peer  model. 

“The  Internet  transformed  the  world  by  linking  computer  networks.  The  World  Wide  Web  transformed  the 
Internet  by  making  it  easy  to  link  files  on  those  computers.  Now  another  major  transformation  is  occurring.  By 
enabling  millions  of  computer  users  to  search  for  files  and  transfer  them  from  one  desktop  computer  to  another, 
instead  of  the  current  model  in  which  files  are  typically  stored  on  and  retrieved  from  a central  Internet  computer 
called  a server,  the  balance  of  power  shifts  from  the  commercial  interests  that  now  dominate  the  Internet  to  the 
individual.”  (Lewis,  2000) 

Since  the  spring  of  2000,  peer-to-peer  computing  has  been  gaining  momentum  in  the  business  community. 
This  spur  in  momentum  is  largely  to  due  the  growing  popularity  of  Napster,  the  music  exchange  service.  One 
company  who  has  been  leading  the  way  with  peer  computing  innovations  is  Groove  Networks.  They  have  developed 
an  Internet  communications  software  program  called  Groove,  which  allows  people  with  shared  interests  to  make 
direct  connections  for  real-time  interaction.  According  to  the  company,  “Groove  moves  beyond  the  World  Wide 
Web  paradigm,  leveraging  the  two-way  capabilities  of  the  Internet  to  provide  a peer  computing  platform  for  use  by 
individuals  and  small  groups,  in  both  a business  and  personal  context.” 

According  to  Groove  Networks,  peer  computing  innovations  have  come  in  three  different  flavors:  Direct 
access  to  information.  Direct  access  to  computing  power  and  Direct  access  to  people.  At  the  core  of  these  three  peer- 
to-peer  innovations  is  the  type  of  ACCESS  we  as  individuals  require/enjoy.  Peer-to-Peer  computing  places  the  user 
in  control.  It  is  clearly  a user-centered  approach  to  computing.  People  are  attracted  to  peer  computing  for  (a) 
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creating  a shared  context  among  a group,  (b)  having  the  flexibility  to  add  functions  on  an  as  needed  basis,  and  (c) 
making  connections  without  having  to  go  through  a centralized  resource.  These  reasons  naturally  combine  to  allow 
people  to  gather  together  to  make  informed  decisions  quickly. 

Purpose  of  the  Study 

The  purpose  of  this  study  is  to  examine  major  problems  and  needs  with  the  current  computer-based 
communication  systems  and  to  initially  test  and  revise  a new  and  different  paradigm  for  e-collaboration  - 
Relate@IU. 

Research  Questions 

This  research  focused  on  two  sets  of  questions.  The  first  set  focused  on  the  general  use  of  the  current 
electronic  communication  systems.  The  second  set  focused  on  the  capabilities  of  Relate@IU.  The  general  questions 
were:  (a)  How  are  you  using  electronic  communications  technologies?  (b)  What  major  problems  have  you 
encountered  with  electronic  communications  technologies?  (c)  What  types  of  improvements  would  you  like  to  see  in 
the  future?  (d)  If  you  could  dream  of  a system,  any  system,  that  could  make  your  communication  life  simpler,  what 
would  it  look  like?  and  (e)  Do  you  feel  that  current  online  collaboration  tools  are  meeting  your  needs?  The  specific 
questions  related  to  Relate@IU  were:  (a)  Are  there  any  aspects  of  the  prototype  that  still  need  clarification?  (b) 

What  problems  do  you  see  with  the  current  state  of  the  prototype?  (c)  What  critical  characteristics  of  online 
collaboration  are  missing  from  the  prototype  that  you  would  like  to  see  included?  and  (d)  How  would  you  improve 
the  interface? 

Methodology 

Relate@lU 

Relate@lU  is  not  the  same  as  e-mail,  the  Web,  or  asynchronous  conferencing  as  typically  implemented.  It 
is  a new  e-collaboration  tool  that  is  distinguished  from  the  current  computer-based  electronic  communication 
system. 


Table  I.  Current  Computer-Based  Communication  Systems  vs.  Relate(a)JU 


Current  Systems 

Relate@IU 

E-mail 

Collaboration  Tool 

Relationship 

Not  negotiated 

Negotiated 

Negotiated 

Location 

Multiple  E-mail  Servers 

Shared  Server 

My  Web  Space 

of  Files 

Ownership 

Creator  & Recipients 

Creator  & Recipients 

Creator 

Item 

Message,  File  & Link 

Message,  File,  & Link 

Subject  Line  & Link 

Exchanged 

Number 

Number  of  recipients 

One 

One 

of  Copies 

Negotiation  of  relationships.  With  the  current  computer-based  electronic  communication  systems, 
relationships  are  not  negotiated.  With  e-mail  addresses,  anyone  can  send  another  person  as  much  information  or  as 
many  files  as  they  like.  Regarding  unwanted  e-mails,  people  spend  several  hours  a week  just  deleting  them.  No 
control  exists  between  a sender  and  a recipient,  and  it  makes  people  suffer  from  such  problems  as  “e-mail  assault”, 
over-quota  in  the  recipients*  e-mail  inboxes,  etc.  In  order  to  increase  effective  and  efficient  use  of  electronic 
communication  systems,  selective  information  needs  to  be  exchanged  with  selective  people.  With  the  new  system,  it 
is  envisioned  that  negotiation  must  occur  among  people  who  agree  to  collaborate. 

File  linking  system.  With  our  current  electronic  communication  systems,  we  send  out  messages,  files,  or 
links.  Besides  unwanted  e-mails,  what  slows  down  swift  communication  is  the  exchange  of  large  files.  Under  the 
current  e-mail  systems,  files  are  sent  out  as  attachments.  If  modification  is  needed  after  a file  is  sent  out,  a different 
version  of  file  is  sent  out.  If  modifications  are  made  frequently,  outdated  files  pile  up  in  the  recipient’s  mailbox  until 
the  recipient  deletes  them;  this  can  cause  the  mailbox  to  reach  its  quota  and  in  worst  case,  new  e-mail  messages  are 
rejected  by  the  system.  Under  the  new  system,  people  never  send  out  files;  what  is  sent  to  others  in  the  collaborating 
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group  is  a link  to  the  owner’s  file.  The  new  system  is  a file  linking  system,  and  the  access  to  the  file  is  restricted 
exclusively  to  the  collaborating  group. 

One  file  in  one  place  - control  on  ownership.  With  the  current  electronic  communication  system, 
exchanged  files  reside  in  e-mail  servers  or,  in  the  shared  server  of  an  electronic  collaboration  system.  In  order  to 
distribute  a file  to  multiple  individuals,  essentially  the  initiator  is  making  multiple  copies  of  that  file.  Under 
Relate@IU.  every  person  saves  his  own  files  on  his  Web  space  and  can  exercise  controls  on  the  relationships  with 
people  in  terms  of  accesses  to  the  files:  read-only  or  modification-enabled.  Files  are  never  sent  out  and  remain 
property  in  possession  of  the  owner,  and  people  are  given  permission  to  view  and/or  modify  the  files.  However, 
once  another  person  is  viewing  a file,  he  or  she  normally  cannot  be  prevented  from  saving  it  on  his/her  own  storage 
device  (i.e.,  cannot  be  prevented  from  making  a duplicate  copy). 

Design  Prototype 

The  researchers  held  a brainstorming  session  to  begin  to  conceptualize  the  fundamental  principles  of  the 
new  electronic  communications  system,  Relate@IU.  One  of  the  important  products  of  our  brainstorming  session  was 
a comparison  table  highlighting  the  key  conceptual  differences  between  the  current  electronic  communications 
systems  and  the  ReIate@IU  system  (see  Table  1).  Also,  sketches  of  the  initial  prototype  and  development  notes  were 
developed  during  the  brainstorming  session.  From  these  sketches,  we  were  able  to  design  an  initial  prototype  to 
present  during  our  focus  group  sessions. 

Fortunately,  our  focus  group  audience  understood  the  nature  of  rapid  prototyping,  and  that  this  initial 
prototype,  although  presented  in  an  electronic  form  (Microsoft  PowerPoint),  was  just  the  first  of  many  iterations. 

Configurations  of  current  electronic  mail  (e-mail)  systems  drove  the  development  of  the  Relate@IU 
prototype.  Current  e-mail  systems  can  organize  messages  in  ascending  or  descending  order  by  when  the  message 
was  received,  who  sent  the  message,  if  the  message  was  red  flagged  (used  to  note  high  importance),  or  if  the 
message  contains  an  attachment.  Beyond  this,  users  also  have  the  ability  to  sort  their  messages  into  sub-folders. 

This  organizational  structure  was  also  incorporated  into  the  Relate@IU  prototype.  Our  prototype  presents  users 
with  sub-folders  that  represent  each  relationship  they  belong  to.  From  this  point,  requests  are  presented  in  the  order 
that  they  were  received;  however,  they  could  be  re-organized  in  ascending  or  descending  order  by  action  requested 
(read  or  review),  subject,  file  location,  or  date  received. 

At  this  point,  a functional  overview  of  the  current  prototype  may  be  beneficial.  To  start,  a user  would  open 
their  ReIate@lU  web  page,  and  then  be  prompted  to  log  into  the  system.  The  next  screen  to  be  displayed  would 
have  a list  of  current  relationships  on  the  left,  a toolbar  to  create  relationships  and  send  requests  at  the  top,  and  the 
main  information  display  window  located  to  the  right  of  the  relationships  list.  The  main  information  display  area 
contains  two  types  of  information,  requests  that  individuals  or  groups  have  made  to  you,  and  your  requests  to 
individuals  or  groups.  The  content  that  is  displayed  in  this  area  is  controlled  by  your  selection  of  an  existing 
relationship  from  the  relationships  list  on  the  left  of  the  screen.  The  tool  menu  at  the  top  of  the  screen  also  affects 
the  relationship  that  has  been  selected  from  the  relationships  list  on  the  left  side  of  the  screen. 

Forming  relationships  can  be  done  at  any  time  by  selecting  the  “Create  / Modify  Relationship”  tool.  A 
pop-up  window  appears  and  allows  the  user  to  enter  a title  of  the  relationship,  who  is  involved  in  the  relationship, 
and  what  privileges  they  have  at  the  folder  level  (read  only  or  modify). 

The  ReIate@IU  system  differs  from  the  traditional  e-mail  system  in  the  fact  that  it  does  not  send  an  actual 
message.  This  new  system  sends  a subject  line  and  a link  to  a file.  This  file  could  be  anything  from  a word 
processing  document,  to  a digital  image,  to  an  audio  file. 

Understanding  the  concept  of  sending  links  to  resources,  in  place  of  sending  the  resource  is  at  the  core  of 
how  ReIate@IU  differs  from  the  traditional  communications  models.  In  order  to  facilitate  our  focus  group 
participants  understanding  of  this  concept,  we  first  asked  them  a series  of  questions  designed  to  have  them  think 
about  their  experiences  with  current  electronic  communications  systems. 

Next,  we  presented  a comparison  table  highlighting  the  key  conceptual  differences  between  the  current 
communications  systems  and  the  ReIate@IU  system.  At  this  point,  we  showed  the  focus  group  participants  our 
prototype,  and  talked  through  a scenario  of  how  the  tool  would  be  used. 

The  Focus  Group  Interview 

We  used  focus  groups  as  a primary  method  of  data  collection  in  order  to  stimulate  new  ideas,  creative 
concepts,  diagnose  potential  problems  and  to  generate  impressions  of  our  new  electronic  communications  system, 
ReIate@IU.  We  conducted  three  separate  focus  group  interviews.  Two  of  these  sessions  were  student  focus 
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groups,  and  the  third  session  was  a faculty  focus  group.  We  collected  the  data  during  the  focus  groups  by  video 
taping  the  sessions.  Stewart  and  Shamdasani  (1990)  recommend  ordering  the  guiding  questions  from  general  to 
more  specific.  An  agenda  containing  guiding  questions  for  the  focus  group  sessions  was  designed  by  the  researchers 
to  guide  the  flow  of  the  discussion.  Each  participant  was  provided  with  a copy  of  the  agenda.  The  researchers 
decided  to  conduct  the  focus  group  sessions  prior  to  the  one  on  one  interviews  in  order  to  solve  initial  conceptual 
problems  with  the  prototype.  In  addition,  the  researchers  wanted  to  gather  data  that  would  aid  in  improving  the 
interface  of  the  prototype,  prior  to  presenting  it  in  a one  on  one  interview  situation.  According  to  Morgan  (1997) 
“...follow-up  interviews  can  help  provide  depth  and  detail  on  topics  that  were  only  broadly  discussed  in  group 
interviews.”  Focus  groups  are  a somewhat  informal  technique  that  can  help  assess  user  needs  and  feelings  both 
before  interface  design  and  long  after  implementation  (Nielson,  1997). 

Research  Site 

Participants  for  this  study  were  drawn  from  the  Indiana  University  Bloomington  (lUB)  campus,  and  more 
specifically  the  Instructional  Systems  Technology  Department  (1ST).  The  1ST  Department  is  located  within  the 
School  of  Education.  We  conducted  our  focus  group  sessions  in  the  main  1ST  classroom. 

Research  Participants 

A purposeful  sampling  approach  was  used  in  this  study  in  order  to  maximize  variation  among  the  research 
participants.  Purposeful  sampling  is  used  as  a strategy  when  one  wants  to  learn  something  and  come  to  understand 
something  about  certain  select  cases  without  needing  to  generalize  to  all  such  cases  (Patton,  1980).  Each  participant 
was  contacted  via  e-mail,  informed  of  the  purpose  of  the  study,  and  was  asked  to  participate.  In  particular,  we  were 
interested  in  selecting  a variety  of  1ST  masters  and  doctoral  students,  early  or  late  in  their  program,  spoke  English  as 
a first  or  second  language  and  were  male  or  female.  In  addition,  we  were  also  interested  in  selecting  1ST  faculty. 

Study  participants  include  three  masters,  four  doctoral,  and  three  faculty.  Among  the  students,  three  were 
male  and  four  were  female,  and  among  the  faculty,  one  was  a female  and  two  were  male.  All  except  for  two 
participants  spoke  English  as  a first  language. 

Results 

General  Questions 

Below  are  the  general  questions  from  the  focus  group  interview,  followed  by  key  points  that  were  raised. 

1.  How  are  you  using  electronic  communication  technologies? 

All  participants  indicated  that  they  utilized  SiteScape  Forum  (SSF)  for  document  sharing  with  classes.  E- 
mail  was  also  used  by  all  participants  for  academic  and  personal  communications.  One  participant  stated  that  File 
Transfer  Protocol  (FTP)  was  still  utilized  for  posting  documents  to  a server  space.  Most  participants  stated  that  they 
utilized  online  search  engines  to  locate  information  and  updates  on  the  World  Wide  Web  (WWW). 

2.  What  major  problems  have  you  encountered  with  electronic  communications  technologies? 

A majority  of  participants  reported  problems  with  a lack  of  system  interoperability  (e.g.  MS  Word  and 
WordPerfect,  Macintosh  and  PC  platforms).  This  was  mirrored  by  a large  number  of  participants  reporting 
difficulties  in  moving  a document  from  one  system  to  another. 

Electronic  mail  was  another  source  of  difficulties  for  our  participants.  Few  reported  problems  with  junk 
mail  (Spam),  about  half  reported  difficulty  with  reading  attachments  (associated  with  the  PINE  e-mail  system). 
Quickly  reaching  the  maximum  quota  (file  storage  allotment)  was  also  of  concern  to  about  half  of  our  participants. 

A few  participants  commented  about  the  difficulty  in  replying  to  challenging  e-mails,  stating  that  they  don’t  know 
how  to  answer  or  what  to  do  with  the  message. 

When  dealing  with  multiple  communications  systems,  a majority  of  participants  had  commented  on  the 
tedium  of  logging  in  to  multiple  forums,  specifically  with  SSF,  and  then  having  to  remember  not  only  multiple 
passwords,  but  also  which  forum  to  access.  A few  participants  had  difficulty  with  the  unintuitive  interface  with 
SSF.  Our  faculty  focus  group  noted  that  there  is  no  integration  between  the  various  commonly  used 
communications  systems  such  as  SSF  and  e-mail.  SSF  server  downtime  was  noted  by  a just  under  half  of  our 
participants  as  a major  source  of  difficulty. 
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3/4.  What  would  your  dream  system  look  like? 

Although  we  did  receive  a range  of  responses  to  this  question,  there  was  no  one  feature  that  was  repeated 
with  a high  frequency.  Almost  half  of  the  participants  had  indicated  that  they  wished  to  have  greater  control  of  how, 
and  with  whom,  they  collaborate.  One  third  of  the  participants  would  like  to  see  human-like  intelligence  with 
advanced  filtering,  prioritizing,  and  automatic  archiving  of  electronic  mail  and  data.  The  ability  to  customize  the 
system,  and  visual  communications  were  both  mentioned  by  one  fifth  of  our  participants.  There  were  a number  of 
responses  that  were  unique.  These  suggestions  ranged  from  a seamless  integration  of  the  most  commonly  utilized 
systems,  to  having  a system  be  able  to  recognize  what  type  of  file  is  being  used,  to  a tracking  and  check-in  / check- 
out system  for  working  with  documents  in  an  online  collaborative  file  sharing  environment.  One  common  dream 
was  that  of  unlimited  storage  space 

5.  Do  you  feel  the  current  system  is  meeting  your  needs? 

Half  of  all  participants  responded  that  yes,  the  current  communications  systems  are  meeting  their  needs. 
However,  there  were  a number  of  conditions  that  were  placed  on  this  response.  One  condition  was  that 
collaboration  at  a distance  works  with  a relatively  small  number  of  people.  Another  condition  is  that  the  current 
technology  is  forcing  users  to  follow  specific  rules,  in  other  words  users  are  submissive  to  the  technology.  One 
participant  stated  that  due  to  limited  experience  with  electronic  collaboration,  that  they  were  uncertain  as  to  what 
there  needs  were,  and  therefore  was  uncertain  if  the  current  systems  were  meeting  those  needs.  Another  participant 
was  not  satisfied  with  the  inadequacies  in  the  discussion  type  tools,  particularly  for  students  and  classes.  A few 
participants  made  the  comment  that  current  online  collaboration  tools  are  meeting  many  of  my  needs  and  generating 
new  needs  that  are  not  being  met. 

Responses  to  the  Prototype 

Below  are  the  focus  group  responses  to  the  current  Relate@lU  prototype. 

Comment  1:  If  you  send  a link  to  10  people,  how  do  you  know  who  read  it  if  when  one  person  reads  it  and  it 
changes  to  non-bold? 

At  the  present  time  there  is  no  understanding  on  our  part  as  to  how  this  tool  could  display  which  members 
of  a group  have  opened  the  document,  and  who  has  not. 

Comment  2:  Would  it  send  you  some  sort  of  notification  (if  someone  sends  you  a request  to  collaborate)? 

The  current  system  was  designed  with  the  idea  that  it  would  be  its  own  system,  not  tied  into  other  lU 
systems.  People  would  agree  to  work  together  via  current  methods,  and  then  go  to  the  Relate@lU  tool  to  assist  them 
in  their  collaboration.  Relate@lU  was  not  viewed  as  an  initial  step;  however  there  is  room  for  redesign. 

Comment  3:  I’m  thinking  four  people  in  a group,  3 classes,  that’s  12  individuals  - do  I want  to  set  up  12 
relationships? 

For  three  different  groups  you  would  have  to  form  three  different  relationships.  It  is  possible  to  have 
multiple  individuals  involved  in  one  relationship.  SiteScape  Forum  (SSF)  might  be  a more  appropriate  tool  for 
collaboration  in  cases  similar  to  this. 

Comment  4:  So  this  wouldn’t  be  for  sharing  documents  with  a group,  I would  share  my  stuff,  but  it’s  not  a 
place  where  the  group  has... 

A group  of  individuals  may  have  access  to  read  or  modify  documents  that  currently  exist  in  your  web 
space;  however,  the  current  design  of  the  system  does  not  allow  users  to  place  a file  in  another  user’s  space.  One 
member  of  a group  can  send  a link  to  the  file,  and  other  members  of  the  group  can  access  the  file,  but  there  is  no 
central  repository  for  the  group  to  store  documents. 

Comment  5:  (A)  Problem  would  be  that  there  is  no  identification  of  who  is  sending  a request  when  multiple 
people  belong  to  a group  (i.e.  project  team) 

Currently,  the  system  recognizes  users  as  either  an  individual  user,  or  as  one  user  accessing  an  account  that 
has  multiple  users.  The  system  can  be  redesigned  to  utilize  usernames  to  identify  the  individual  who  sent  the 
request,  as  well  as  to  identify  which  users  have  accessed  documents. 

Comment  6:  All  most  everyone  I know  I have  multiple  types  of  relationships  with,  how  would  I manage  that? 
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Comment  7:  What  Fm  seeing  is  that  the  relationships  are  all  about  document  sharing. 

Relate  Becomes  Share@IU 

A paper  prototype  was  built  to  simulate  the  functions  that  were  determined  in  the  Relate@IU  research 
study.  Usability  tests  were  conducted  with  this  paper  prototype.  Some  problems  were  found  with  the  paper  prototype 
and  users  suggested  some  changes  to  this  prototype.  Due  to  the  nature  of  this  study,  we  were  open  to  ideas  and 
suggestions  of  the  users.  Furthermore  with  the  considerations  of  these  problems  and  suggestions,  two  more 
computer  prototypes  were  developed  and  further  usability  tests  were  done.  During  this  development  stage,  we  were 
determined  to  develop  a user-friendly  system.  If  this  new  system  is  seen  as  a new  tool  to  learn  or  require  training, 
people  might  resist  using  it.  We  wanted  to  make  sure  that  it  was  usable  by  various  types  of  computer  users  and  we 
tested  this  with  a wide  variety  of  users. 

When  we  started  to  mock  up  Share@IU,  we  identified  the  critical  functions  required  for  the  system.  We 
recognized  that  there  were  fundamentally  three  different  categories  of  functions:  File  management,  communication 
and  group  functions.  We  identified  the  sub  functions  under  these  categories.  The  File  Management  system  is  similar 
to  Windows  Explorer  or  generic  FTP  programs.  We  developed  a similar  interface  for  the  File  Management  System 
part  of  Share@IU.  Communication  and  group  level  functions  were  added  to  this  interface. 

In  the  File  Management  Functions,  the  user  (owner  of  Share@IU  space)  can  create  a folder,  upload  files, 
determine  where  s/he  is  working,  navigate  between  folders  (up  a level  or  down  to  subfolders),  and  determine  the 
files  in  the  folder  with  their  size  and  modification  date.  In  the  Group  Functions,  the  user  can  determine  who  can 
have  access  to  that  specific  folder.  In  the  Communication  Functions,  the  user  can  send  a message  to  the  group  about 
the  uploaded  files  (see  Figure  1). 

In  the  following  figures,  some  shots  of  the  electronic  prototype  of  Share@IU  are  shown.  This  is  the  view  of 
a Share@IU  owner.  If  the  user  sends  a message  to  his/her  group,  they  don’t  see  the  files  in  this  way  they  just  follow 
the  link,  and  can  see  what  is  allowed  for  him/her. 
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Figure  I : Share@IU  Computer  Prototype:  General  View  (Separator  lines  are  superimposed  here,  but  not  seen  by 
user) 
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Figure  2:  Share@IU  Computer  Prototype:  Choosing  a GroupType 


The  user  determines  his/her  folder  access  level.  S/he  can  choose  either  of  these  options  and  accordingly  sets 
the  folder  accessibility  to  the  defined  group  (see  Figure  2). 


Figures : Share@IU  Computer  Prototype:  Creating  Own  Group 

If  the  user  wants  to  determine  specific  users,  s/he  can  choose  “create  my  own  group”  option  (see  Figure  2) 
and  determine  the  user  ids  and  add  them  to  the  list  (see  Figure  3). 
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Figure4:  Share@IU  Computer  Prototype:  After  Group  Access  Changes 

After  user  determines  the  accessibility  of  the  folder,  the  folder’s  accessibility  level  changes  and  files  and 
folders  in  that  folder  change  automatically  (see  Figure  4). 
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Figure  5:  Share@IU  Computer  Prototype:  Group  Notification  of  the  Uploaded  Files 

If  the  user  wants  to  inform  the  group  about  the  uploaded  files,  s/he  can  send  a message  by  clicking  on  “E- 
mail  group”  option  (see  Figure  4).  They  can  add  a message  and  subject  for  their  e-mail  (see  Figure  5) 

Limitations  and  Future  Plans 
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Due  to  purposive  sampling  in  the  initial  qualitative  study,  generalizability  of  findings  beyond  the  specific 
group  investigated  is  not  warranted.  The  results  were  nonetheless  useful  in  shaping  the  direction  of  subsequent 
prototypes,  and  further  usability  testing  of  those  prototypes  helped  to  improve  their  design. 

Share@IU  is  still  a prototype  (simulation)  that  has  not  yet  been  implemented  as  a working  product.  As  we 
develop  a functioning  product,  additional  usability  tests  will  be  conducted  to  refine  the  design.  It  is  our  hope  that 
Share@IU  can  be  implemented  on  a variety  of  operating  systems  (e.g.,  Unix,  Linux,  MS  Windows,  Macintosh) 
running  standard  Web  servers  (e.g.,  Apache,  IIS).  Scripts  will  be  written  in  PHP,  which  will  run  as  CGI  applications 
on  the  server  side.  Users  will  be  able  to  access  Share@IU  through  standard  Web  browsers  (e.g.,  Netscape,  Explorer, 
AOL,  Opera)  with  normal  Internet  connections  (PPP),  and  read  messages  generated  by  Share@IU  with  any  standard 
e-mail  system  (e.g.,  Eudora,  Outlook,  Messenger). 
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Collaborative  Cultural  Studies  over  the  Internet;  Learning  cultures  with  virtual  partners. 
A Project  between  Baylor  University  and  Tokyo  Institute  of  Polytechnics 

Masamichi  Okubo 
Tokyo  Institute  of  Polytechnics 
Hajime  Kumahata 
Baylor  University 


Introduction 

“Culture  is  a complex,  abstract,  and  pervasive  matrix  of  social  elements  that  functions  as  an  all- 

encompassing  form  of  pattern  for  living  by  laying  out  a predictable  world  in  which  an  individual  is  firmly 

oriented.” 

Richard  Porter  & Larry  Samover 

Offering  a predictable  world  and  orienting  students  in  that  controlled  environment  offer  the  faculty  a new 
level  of  challenge.  Technology  makes  this  task  possible.  The  ultimate  goal  of  this  project  challenges  us  to  create 
and  provide  a communicative  environment  in  which  is  most  natural  to  L2  and  C2'.  We  seek  to  teach  in  the  world 
where  the  instructors  and  students  coexist  in  non-artificial  simulation. 

The  computer-assisted  classroom  is  no  longer  a simulation,  but  a real  life.  The  idea  of  “Learning  can  be 
undertaken  in  state-of-the-art  classrooms,  but  can  also  be  integrated  into  the  living  arrangements”  (Gilman)  is 
obsolete.  In  21®*  century  life,  our  lives  globally  connect  to  each  other  with  technological  network.  The  networked 
life  is  the  living  life.  In  that  environment,  one  of  the  most  important  issues  is  the  understanding  of  L2  and  C2  to 
make  successful  communications. 

Since  the  fall  semester  of  1995,  students  at  Baylor  University  taking  Japanese  and  students  at  Tokyo 
Institute  of  Polytechnics  have  had  opportunities  to  communicate  each  other  and  learn  together  at  in  a real- world 
situation.  Until  the  fall  2000,  the  format  limited  them  to  e-mail  communication.  Today,  we  have  begun  a new 
adventure  in  learning  culture  and  communicating  appropriately  with  the  colleagues  from  the  other  side  of  this  planet 
using  other  Internet  tools. 

Software  and  Hardware  Used 

Communication  software,  iVisit,  utilizing  web-based  camera  provides  an  instant  visual,  oral,  and  aural 
transoceanic  communication  along  with  text  chat.  This  method  brings  a classroom  halfway  across  the  globe  next 
door.  We  use  this  web-based  free-of-charge  software  in  the  introductory  and  concluding  discussions.  Due  to  our 
hardware  limitation,  each  school  used  one  web-based  camera. 

FirstClass  offers  students  to  communicate  in  the  manner  of  a bulletin  board  service.  It  allows  students  to 
leave  messages,  possibly  enhanced  with  graphical  and  audio  files,  anytime  of  the  day  from  anywhere  in  the  world. 

It  allows  students  to  read  the  postings  and  share  ideas.  Contrary  to  iVisit,  FirstClass  gives  time  to  students  to  think 
and  examine  what  to  post.  The  client  software  is  free-of-charge  and  supports  many  different  platforms  and 
languages. 

Criteria 


For  this  study,  Baylor  University  selected  the  parameter  range  of  students  to  be  3*^*^  semester,  150  contact 
hours,  or  higher.  Tokyo  Institute  of  Polytechnics  has  selected  students  from  Intercultural  Communication  Studies 
who  have  been  studying  English  for  at  least  6 years.  Students  from  both  universities  interacted  in  LI  and  L2  to 
complete  the  task. 

The  Project  Time  Table 

Introductory  iVisit  session 


’ LI  is  the  first  language  and  L2  is  the  second  language.  Cl  is  the  culture  of  the  students’  own,  and  C2  is  the  culture 
which  student  is  learning  as  the  second  culture. 
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At  the  end  of  September  2000,  the  first  iVisit  session  took  place.  Baylor  class  met  at  7p.m.  to 
accommodate  the  time  difference  of  14  hours.  Students  answered  Pre-Project  questionnaires  to  examine  what 
technological  and  cultural  experiences  they  have  had  and  how  those  experiences  will  change  them  at  the  completion 
of  the  project. 

During  this  one-hour  introductory  session,  instructors  shared  the  intention  and  schedule  of  the  project  first. 
Then  many  students  took  the  opportunity  to  introduce  themselves  and  asked  some  questions  such  as  weather, 
fashion  fads,  popular  music,  politics,  and  other  current  events.  Some  questions  were  prepared  and  others  were 
spontaneous. 

Research  in  Cl 

The  myth  exists  among  Japanese  that  folks  in  Texas  still  use  horses  as  the  main  transportation  and  chases 
and  feeds  livestock  all  day.  Then  there  is  a myth  among  people  in  the  U.S.  that  Japanese  still  walk  around  with 
katana  swords  in  their  sash  and  drink  sake  with  geisha  girls.  Although  such  misunderstanding  exists,  students  have 
very  shallow  understanding  of  their  own  culture  to  help  others  understand  the  truth.  Therefore,  in  2000,  we  assigned 
students  to  research  in  Cl  to  deepen  their  knowledge  before  learning  C2  any  further.  Japanese  students  took  the 
subject  of  bushi/samurai  while  U.S.  students  researched  all  about  cowboys. 

Due  to  the  great  number  of  students  involved  in  this  project,  the  students  were  divided  into  six  groups; 
history,  jobs,  appearances,  life  style,  culture,  and  beliefs  and  spirits.  By  mid  October  2000,  students  posted  their 
research  results  in  assigned  rooms  on  FirstClass  server  in  Tokyo.  We  chose  students  to  post  their  messages  of  Cl  in 
LI. 


Read  in  L2  and  learn  C2,  then  post  replies  in  text  and  video 

Students  read  the  posted  messages  in  L2  and  learn  C2.  Then  they  are  required  to  post  a response,  including 
a question,  in  the  mixture  of  LI  and  L2  by  the  end  of  October  2000.  The  interface  of  FirstClass  is  very  similar  to  that 
of  many  popular  email  applications.  However,  the  messages  are  threaded  for  better  organization.  In  2001,  we 
added  another  method  of  communication  using  video  letters.  Two  streaming  servers,  a Quick  Time  server  at  Tokyo 
Institute  of  Polytechnics  and  a Real  server  at  Baylor,  hosted  this  project. 

Read  and  post  in  the  mixture  of  LI  and  L2  ^ 

By  middle  of  November  2000,  students  have  read  the  comments  and  questions  toward  their  research. 

These  comments  and  questions  required  further  research  in  Cl . By  this  time  the  communications  are  getting  less 
formal.  Students  are  developing  healthy  friendship  within  their  small  groups.  We  encourage  students  to  correspond 
as  much  as  they  prefer. 

Concluding  iVisit  session 

At  the  beginning  of  December  2000,  we  conducted  a farewell  iVisit  session.  Students  from  U.S.  met  at  6 
p.m.  to  accommodate  the  time  difference.  (A  cultural  lesson,  meeting  time  in  U.S.  changes  by  one  hour,  since  there 
is  no  Daylight  Saving  Time  in  Japan.)  The  excitements  filled  the  both  rooms  to  meet  their  cyber  friends  from  their 
small  groups  and  finally  put  the  name  and  face  together.  Students  answered  the  Post-Project  questionnaires 
consisting  44  questions,  to  complete  this  project. 


Five  C’s 


“Culture  is  not  the  people  but  the  communication  that  links  them  together.” 


Mary  Jane  Collier 


Our  goal  is  to  see  how  students  can  learn  C2  through  their  communication  in  more  realistic  way  through 
technology.  In  order  to  complete  this  approach,  we  must  consider  and  examine  the  Five  C’s  and  apply  this  idea  to 
enhance  learning. 

In  the  first  C,  communication , we  use  iVisit  to  encourage  and  enhance  students’  speaking  and  listening 
skills.  In  contrast,  during  FirstClass  sessions  students  applied  mainly  their  reading  and  writing  skills.  Many 
students  enhanced  their  FirstClass  presentation  with  visual  and  aural  communication  tools,  such  as  sound  and 
graphic  flies  to  deepen  the  communication. 
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As  mentioned  earlier,  this  project  is  not  only  learning  C2,  but  begins  from  examining  Cl.  This  enables  the 
complete  reflection  and  learning  of  cultures.  Examining  and  understanding  oneself  only  enhances  understanding 
others.  FirstClass  sessions  allowed  students  to  examine  the  myth  and  find  the  truth.  iVisit  sessions  filled  students’ 
appetite  with  better  understanding  of  contemporary  popular  culture. 

As  students  learn  another  culture  in  a language  class,  there  is  an  opportunity  to  make  connections  to  other 
academic  fields.  The  following  chart  indicates  some  of  what  students  had  shared  and  learned  throughout  the 
project. 


iVisit 

FirstClass 

Politics 

Discussion  of  contemporary  agenda 

Political  system  from  the  appropriate  era. 

History 

N/A 

Cowboys  and  Samurai 

Music 

Discussion  on  contemporary  music 

Music  that  cowboys  and  samurai  listened. 

Art 

Contemporary  fashion  fads. 

Paintings  depicting  the  appropriate  era 
were  shared. 

Literature 

N/A 

Appropriate  literatures  were  shared. 

Sociology 

Experiencing  such  traits  as  tardiness  and 
shyness  of  students  make  a reflection  to  its 
culture. 

Fashion,  jobs  and  other  related  issues  were 
discussed. 

From  entire  project  students  received  opportunities  to  make  comparisons.  One  way  is  to  compare  and 
contrast  now  and  then.  It  is  done  in  different  subject  fields,  which  we  discussed  previously  as  a part  of  connection. 
Comparison  of  Cl  and  C2  within  the  particular  time  occurs  concurrently.  These  comparisons  take  place  during  both 
iVisit  and  FirstClass  sessions. 


Then 

Cl  <- 

->  C2 

A 

A 

V 

V 

Now 

Cl  <- 

->  C2 

To  provide  Communities,  we  used  iVisit  for  its  visual,  oral,  and  aural  communications.  This  community 
was  a large  community  where  students  shared  one  camera,  one  microphone  and  one  screen.  FirstClass  clients  were 
more  intimate.  It  used  small  groups  for  the  reading  and  writing  communit  ies.  Individual  involvement  became  more 
crucial  in  this  exercise.  Yet  we  realize  that  a true  global  community  evolves  during  the  project.  Both  software  can 
be  used  from  anywhere  in  the  world. 

These  five  c’s  take  significant  part  of  this  project.  Each  c depends  on  each  other  to  make  this  collaboration 
effective.  Only  when  these  five  c’s  become  interactive,  this  project  claims  success. 

Data 

Since  the  first  semester,  all  of  the  targeted  Baylor  students  have  already  been  assisted  on  their  language 
acquisition  by  computers  from  their  “E-mail  Pen  pal”  programs.  However,  their  change  in  opinion  and  attitude 
toward  computer-assisted-leaming  has  not  yet  been  measured  until  now.  Students  have  and  will  complete  two 
questionnaires  for  this  collaboration.  One  must  be  taken  before  the  project  and  the  other  immediately  after 
completion  of  the  project.^ 

Although  current  result  limits  in  accuracy  to  determine  its  outcome  from  the  data  of  only  one  project, 
preliminary  data  result  from  Baylor  students  shows  the  effectiveness  of  computer-assisted  acquisition.  In  asking, 
“Do  you  think  computer  is  a useful  learning  tool  in  academic  setting?”  clearly  there  is  an  increase  in  recognizing  the 
usefulness  and  effectiveness  in  application  of  technology  in  education  during  one  semester. 


^ Questionnaires  are  available  for  the  viewing  purpose  only  at  http://www.baylor.edu/~Japanese/interculture.html. 
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Another  interesting  question  asked,  “Are  you  interested  in  foreign  cultures?”  The  interest  in  foreign 
cultures  increased  after  experiencing  this  collaborative  project.  Therefore,  as  predicted,  the  technologically  enhance 
culture -learning  collaboration  place  learners  in  positive  and  predictable  communicative  world  in  which  learners 
must  experience  to  acquire  another  culture. 


70% 
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20% 
10% 
0% 


nPre-Project 

BPost-Project 


On  the  Japanese  side,  the  questionnaire  results  show  no  statistically  significant  change  of  attitudes  among 
students.  However,  quite  a few  post  project  comments  from  these  Japanese  students  tell  that  they  became  able  to 
perceive  their  virtual  partners  as  real  persons  and  build  a kind  of  relationship  that  promotes  learning.  The 
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importance  of  personal  relationship  for  network -based  learning  programs  has  been  suggested  by  several  Japanese 
researchers,  and  this  study  meets  such  a claim.  ^ 


Problems  to  be  solved 

As  with  any  project,  we  can  expect  some  problems.  The  problem,  which  we  cannot  resolve,  is  the  time 
difference  during  i Visit  sessions.  14  or  1 5 hours  in  time  difference,  depends  on  the  Daylight  Savings  Time, 
becomes  a menace.  Another  problem  in  iVisit  session  is  the  poor  connection  speed.  The  poor  connection  speed  will 
result  in  jerky  and  dropped  video  and  audio  feed.  Another  result  of  the  poor  connection  speed  is  the  extreme 
reverberation  of  audio  feed  which  disables  the  recognition  of  the  language. 

Limitation  in  the  number  of  video  windows  forces  us  to  use  only  one  camera.  It  would  be  ideal  to  have  all 
30+  students  to  appear  in  individual  windows.  But  software  issue  and  connection  speed  limits  us  to  perform  at  that 
level  yet. 

In  FirstClass  neither  the  time  nor  connection  speed  raise  serious  concern.  The  limitation  in  number  of 
students  who  can  login  simultaneously  brings  a serious  concern.  This  problem  disables  to  login  to  FirstClass  server 
as  an  individual  during  a class  and  has  discussion  on  particular  postings. 

Conclusion 

After  all,  “when  we  are  merely  being  ourselves,  acting  according  to  our  deepest  instincts,  human  being 
reveal  fundamental  differences  in  what  we  all  tend  to  think  of  as  normal  behavior.”  (Storti)  Communication  errors 
easily  and  rapidly  occur  in  our  networked  life.  Strangers  from  all  over  the  world  gather  in  the  online  community. 
Without  proper  understanding  of  cultures,  the  behavior  progress  inappropriately. 

Five  c’s  take  an  important  role  in  teaching  culture  in  foreign  language  courses.  In  the  acquisition  of 
language  and  culture,  the  technology  assists  teachers  to  offer  innovative  and  effective  method  of  educating. 
Technology  aids  communication,  assists  to  learn  cw//wre  ofL2,  make  connections  speakers  of  the  target 
language,  provides  between  LI  -L2  and  C1-C2,  and  offers  to  participate  in  communities  using 

languages  other  than  L 1 . 


^ The  importance  of  relationship  building  for  network-based  corroborative  learning  is  discussed  in  the  following 
books  and  the  article. 

Saeki,  Y.,  Fujita,  H.,  and  Sato,  M.  eds.  (1996),  Manabiau  Kyoudoutai  (The  Community  for  Learning 
Corroboratory),  Tokyo:  University  of  Tokyo  Press. 

Mima,  N.  (1997).  Fushigikan  Network  no  Kodomotachi  (Children  of  Fushigikan  Network),  Tokyo:  Just  System 
Okubo,  M (2000),  “Learn  and  Express  through  the  Internet”  Proceedings  of  the  Sixth  Joint  Conference  on 
Educational  Technology,  pp.  873-874. : JCET  2000,  Naruto,  Tokushima,  JAPAN. 
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Abstract* 

As  technology  improves,  everyday  new  devices  are  developed  in  more  sophisticated  ways.  This  is  also  true 
for  classroom  and  teaching  tools.  If  technology  is  to  be  integrated  into  schools  successfully,  then  teachers 
must  understand  that  instructional  technology  is  not  just  hardware  or  software,  but  rather  a process  and/or 
approach  to  teaching  and  learning.  There  are  many  benefits  and  barriers  of  using  technology  in  teaching. 
First,  with  the  use  of  technology  the  lesson  can  be  adapted  to  accommodate  special  needs  students. 
Technology  helps  the  teacher  individualize  the  lesson  to  meet  all  students'  needs.  Second,  technology  also 
helps  make  learning  meaningful.  Through  the  use  of  technology  learning  can  be  presented  in  a novel  and 
creative  way.  When  students  take  personal  interest  in  a topic  they  begin  to  take  ownership  of  their  newly 
acquired  knowledge.  Even  though  the  Internet  has  many  benefits  it  also  has  weaknesses  with  the  use  of 
technology.  This  study  is  a review  of  different  genres  of  online  educational  tools  and  aimed  at  elucidating 
the  purposes  benefits,  and  barriers  of  using  different  Internet  genres. 

(*)  An  initial  draft  of  this  paper  was  submitted  as  a part  of  the  requirements  of  the  course,  EME  5207: 
Designing  Technology  Rich  Curricula,  offered  by  Asst.  Prof  Colleen  Swain  in  the  College  of  Education  at 
The  University  of  Florida  in  Spring  200J. 

Envisioning  Use  of  the  Internet  in  the  Classroom 

The  Internet  has  been  introduced  in  schools  for  1 5 years;  however,  it  has  only  been  a few  years  that 
educators  see  the  power  of  the  Internet  in  student  learning  and  achievement.  The  Internet  may  play  various  make 
roles  in  education.  It  is  the  responsibility  of  the  teacher  to  decide  how  to  best  use  of  the  Internet  Genres  to  support 
student  learning.  When  teachers  integrate  the  genres  into  their  curriculum,  they  can  enhance  their  lesson.  The 
Internet  genres  give  students  access  to  real  life  situations,  not  text  book  cases.  Students  sense  the  difference  and  are 
more  involved,  their  learning  is  more  in-depth,  because  they  are  using  their  critical  thinking  skills,  making 
judgments.  This  article  mainly  points  benefits  and  impediments  of  the  Internet  Genres  for  educators.  The  Internet 
was  originally  designed  to  allocate  information  in  an  interactive  way  and  to  allow  people  to  communicate  with  each 
other  and  with  machines  (Bemars-Lee,  1996).  It  hoped  that  the  following  framework  sheds  light  on  education  in 
classroom. 

Classifying  and  defining  different  aspects  of  the  Internet  Genres  allows  teachers  to  use  technology  to 
enhance  students’  learning,  and  to  improve  teaching.  However,  without  an  educator  who  facilitates  learning,  the 
richness  of  Internet  in  the  classroom  setting  is  not  effective.  March  in  his  article  “Working  the  Web  for  Education” 
compares  traditional  and  web-based  education  (March,  2001). 

There  are  many  benefits  and  barriers  of  integrating  technology  in  class.  First,  with  the  use  of  technology  the 
lesson  can  be  adapted  to  accommodate  special  needs  students.  Technology  helps  the  teacher  individualize  the  lesson 
to  meet  a//  students’  needs.  Technology  also  helps  to  make  learning  meaningful.  Through  the  use  of  technology 
learning  can  be  presented  in  a novel  and  creative  way.  When  students  take  personal  interest  in  a topic  they  begin  to 
take  ownership  of  their  newly  acquired  knowledge. 

Even  though  the  Internet  has  many  benefits,  it  also  has  weaknesses.  It  can  be  totally  useless  or  less  useful 
than  traditional  teaching  when  weaknesses  are  not  understood.  One  disadvantage  of  using  the  web  quest  format,  for 
example,  is  that  students  could  get  too  wrapped  up  in  serendipitous  learning.  While  they  could  go  to  a site  carefully 
chosen  by  the  teacher,  they  can  link  to  other  material  on  that  site  or  other  sites  that  they  find  personally  interesting. 

In  this  article  we  are  going  to  discuss  the  purposes,  benefits,  and  barriers  of  using  different  Internet  genres. 

WebQuest: 

WebQuests  include  lesson  plans  developed  and  posted  by  teachers.  The  plans  incorporate  the  use  of  the 
World  Wide  Web  sites  and  allow  for  student  interaction  with  the  sites  in  order  for  them  to  solve  problem  and  make 
decisions  (March,  1998).  [http://www.ozline.com/webquests/intro].  Students  are  given  a scenario  and  specific  tasks 
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to  complete  to  solve  a problem  or  finish  a project.  Webquests  have  most  of  the  critical  attributes  of  learning  as 
define  by  Jonnassen  (1997) 

Benefits: 


• Different  perspectives 

• Meaningful  knowledge 

• Promote  higher-order  thinking 

• Afford  students  the  opportunity  to  practice  collaborative  thinking  and  cooperative  learning  skills. 

• Comparing  and  contrasting 

• Motivational 

• Student  have  role  to  play 

• Interdisciplinary  & enabling  cognitive  flexibility 

• Involves  interactive  learning 

• Gain  more  in-depth  learning  skills 

• Teacher  (easy  format)  and  student  (safe  links)  friendly 


Barriers: 

• Time  consuming  (too  much  to  cover) 

• Depends  on  the  maintenance  & speed  of  the  network 

• Students’  adaptation  problems 

• It  might  be  hard  to  understand 

Online  Lesson 

Online  lesson  is  a continuous  portion  of  teaching  given  to  a certain  number  of  learners  over  the  Internet. 
Benefits: 


• Provides  access  to  knowledge  that  learners  need  whenever  they  need  them. 

• Makes  learners  more  independent  and  self-responsible. 

• Easy  to  access 

• Video  and  audio  formats  can  add  into  lesson 

• Accommodates  multi-model  learners 

• Enables  individual  pace 

• Easy  to  update  information 

• Economical 

• Easy  to  access 

• Easy  to  print  presented  information. 

• Students  can  construct  their  knowledge  without  instructor 


Barriers 


• Affected  by  slow  and  crashing  networks 

• Need  backup  plan  if  not  accessible 

• Decreases  student  teacher  communication  and  interaction 

• Need  content  and  format  preparation 

• Motivational  factors  are  not  enough 

• It  is  not  easy  to  control  learners 

• Students  can  feel  lost 

Tutorials 
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• Tutorial  is  a period  of  online  instruction  given  by  teacher  to  the  learners. 

Benefits 

• Flexible.  It  can  take  place  24  hours  a day  at  learner’s  convenience. 

• Learner  can  access  from  any  location  with  the  Internet  access. 

• Economical 

• Easy  to  follow  program 

• Very  easy  to  present 

• Easy  to  respond  questions 

• Experiment 

Barriers 

• Rigid-fixed  for  one  type  of  learner 

• Teacher  can  not  adapt  individual 

• So  much  scripted-text  based 

Simulation 

The  Oxford  English  Dictionary  (online  2001)  gives  the  following  definition  for  simulation:  ‘The  technique 
of  imitating  the  behavior  of  some  situation  or  process  (whether  economic,  education,  military,  mechanical,  etc.)  by 
means  of  a suitably  analogous  situation  or  apparatus,  especially  for  the  purpose  of  study  or  personnel  training”. 
Simulation  is  intended  to  be  discovery  learning  tool  that  allows  the  learner’s  freedom  of  exploration  by  giving 
learning  environments  without  specific  directions  or  explanations.  Simulation  occupies  very  important  places  in 
education  (Lee,  1999).  Simulations  provide  a unique  environment  for  exploring  new  concepts,  for  gaining  an 
understanding  of  the  interplay  between  related  complex  phenomena,  and  for  the  construction  of  simplified  working 
models  of  topics  under  study.  Simulations  are  also  one  area  in  which  computing  technology  is  uniquely  suited  as  the 
delivery  mechanism  for  an  educational  experience. 

Benefits 


• Helps  students  gain  understanding 

• Allows  student  to  learn  by  doing 

• No  risk,  no  danger 

• Rich  economical  experience,  at  anywhere,  for  doing  anything 

• Allows  students  to  visualize  abstract  concepts 

• Simplification  of  reference  system 

• Experience  situation 

• No  consequences,  freedom  in  exploration 

• Enables  higher-order  thinking 

• Creativity  increases  while  possible  negative  reactions  decrease 

• Control  more  elements  and  better 

• Mentally  active  gains 

• Multidimensional  mental  representations 

• Address  variety  of  learning  styles 

• Novel  experiences 

• Provides  a guiding  context  within  which  students  can  integrate  what  they  learn. 


Barriers 

• Can  some  times  fake 

• Not  good  for  multiple  uses -contextual 

• Can  expose  violence 

• Student  imitate  action  in  simulation  in  the  real  life-misgeneralization 
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Repetition  can  be  boring 

Within  presentation  mode  pure  simulation  is  of  low  level  interaction 


Games 


The  Oxford  English  Dictionary  (online  2001)  gives  a definition  of  game  as  below.  Game  is  a diversion  of 
nature  of  a contest,  played  according  to  rules,  and  displayed  in  the  result  of  superiority  either  in  skill,  strength,  or 
good  fortune  of  the  winner  or  winners. 

Benefits 

• Has  scores  reinforcing 

• Motivational 

• Has  fun 

• Engaging 

• No  danger 

• No  consequences 

• Enables  higher-order  thinking 

• Offers  big  picture  view  leading  to  analysis  and  synthesis 

• Communicate  players  instead  of  dialogue 

• Explore  problems  and  different  perspective 


Barriers 

• More  fun  than  learning 

• Time  consuming 

• Possibility  of  exposure  to  violence 

• Less  control  over  students 

• Repetition  may  be  boring 

Telecollaborative:  Interpersonal  Exchange 

Telecollaborative/Interpersonal  Exchanges  are  the  structured  activities  in  which  students  use  the  Internet 
tools  such  as  e-mail,  chat,  and  the  world  wide  web  to  access,  process,  and  share  data  and  to  commu  nicate,  and 
cooperate.  Interpersonal  Exchanges  include:  key  pals,  global  classrooms,  electronic  appearances,  telementoring, 
question-and-answer  activities,  and  impersonations  (Harris,  2001) 

Benefits 

• Can  talk  to  guest  speakers 

• Provides  multiple  perspectives 

• Share  ideas 

• Connect  to  people  outside  of  classroom  around  the  world 

• Bring  expert  to  the  classroom 

• Generally  free 

Barriers 

• Requires  background  planning  time,  preparation  takes  time 

• Needs  equipment  to  practice 

• Depend  on  network 

Telecollaborative:  Information  Collection  and  Problem  Solving 

Information  Collection  and  Analysis  are  the  activities  that  allow  students  to  be  involved  in  collecting, 
compiling,  and  comparing  different  types  of  information.  Information  Collection  and  Analysis  activity  structures 
include:  information  exchanges,  database  creation,  electronic  publishing,  telefieldtrips,  and  pooled  data  analysis 
(Harris,  2001) 
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Benefits 


• Scientists  and  students  can  come  together 

• Collect  data 

• Share  different  data 

• Gain  access  to  information  from  experts 

• Apply  knowledge  in  real  situation 

• Promote  higher-order  thinking 

• Analyze  different  views,  evaluating  information 

• Includes  learning  activities 


Barriers 

• Information  is  not  always  evaluated  for  accuracy,  relevance,  and  currency 

• Information  may  be  too  difficult  or  easy  for  students  grade  level 

• Difficult  to  control 

• Need  technological  tools 

• Limited  time,  time  consuming 

• May  not  be  appropriate  student  level  and  decide  what  kind  of  genres  will  be  fit  best  to  content  that 
is  going  to  teach.  The  new  “representation  infrastructures 

• 

Conclusion 

New  representative  distributed  learning  tools  have  both  advantages  and  disadvantages.  The  incoming 
information  is  constantly  growing.  However,  being  able  to  access  data  and  information  do  not  automatically  increase 
students’  knowledge  (Dede,  1996).  Accessing  to  the  Internet  Genres  does  not  improve  student  learning  as  well. 
Teachers  must  investigate  both  advantages  and  disadvantages  of  the  Internet  genres”  brings  numerous  opportunities 
for  both  teachers  and  learners.  However,  learners  also  need  enthusiastic,  well-educated,  and  caring  teachers,  better 
facilities  and  equipment.  Without  removing  barriers  (such  as  the  learner’s  comfort  level  with  technology,  technical 
shortage,  level  of  interaction,  the  level  of  learner’s  psychological  readiness,  cultural/individual  characteristics,  and 
environmental  factors)  in  front  of  the  learner,  we  cannot  achieve  our  goal,  which  is  transformation  of  required 
knowledge. 
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Abstract* 

Apparently,  the  Internet  has  become  an  indispensable  medium  of  the  century.  Millions  of  people  are 
using  the  Internet  for  exchanging  information,  surfing  on  virtually  any  topic,  communicating  with  the 
world,  participating  in  discussion  groups,  shopping,  traveling,  and  doing  many  other  online  activities.  The 
World  Wide  Web  is  constantly  growing.  The  Internet  offers  comprehensive  information,  instant 
transactions,  communications,  and  entertainment.  The  Internet  can  be  used  to  access  reference  information 
such  as  news,  weather,  sports,  stock  quotes,  movie  reviews,  encyclopedias,  airline  fares,  and  conducted 
transactions  such  as  trading  stocks,  making  travel  reservations,  and  banking  and  shopping  online.  As  an 
educational  tool  users  have  the  opportunity  to  learn  virtually  anything.  Although  the  World  Wide  Web  can 
be  a world  of  fascinating  and  colorful  information  for  students,  there  are  many  dangers  that  children 
should  to  avoid,  such  as  sexual,  hateful,  or  violent  materials.  This  paper  is  a literature  review  on  the 
Internet  safety  strategies.  The  purpose  of  this  paper  is  to  provide  parents  and  teachers  some  common 
Internet  safety  strategies  so  that  they  can  protect  children  from  the  harms  of  the  Internet,  based  on  the 
literature. 

(*)  An  initial  draft  of  this  paper  was  submitted  as  a part  of  the  requirements  of  the  course,  EME  5207: 
Designing  Technology  Rich  Curricula,  offered  by  Asst.  Prof  Colleen  Swain  in  the  College  of  Education  at 
The  University  of  Florida  in  Spring  2001. 

What  is  the  Internet? 

The  Internet  is  a network  that  connects  some  hundred  millions  of  users  from  all  over  the  World.  It  is  a place 
where  not  only  adults  but  also  children  have  access  to  and  can  interact  with  a wide  range  of  people.  Although  the 
Internet  has  been  in  existence  for  15  years,  the  number  of  the  Internet  users  has  reached  127  million  and  has 
constantly  been  growing  as  seen  in  figure  1 (Shelly,  Cahsman,  Gunter,  1999),  The  Internet  is  commonly  being  used 
for  exchanging  information,  making  search  in  various  topics,  communicating  with  the  world,  and  participating  in 
discussion  groups,  shopping,  entertaining,  traveling  and  doing  many  other  online  activities.  Experience  of  children 
with  the  Internet  can  be  fascinating,  colorful,  fruitful,  and  enjoyable.  However,  there  are  also  sites  on  the  Internet 
that  are  inappropriate  and  uncomfortable  for  a young  audience.  Thus,  great  care  should  be  taken  while  providing 
children  access  with  the  Internet.  School  is  the  most  popular  access  point,  with  more  than  80%  of  youths  between 
ages  1 0 and  1 7 saying  they  surf  the  Net  at  school,  according  to  a joint  study  by  the  Kaiser  Family  Foundation  and 
National  Public  Radio  (figure  2).  Almost  63%  of  U.S.  public -school  classrooms  had  Internet  access  by  the  end  of  the 
year  1999,  up  from  just  14  % in  1996,  according  to  the  National  Center  for  Education  Statistics.  Once  they  are 
online,  children  are  may  be  hooked  for  hours.  Among  those  who  use  the  Internet,  70%  of  the  students  say  that  they 
use  the  Web  at  least  once  a week  at  home  or  school  and  35%  report  using  the  Net  almost  every  day.  On  the  average, 
they  spent  almost  7 hours  per  month  on  the  Internet  - three  hours  less  than  adults  (Lake,  D.2001). 


Figure  L Internet  User  Growth  Worldwide 


Figure  2.  Percentage  of  Youths  Between  Ages  10-1 7 That  Access  the  Internet  at  School 

Using  the  Internet:  Why  It  is  Worth  the  Risk 

There  are  numerous  benefits  of  using  the  Internet  in  the  classroom,  which  would  otherwise  have  been 
impossible  a decade  ago.  First,  the  Internet  can  expand  students*  access  to  resources  dramatically  by  making 
resources  from  all  over  the  world  for  them.  Second,  students  can  communicate  with  other  students  around  the  world 
via  e-mail,  chat,  and  instant  messaging.  Communicating  each  other,  sharing  information,  exchanging  ideas  and 
cultures  may  improve  collaborative  learning  (Newby,  Stepic,  Lehman,  Russell,  1999).  Third,  students  can  access 
graphics,  text,  data,  articles,  papers,  electronic  books,  and  video  and  audio  clips  that  enhance  their  reading, 
analyzing,  evaluating,  and  higher-order  thinking  skills.  Also,  as  an  educational  tool,  the  Internet  allows  students  to 
navigate  virtually  museums,  libraries,  and  places.  Finally,  students  can  publish  their  knowledge  and  work  on  the 
Internet,  which  is  easier  and  cheaper  than  other  traditional  ways  of  sharing  knowledge  and  cultural  values. 

How  Can  the  Internet  Be  Used  in  Classroom  Settings? 

There  are  a big  number  of  ways  of  using  the  Internet  as  a learning  and  activity  tool  with  which  students  and 
teachers  may  engage  for  learning  and  sharing  knowledge.  People  who  study  in  the  areas  of  education  and  teaching; 
the  Internet  is  an  ideal  resource,  because  text  and  graphic  based  communication  systems  are  expanded  and  evolved 
into  powerful  multimedia  communications  today  (Shelly,  Cahsman,  Gunter,  1999).  The  Internet  and  the  World 
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Wide  Web  are  by  providing  a variety  of  learning  tools  changing  the  way  teachers  instruct  and  the  way  students  learn 
basic  skills  and  subjects. 

There  are  several  ways  to  use  the  Internet  in  the  classroom.  First,  the  Internet  can  be  used  as  a research  tool 
for  projects.  Students  can  easily  access  a wide  range  of  literature  on  any  topic.  Second,  it  can  be  used  for  reinforcing 
topics  already  covered  in  class,  for  example,  for  expanding  the  multicultural  aspect  of  the  classroom  and  as  a 
supplementary  tool  for  curriculum  or  online  reading  in  classroom.  Third,  the  teacher  may  keep  in  contact  with 
students  and  other  teachers  around  the  world.  The  Internet  can  also  be  used  for  practical  applications  of  theories. 

What  Risks  are  Involved  with  Using  the  Internet  in  the  Classroom? 

The  Internet  is  a mirror  of  the  society  it  is  being  used  by.  Currently,  web  publishing  is  not  controlled  by  any 
particular  organization  and  there  are  no  regulations  in  place  to  verify  the  information  posted.  Although  the  Internet 
is  a comprehensive  information  resource  and  a useful  teaching  and  learning  tool  for  education,  there  are  some  areas 
of  the  Internet  that  are  not  appropriate  for  a young  audience.  There  are  sites  that  depict  nonconsensual  act  of 
violence,  explicit  sexual  sites  or  promote  racism,  anti-Semitism,  and  hatred  (Magid,  2000).  Another  risk  is  that, 
while  online  a student  may  provide  information  or  arrange  an  encounter  that  could  risk  his/her  safety  or  safety  of 
other  persons  (Magid,  1998).  Students  have  been  known  occasionally  to  provide  personal  identifying  information  to 
strangers -either  knowingly  or  not-such  as  password,  home,  or  school  address,  phone  number  that  could  endanger 
their  mental  and  physical  condition.  Also  some  information,  graphics,  and  video  and  audio  clips  acquired  from  the 
Internet  may  not  always  be  accurate  and  appropriate.  Additionally,  the  Internet  can  be  used  as  a device  with  which 
to  play  or  spend  time  off  task.  It  is  als  o viable  to  break  numerous  laws  including  “computer  hacking”  and  copyright 
violation.  Educators  should  preview  materials  and  provide  supervision  for  ensuring  children’s  safe  use  of  the 
Internet.  These  strategies  and  approaches  can  be  different  in  different  circumstances  depending  on  the  audience’s 
age,  knowledge,  and  cultural  background.  Indeed,  violence,  pornography,  and  predators  on  the  Web  are  more 
serious  threats  to  children’s  exposure  than  other  threats. 


Unique  Internet  Visitors  by  Age  Group 


Age  Group 


Figure  3.  Unique  Internet  Visitors  by  Age  Group 


A recent  survey  certainly  indicates  that  kids  are  fueling  the  Internet  explosion.  Polling  firm  Grunwald 
Associates  found  in  their  Just-released  study  that  25  million  2 to  17-year-olds  are  online  in  the  US  right  now,  up  from 
8 million  in  1997. 
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Figure  4.  Education  Breakdown  for  Online  Youths 

About  one  in  five  10  to  17-year-olds  surveyed  said  they  have  received  sexual  solicitation  or  approach  via 
the  Internet  in  the  past  year,  according  to  a study  released  this  week  by  the  National  Center  for  Missing  and 
Exploited  Children  (NCMEC).  Another  key  finding  was  that  one  in  four  of  the  children  surveyed  said  they've  had 
unwanted  exposure  to  pictures  of  naked  people  or  people  having  sex  in  the  past  year,  The  study  “Online 
Victimization”  available  online  at  www.netfamilivnews.org  is  an  important  contribution  to  the  public  discourse 
about  the  Internet's  impact  on  kids.  It  calls  for  "private  and  public  initiatives  to  rais  e awareness  and  provide 
solutions,"  including  strategies  that  reduce  the  amount  of  offensive  behavior  toward  kids,  help  shield  parents  and 
teachers  from  it,  and  give  them  the  tools  they  need  to  cope  with  both  the  behavior  and  its  consequences.  According 
to  the  study  large  numbers  of  youths  encounter  offensive  experiences  on  the  Internet.  Twenty  percent  were  sexually 
solicited,  6%were  harassed.  The  offenses  and  offenders  are  diverse,  not  just  men  trolling  for  sex.  Much  of  the 
offensive  behavior  comes  from  other  youth,  and  some  from  women  as  well.  Teenagers  are  the  primary  targets, 
creating  a different  sort  of  challenge  than  would  be  the  case  if  younger  children,  over  whom  parents  have  more 
control,  were  the  primary  targets.  Although  most  solicitations  fail,  the  sheer  numbers  are  alarming.  Several  million 
young  people,  ages  10  to  17,  are  sexually  propositioned  on  the  Internet  every  year  (Dede,  2001). 


Figure  5.  1 6-22  Year-olds  * Reasons  to  use  the  World  Wide  Web 

Safety  Issues  in  The  Oassroom:  What  teachers  should  do 

This  paper  focuses  mainly  on  the  safety  strategies  for  6-12  grade  level  at  schools,  because  they  are  thought 
to  be  the  most  affected  persons  by  harms  of  the  Internet.  At  these  grades  students  are  just  beginning  their  move 
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toward  independence  and  can  no  longer  be  told  not  to  do  something  without  being  given  a well-argued  reason  for 
the  rule.  At  these  ages,  students  need  guidelines  and  assistance  with  their  decision-making.  They  also  need  clear 
guidance  for  appropriate  uses  of  the  Internet,  while  given  information  to  understand  the  reasons  behind  the  enforced 
directions.  Careful  preparation  is  the  key  to  effective  and  safe  use  of  the  Internet  in  the  classroom. 

Preparing  Students 

In  order  for  them  to  be  completely  prepared  fo  r navigating  the  Internet  safely  in  the  classroom,  students 
need  to  learn  what  defines  ethical  behavior  on  the  Internet.  Essentially  ethical  behavior  on  the  Internet  is  the  same  as 
ethical  behavior  in  every  other  aspect  of  life.  Students  need  to  know  that  they  should  never  menace  or  harass  people 
they  meet  online  or  offline.  They  also  need  to  know  that  they  should  not  use  language  that  is  offensive  to  others.  A 
good  general  rule  that  the  students  can  fallow  is  “Don’t  say  anything  to  people  you  meet  online  that  you  would  not 
say  to  your  parents,  teachers,  friend,  or  religious  leader”. 

Teacher  preparation 

Students  need  the  teacher’s  support  and  involvement  in  learning  how  to  make  positive  choices.  Teaching 
involves  helping  students  to  cope  with  problems  encountered  online  and  providing  them  with  the  set  of  skills,  they 
will  need  to  make  the  right  choices  in  the  absence  of  their  teacher  or  parent. 

There  are  several  ways  teachers  can  prepare  themselves  and  students  while  surfing  online.  First,  before 
starting  to  use  the  Internet,  teachers  may  give  a short  lecture  about  how  to  use  the  Internet  browser  software. 

Second,  to  teach  ethical  behavior  on  the  Internet  the  teacher  may  open  a discussion  about  the  issue.  Opening  a 
discussion,  they  enable  students  to  think  critically  about  issues  as  the  teacher  guides  them  toward  a better 
understanding.  Teachers  can  also  have  the  students  role-play  different  situations  on  the  Internet  or  can  use  examples 
to  help  students  to  come  to  a better  understanding  of  the  issues.  Students  should  be  informed  of  the  Internet  use 
policy  including  the  rules  of  computer  use  and  Internet  access.  The  teacher  must  make  sure  that  the  students  have 
read  and  signed  the  Internet  Use  Policy.  Making  clear  what  the  penalties  are  for  misusing  of  the  Internet  may 
reinforce  students  to  use  the  Internet  in  a proper  way.  Teachers  must  also  forward  the  policy  to  the  parents.  It  is 
very  important  to  keep  parents  informed  about  the  activities  that  the  students  are  involved  in  the  classroom.  Sending 
a newsletter  and  the  Internet  and  computer  use  policy,  and  requiring  students  to  return  signed  parental  forms  prior  to 
each  semester  are  some  suggestions  for  teachers.  They  can  also  provide  a computer  workshop  for  parents  prior  to 
each  semester  in  order  to  allow  parents  to  review  both  the  hardware  and  software  their  children  will  be  using. 
Teachers  make  their  phone  number  and  e-mail  address  available  to  all  parents  to  give  the  parents  the  ability  to 
contact  the  teacher  with  any  questions  they  may  have. 

Another  preparation  strategy  for  teachers  is  to  provide  a list  of  web  site  addresses  that  will  be  used  in  the 
class  prior  to  classroom  use.  Students  must  be  monitored  while  they  are  surfing  on  the  Internet.  Also  filtering  or 
blocking  software  can  be  used  in  the  classroom  computers  to  protect  students.  However,  filtering  software  is 
practically  not  often  capable  of  filtering  out  sites  that  contain  unwanted  materials  for  students.  Also  sometimes  a 
valid  content  may  be  screened  out  inappropriately.  Thus  they  are  not  to  be  relied  on  completely  instead  they  should 
be  supplemented  with  other  strategies.  It  is  a good  idea  to  make  sure  computer  screens  can  easily  be  seen  by  the 
teacher  in  the  classroom  (Magid,  2000).  During  an  Internet  lesson,  teachers  may  roam  the  classroom  observing  what 
students  are  viewing.  Most  browsers  keep  in  their  history  a number  of  short-term  records  of  the  recently  visited 
sites.  There  are  also  monitoring  systems  that  provide  a record  of  the  web  sites  a person  has  visited  on  the  Internet. 
The  monitoring  systems  and  browsers’  short-term  records  are  low  cost  solutions  for  teachers  to  monitor  their  student 
in  the  classroom  (Dede,  2001).  Lacking  the  awareness  of  dangers  the  Internet  can  pose  students  unintentionally 
wonder  places  where  their  innocence  and  safety  are  compromised. 

Safety  Rules  and  Guidelines  for  Student  Classroom  Internet  Use:  What  Students  Need  to  Know 

The  use  of  school  computers  and  computer  networks,  computer  software,  data  files,  Internet  access,  and 
intellectual  property  in  the  classroom  is  a privilege  and  is  intended  for  educational  purposes  only. 

The  following  list  contains  guidelines  by  according  to  which  students  should  use  the  Internet  during  both 
their  usual  class  hours  and  any  time  they  may  spend  on  the  school’s  computers.  This  list  is  not  meant  to  be  a final 
list  of  what  is  or  is  not  acceptable  while  using  the  Internet  in  class;  however  it  provides  some  insightful  ideas.  If 
students  are  not  sure  about  how  to  behave  in  a certain  case  they  should  ask  their  teacher.  The  set  of  rules  students 
should  obey  in  class  include  but  are  not  limited  to: 
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• Share  computer  account  usernames  or  and  passwords  except  when  authorized. 

• Create,  copy,  receive,  or  use  data,  language,  or  graphics  which  are  obscene,  abusive,  or  otherwise 
inappropriate  at  school 

• Access,  change,  or  delete  computer  programs,  data  files  or  electronic  mail  belonging  to  others 
without  permission. 

• Steal  or  destroy  school  computer  hardware  or  peripherals. 

• Steal  or  destroy  computer  software  or  data  files  owned  by  the  school  or  others. 

• Violate  or  attempt  to  violate  the  security  of  computer  network  systems. 

• Take  unauthorized  actions,  which  deny  access  to,  disrupt,  or  destroy  the  service  of  the  computer 
network  system. 

• Make  unauthorized  or  unlawful  installation  of  personal  computer  software  on  the  school 
computers  or  networks.  Including,  but  not  limited  to  games,  virus  programs,  and  applications 
software.  Student  provided  software  must  be  used  only  in  accordance  with  the  license. 

• Use  computers,  networks,  or  peripherals  to  create  a forged  instrument. 

• Use  computers,  networks,  software,  data  files,  or  intellectual  property  in  any  unauthorized  way. 

(This  list  of  classroom  rules  was  adopted  from  “School  Board  of  Alachua:  County  District  Technology 
Plan”  Available  on-line  at  http://www.sbac.edU/district/techplan.html#anchorl6496056,  p.l2.) 


Computer  Use  Policy  in  the  Classroom 


Computers  should  only  be  used  for  school  related,  educational  activities. 

Chat  rooms  and  Newsgroups  may  not  be  accessed. 

Purchases  over  the  Internet  are  not  permitted. 

Student  cannot  give  out  information  about  himself/herself  such  as  personal  phone  numbers, 
address,  passwords,  last  and  first  name,  and  other  identifying  information. 

Students  cannot  send  a picture  over  the  Internet  or  through  regular  e-mail  to  anyone  without  the 
teacher's  permission. 

Student  cannot  respond  to  any  messages  that  are  mean  or  in  any  way  make  him/her  feel 
uncomfortable. 

Student  must  tell  his/her  teacher;  if  something  seems  confusing,  scary,  or  threatening  online. 
Student  cannot  have  a face-to  face  meeting  with  someone  he/she  has  met  online. 

Students  and  teachers  will  be  required  to  log-in  and  log-out  on  the  Internet. 


(Adopted  from  “Hidden  Oak  Elementary  School  Gainesville,  FL”  computer  use  policy  form.) 


Conclusion 

Most  educators  try  to  integrate  technology  and  the  Internet  into  classes  to  enhance  students’  learning  and 
achievement.  Although  the  Internet  is  an  invaluable  resource  and  a comprehensive  research  tool,  it  includes  potential 
dangers  and  unsafe  places  as  well.  The  speedy  growing  use  of  the  Internet  in  the  classroom  by  students  has  caused 
teachers  to  be  increasingly  concerned  about  students’  safety  using  the  Internet.  The  best  way  to  assure  that  students 
are  having  fruitful  online  experiences  is  to  monitor  what  they  are  doing.  Teachers  should  spend  time  with  students 
while  they  are  online.  The  next  step  is  to  establish  some  basic  rules  for  students’  use  of  the  Internet.  While  students 
need  a certain  amount  of  privacy,  they  also  need  guidance  and  supervision  in  their  lives.  By  taking  responsibility  for 
students’  use  of  the  Internet,  teachers  can  greatly  minimize  any  potential  risks  of  being  online. 
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Looking  for  the  hype  in  hypertext:  An  essay  deconstructing  pedagogical  assumptions 
associated  with  online  learning  and  instructional  design. 

Jim  Dwight 
Virginia  Tech 

Abstract 

This  paper  aims  to  debunk  the  metaphysics  of  presence  informing  modernist  pedagogical  assumptions. 
Systematic  instructional  design,  predicated  on  teleological  and  eschatological  modern  metaphysics, 
superordinates  designer’s  goals  at  the  expense  of  learners.  Tracing  structuralist  pedagogical  theory  to 
Bobbitt  (J997)  and  Tyler  (1949),  one  can  readily  see  the  roots  of  popular  instructional  design  models,  such 
as  Smith  and  Ragan  (J993),  Mager  (1997),  and  Dick  and  Carey  (1996),  If  however,  we  look  to  pragmatism 
and  post  structuralism,  we  can  find  alternatives  to  reductive  and  straight-line  pedagogical  theories  and 
thereby  construct  emergent  and  transactional  learning  spaces  in  which  learner  input  is  valued.  Pragmatist 
and  postmodernist  pedagogies,  moreover,  place  an  emphasis  on  mediation. 


Introduction 

The  hype  is  that  online,  virtual  sites  will  free  students,  instructors,  and  administrators  from  many  of  the 
limitations  proximal  lectures  have  presented  over  the  years  from  universal,  homogenized  pedagogies  to  reduced 
infrastructure  costs.  This  paper  will  examine  certain  entrenched  modernist  learning  assumptions  and  counterpoise 
these  with  post-modernist  and  pragmatist  sensibilities  that  seek  to  take  advantage  of  the  current  digital  revolution  in 
order  to  improve  the  educational  possibilities  for  online  learning.  The  emphasis  here  is  pointedly  on  learning  not 
education  as  a social  institution.  As  Dewey  (1944)  noted  in  the  earlier  half  of  the  last  century,  institutionalized 
education  can  stifle  the  natural  urge  to  learn  when  a society’s  goals  become  superordinate  to  the  individual  learner’s, 
thereby  dampening  a learner’s  desire  to  grow  in  an  educational  context.  Typically,  instructional  design  models 
predicated  on  fixed  objectives  seriously  limit  the  likelihood  for  an  emergent  learning  transaction  to  occur.  The 
purpose  of  this  paper  is  to  deconstruct  many  of  the  underlying  pedagogical  assumptions  informing  typical  and 
popular  instructional  design  models,  particularly  systematic  ones,  and  to  offer  some  alternatives.  Using  such 
alternatives  will  hopefully  open  hypertextuality  to  more  dynamic  potentials  for  realistic  learning. 

Modernist  pedagogies 

The  torturous  route  to  Modernist  assumptions  of  knowledge  have  roots  dating  back  to  the  Platonic 
dialogues  and  Aristotle’s  philosophical  works,  through  the  Neo-Platonists  (Augustine)  and  Medieval  Scholastics 
(Thomas  of  Aquinas),  adopted  by  such  seventeenth  century  philosophers  as  Descartes  and  Newton,  and  passing 
through  Philosophers  such  as  Diderot  and  Rousseau.  In  the  modem  era,  the  two  most  commanding  figures 
influencing  curriculum  theory  and  underlying  assumptions  of  knowledge  transfer  are  Bobbitt  (1997)  and  Tyler 
(1949)  (Applebee,  1996;  Gress  & Purpel,  1979;  Flinders  & Thorton,  1997;  Walker  & Soltis,  1986). 

Bobbitt’s  Curriculum 

Franklin  Bobbitt’s  The  Curriculumn997J  initiated  modem  curriculum  theory  (Walker  & Soltis,  1986). 
Bobbitt  should  be  considered  what  Eliot  Eisner  (1994)  refers  to  as  a rational  humanist.  A rational  humanist  believes 
that  foremost,  humans  are  rational  animals  capable  of  discerning  truth  from  dedicated  and  exhaustive  empirical 
study.  While  this  seems  a Just  attitude  for  determining  intelligence  at  first  glance,  a more  dedicated  and  exhaustive 
examination  reveals  that  the  rationalist  believes  that  the  universe  is  ultimately  knowable  if  one  only  discovers 
certain  physical  tmths.  The  pseudo -scientific  emphasis  then  resorts  to  what  Dewey  (1944)  calls  the  metaphysical 
fallacy  that  knowledge  preexists  inquiry  and  is  fixed  with  a final  end.  Even  Rousseau’s  Emile  (1962)  underscores 
this  faith  in  ultimate  and  final  knowledge  that  the  enlightened  mind  can  achieve  when  living  in  accord  with  one’s 
natural  attributes  uncormpted  by  society.  Dewey  critiques  this  fallacious  assumption: 

But  the  notion  of  a spontaneous  normal  development  of  these  activities  is  pure  mythology.  The  natural,  or 
native,  powers  furnish  the  initiating  and  limiting  forces  in  all  education;  they  do  not  furnish  its  ends  or  aims 
(p.  114). 

The  problem  with  both  Rousseau’s  natural  development  pedagogy  and  Bobbitt’s  social  efficiency 
pedagogy  is  that  they  are  based  on  teleological  paradigms:  for  Rousseau,  the  best  education  takes  advantage  of  a 
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person’s  innate  abilities  as  if  abilities  are  a priori  and  not  learned  behaviors;  for  Bobbitt,  a society  should  train  future 
citizens  to  do  the  job  of  today  for  30  or  more  years  in  the  future  as  if  economic  and  social  needs  will  remain  static. 
Dewey  points  out  just  how  myopic  Bobbitt’s  social  efficiency,  structuralist  planning  model  is: 

Industry  at  the  present  time  undergoes  rapid  and  abrupt  changes  through  the  evolution  of  new  inventions. 
New  industries  spring  up,  and  old  ones  are  revolutionized.  Consequently,  an  attempt  to  train  for  too  specific 
a mode  of  efficiency  defeats  its  own  purpose.  When  the  occupation  changes  its  methods,  such  individuals 
are  left  behind  with  even  less  ability  to  readjust  themselves  than  if  they  had  a less  definite  training  (p.  1 18). 
A social  efficiency  progressive  believes  that  society  can  determine  what  is  best  for  itself  and  use  this 
knowledge  to  call  upon  certain  educational  reforms  aimed  at  improving  the  nation.  Such  assumptions  underscored 
the  drive  for  Physics  and  Mathematics  after  the  Soviets  launched  Sputnik  and  the  current  emphasis  on  business 
training  in  education  so  that  American  educational  products  can  compete  in  a global  market.  This  underscores 
education  external  to  a learner,  that  society  determines  what  is  best  for  the  pupil.  In  contrast,  social  justice 
progressives,  championed  by  Dewey,  sought  expanded  democratic  participation,  social  reform,  and  more  equitable 
wealth  distribution.  Bobbitt  favored  preparing  students  for  society,  as  expert  planners  perceived  it  actually  existed  or 
would  exist.  The  difference  lies  in  that  social  justice  progressives  favored  a fluid  planning  society  while  social 
efficiency  progressives  favored  a fixed,  planned  society. 

As  Bobbitt  (1997)  sees  it,  “the  era  of  contentment  with  large,  undefined  purposes  is  rapidly  passing.  An  age 
of  science  is  demanding  exactness  and  particularity”  (p.  10).  Then  as  now,  this  stance  suggests  rigid,  external 
objectives,  and  standards  of  learning  determined  by  science.  What  science  meant  for  educators  and  politicians  then 
as  now  is  some  version  of  positivism  with  it  presumed  “hard  facts”  along  with  theory  and  value  neutral  inquiry,  or 
what  Bobbitt  calls  investigations  “without  pre -suppositions”  (p.  13).  Bobbitt  optimistically  announced, 
“Experimental  laboratories  and  schools  are  discovering  accurate  methods  of  measuring  and  evaluating  different 
types  of  educational  processes”  (p.  10).  It  does  not  matter  that  the  positivist  image  of  science  is  theoretically  dead; 
ghoulishly,  it  lives  on  to  dominate  educational  practice. 

Bobbitt  (1997)  supposes  that  aiming  for  externally  expert  determined  goals,  students  have  the  highest 
likelihood  for  succeeding  and  so  did  the  nation.  Bobbitt  wrote  that  to  “train  thought  and  judgment  in  connection  with 
actual  life -situations”  (p.  9),  will  accomplish  his  goals.  Accordingly,  we  can  deconstruct  Bobbitt’s  basic  ideas  from 
the  following  passage: 

Human  life,  however  varied,  consists  in  the  performance  of  specific  activities.  Education  that  prepares  for 
life  is  one  that  prepares  definitely  and  adequately  for  these  specific  activities.  However  numerous  and 
diverse  they  may  be  for  any  social  class,  they  can  be  discovered.  This  requires  only  that  one  go  out  into  the 
world  of  affairs  and  discover  the  particulars  of  which  these  affairs  consist.  These  will  show  the  abilities, 
attitudes,  habits,  appreciations,  and  forms  of  knowledge  that  men  need.  These  will  be  the  objectives  of  the 
curriculum  (p.  11). 

It  is  easy  to  identify  the  false  social  Darwinism  embedded  in  the  idea  that  we  should  educate  social  classes 
for  their  probable  destiny.  The  “rationality”  of  social  efficiency  demands  social  reproduction.  Tracking  and  the 
differentiated  curricula  associated  with  it  serves  as  a social  sorting  machine  for  a society  that  avoids  critical 
democratic  deliberation.  As  Aldous  Huxley  (1965)  wrote  in  Brave  New  World  Revisited  the  social  ethic  that  holds 
humans  as  entirely  social  organisms  programmable  to  social  needs  as  part  of  a collective  hive  undermines  our 
humanity,  our  biological  and  social  uniqueness.  Such  curriculum  planning  as  Bobbitt  advocated  presumes  such 
passivity  and  interchangeability  to  the  socio-economic  machine. 

Gress  and  Purpel  (1979)  remark  that  Bobbitt’s  “model  of  curriculum  planning  . . . [has]  survived  a half 
century’s  thought  and  practice  in  one  form  or  another”  (p.  237).  Walker  and  Soltis  (1986)  write,  “The  performance- 
based  and  competency-based  teacher  education  movement  of  the  1970’s  repeated  this  mode  of  curriculum 
construction”  (p.  55).  The  same  holds  for  the  “standards”  movement  over  the  last  decade.  The  enduring  appeal  of 
Bobbitt’s  objectives  and  standards  approach  lies  in  its  putative  appeal  to  modem  notions  of  “reason,”  objectivity, 
and  measurement.  The  promise  of  permanent  progress  is  also  modem,  though  the  reductive  methodological 
assurances  of  a safe  and  secure,  if  narrow,  path  to  a perfect  and  predetermined  teleological  essence,  is  pre -modem  as 
is  the  metaphysics  that  supports  it. 

Tyler’s  ‘‘Rationale” 

The  most  influential  name  in  curriculum  theory  is  Ralph  Tyler  (1949)  (Applebee,  1996;  Flinders  & 

Thorton,  1997;  Walker  & Soltis,  1986).  Gress  and  Purpel  (1979)  note  that  the  “basic  elements  of’  Bobbitt’s  “work 
underlie  Tyler’s  classic  formulation”  (p.  237).  The  classic  work  is  Tyler’s  Basic  Principles  of  Curriculum  and 
Instruction  (1949).  The  following  excerpt  comes  from  Tyler’s  “rationale:” 
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[F]our  major  tasks  serve  as  the  focuses  of  curriculum  construction:  The  selection  and  definition  of  the 
learning  objectives;  the  selection  and  creation  of  appropriate  learning  experiences;  the  organization  of  the 
learning  experiences  to  achieve  a maximum  cumulative  effect;  and  the  evaluation  of  the  curriculum  to 
furnish  a continuing  basis  for  the  necessary  revisions  and  desirable  improvements  (p.  246). 

Tyler  focuses  on  predetermined  objectives  lying  outside  the  student’s  activity.  Presumably,  these  objectives 
are  so  valuable  they  must  serve  as  the  essential  tools  of  all  learning.  Tyler  assumes  that  concrete  and  predetermined 
objectives  will  make  education  more  efficient  and  effective  regardless  of  academic  discipline;  accordingly,  Walker 
and  Soltis  (1986)  state,  ‘Tyler  . . . proposes  that  a school’s  philosophy  be  used  as  a set  of  standards  to  ‘screen’  the 
objectives  derived  from  this  first  step  in  the  process.  This  will  ensure  that  each  objective  is  in  harmony  with  the 
school’s  general  philosophy  and  ideal  aims”  (p.  56).  The  assumption  is  that  the  philosophy  of  the  school  establishes 
the  valued  objectives  for  which  Tyler  has  a value  neutral  tool  of  means-ends  rationality  for  achieving.  This  tacitly 
assumes  the  old  positivist  fact  versus  value  dualism  as  well  as  the  means  versus  ends  one.  Most  schools,  of  course, 
will  presume  that  his  methods  like  most  others  and  most  media  are  value  neutral  relying  on  traditional  metaphysics’ 
supposition  that  the  ends,  the  content,  are  most  essential  in  education. 

One  should  also  consider  Tyler’s  (1949)  stance  on  learning  experiences.  The  guiding  idea  is  that  of 
“sequence  and  integration”  (p.  251).  Tyler  declares, 

Curriculum  makers  can  also  identify  significant  skills  that  are  sufficiently  complex  and  pervasive  to  serve 
as  organizing  elements  to  achieve  sequence  and  integration.  And,  for  objectives  involving  attitudes,  appreciations, 
interests,  and  personal  commitments,  curriculum  makers  can  identify  important  values  that  can  serve  as  organizing 
elements  (p.  251). 

This  is  the  seductive  old  idea  of  curriculum  vitae  as  a straight  line,  secure,  and  certain  method  for  being 
safely  shepherded  through  hazardous  terrain.  While  this  straight-line  approach,  with  proscribed  learning  goals  as 
predicated  by  Mager  (1997)  and  Dick  and  Carey  (1996),  makes  creating  instruction  easier,  it  does  little  to  prepare 
learners  for  the  unknown  realities  of  tomorrow. 

Finally,  there  comes  evaluation  to  which  the  code  word  today  is  accountability.  “I  employ  the  term,”  writes 
Tyler,  “to  include  the  process  of  comparing  the  ideas  and  assumptions  involved  in  curriculum  development  with  the 
realities  to  which  they  refer”  (p.  252).  Although  he  does  not  say  so,  evaluation  presupposes  a philosophical  bent 
since  evaluation  obviously  requires  that  we  reflect  on  the  values  we  espouse  in  making  our  selections  of  objectives, 
means  for  obtaining  them,  and  the  organization  of  those  means.  What  is  odd  is  that  Tyler,  again  implicitly,  seems  to 
think  he  has  a value  neutral  method  of  evaluation.  Things  are  much  the  same  today. 

Commenting  on  Tyler’s  rationale.  Walker  and  Soltis  (1986)  conclude. 

He  makes  no  commitment  to  certain  ideal  aims,  specific  objectives,  a particular  program,  or  one 
conceptualization  of  curriculum  phenomena  over  another  ....  His  commitment  is  to  a highly 
rationalized,  comprehensive  method  for  arriving  at  logical  and  justifiable  curricula  of  many 
different  kinds  (p.  58). 

Curriculum  is  method’s  child,  and  content’s  orphan;  the  methodological  form  versus  subject  matter  content 
dualism  is  untenable.  Walker  and  Soltis  also  conclude  that  the  Tyler  “rationale”  is  “the  paradigm,  the  dominant 
model  of  twentieth-century  thought  about  curriculum  design  (p.  55).  Nothing  has  changed  in  the  twenty-first  century 
largely  because  the  Tyler  rationale  has  all  the  ingredients  characteristic  of  modem  thinking,  including  a firm 
commitment  to  “rationality,”  progress,  theory  (or  philosophy)  independence  of  fact,  value  neutrality,  a profound 
commitment  to  an  external  tools  as  the  essence  of  action,  and  faith  in  “method”  for  arriving  at  the  highest  value,  the 
summumbonum  the  supposedly  value  neutral  content. 

Pragmatist  and  Postmodernist  Pedagogical  Sensibilities 

Dewey’s  Democratic  Pedagogy 

In  opposition  to  these  rational  humanist  and  social  progressive  philosophical  blinders,  John  Dewey  remarks 
in  Democracy  and  Education  (1944).  education  is  growth.  Living  beings  must  continue  to  learn  in  order  to  sustain 
life:  “life  is  a self-renewing  process”  (p.  9).  Education  occurs  naturally  through  transaction  with  others  and  within 
environments.  Human  society  seeks  to  control,  guide,  and  discipline  this  process  in  order  to  sustain  its  viability:  “In 
directing  the  activities  of  the  young,  society  determines  its  own  future  in  determining  that  of  the  young”  (p.  41 ). 
Hence,  society’s  desire  to  renew  itself,  to  varying  degrees,  can  be  seen  as  a progression  from  an  individual’s  desire 
to  sustain  him  or  herself  Dewey  problematized  his  earlier  distinction  between  education  and  schooling  in  his 
reconsideration  of  Democracy  and  Education  (1944).  Experience  and  Education  (1997).  Growth  as  education  occurs 
naturally  as  a state  of  disequilibrium  in  which  an  individual  attempts  to  reestablish  equilibrium  through  inquiry. 
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Schools,  as  institutionalized  loci  for  disciplined  learning,  seek  to  guide  this  process  so  that  society  can  continually  be 
self-sustaining.  The  problem  lurking  within  this  neat  summation  resides  in  disharmony.  When  either  extreme, 
subject-oriented  education  versus  object-oriented  education,  takes  precedence  over  the  other,  growth  is  hindered. 

The  pendulum  has  swung  back  and  forth  between  the  individual’s  desires  and  the  society’s  desire  since  people  have 
debated  curriculum.  Currently,  this  is  particularly  true  in  instructional  design  with  its  overweening  emphasis  on 
goals.  Dewey  maintained  that  goals  are  important,  but  these  are  goals  in  view  - ideals  of  what  we  want  to  achieve 
that  occur  rarely  in  exactly  the  way  we  had  initially  envisioned.  Goals  are,  therefore,  contingent  and  emergent  by 
nature  because  reality  intervenes  changing  our  goals  to  fit  circumstances  and  ever-changing  contexts. 

Foucault’s  Discipline  and  Punish 

In  modernist  pedagogies  and  systematic  instructional  design  models,  we  see  an  example  of  Foucault’s 
concept  of  “docile  bodies,”  which  manifests  itself  as  the  science  of  behavioral  control  in  a clinical  environment. 
“Docile  bodies”  relates  how  “modem  disciplinary  technology  does  for  the  human  body  and  the  body  politic  what 
Newton  had  done  for  physical  bodies;”  in  other  words,  it  has  created  Man-the-Machine;  as  Garrison  & Burton 
(1995)  claim  instructional  designers  all  too  often  presume  students  are  thinking  machines  and  extensions  of  their 
tools  (pp.  72-3).  Moral  accountability  can  now  be  quantified  as  a numerical  representation,  grades;  political  control 
thereby  manifests  itself  as  the  inexorable  controlling  agent  in  this  utilitarian  rationalization  (p.  73).  As  for  correct 
training,  Foucault  (1979)  delineates  a tripartite  hierarchy  of  power:  hierarchical  observation  - the  teacher  constantly 
monitors  student  activity  exenplified  by  traditional  classroom  organization;  normalizing  judgment- the  culture 
restructuring  itself  by  enforcing  student  accordance  with  a hegemonic  episteme;  and  examination  - determining  if 
students  meet  the  standardized  criteria  that  de  facto  reify  socio-political  norms.  Unfortunately,  Foucault’s  (1979) 
warning  that  such  a system  predicated  on  an  all  seeing  and  centralized  eye,  a panopticon,  can  come  to  fruition  in  this 
climate.  The  reliance  on  a hierarchy  sorts  individuals  as  objects  into  ability  categories  depending  on  how  well  they 
score  on  exams  developed  from  norms  taken,  in  turn,  as  fixed  or  natural  categorization  models.  Such  models  assume 
that  the  norms  are  value  neutral,  but  even  a cursory  glance  at  the  material  constituting  standardized  tests ; one  can  see 
that  the  material  is  biased  towards  the  hegemonic  values  of  a society’s  social  elites.  As  Becker  (1998)  points  out, 
such  tests  are  highly  value  laden  based  on  the  skills  that  dominant  social  groups  value,  and  mistakenly  taken  as  raw 
scores  of  intellectual  ability  and  gauges  for  future  success  - as  long  the  same  dominant  group  defines  success. 

In  a rational  world,  scientists  (social  and  physical)  discover  the  essential  meaning  of  things,  the  monad. 
Latour  (1987)  remarks  how  in  typical  scientific  processes,  real  things  are  abstracted  into  laboratory  symbols 
cleansed  of  interference  from  the  outside  world;  such  abstractions  have  little  to  do,  however,  with  the  initial  thing 
that  actually  exists  in  its  environment.  In  a less  rational  world,  one  that  is  not  reduced  to  the  world  as  a controlled 
laboratory,  essential  meaning  is  a chimera,  so  why  should  we  base  our  pedagogies  on  a worldview  that  purposefully 
ignores  the  richness,  diversity,  and  complexities  of  what  it  is  to  be  human  caught  up  in  the  nebulous  sweep  of 
existence?  Foucault  in  The  Order  of  Things:  An  Archeology  of  the  Human  Sciences  (1971  ^ provides  an  answer: 
because  people  in  power,  who  claim  to  discover  truth,  actually  construct  it.  Foucault  writes,  “the  problem  is  not 
changing  people’s  consciousness  - or  what’s  in  their  heads  - but  the  political,  economic,  institutional  regime  of 
production  of  truth”  (p.  133).  When  we  acknowledge  that  truth  is  not  fixed  in  an  ultimate  origin  (arche),  is  not 
predestined  to  a specific  end  (tools),  nor  has  an  essential  value  (monad),  we  can  see  that  reality  (ousia)  is  contingent 
on  context  and  one’s  perspective  within  a given  locus.  When  this  emancipatory  vision  occurs,  we  can  pull  down  the 
edifices  that  sustain  hierarchies,  rules,  and  categories  as  givens  and  rebuild  pedagogy  around  concepts  of  relevance. 

Friere’s  Pedagogy  of  the  Oppressed 

Instructional  design  methods  typically  rely  on  specious  pedagogical  strategies  of  “facilitat[ing]  knowledge 
transfer,”  which  Friere  (1973)  critiques  as  the  banking  concept  of  learning.  In  Pedagogy  of  the  Oppressed,  Freire 
stipulates  that  humans  exist  to  change  the  world  through  dialogue:  “To  exist,  humanely,  is  to  name  the  world,  to 
change  it”  (p.  1 50).  Naming  the  world  occurs  in  transacting  with  the  world.  Transaction  is  a process  within  a 
functioning  democracy;  domination  of  dialogue,  becoming  a monologue  of  the  dominator  transferred  onto  the 
dominated,  manifests  itself  as  pedagogical  sadism.  Tragically,  this  sadism  is  the  typical  instructional  design 
mentality  in  which  the  content  and  content  specialist,  master  the  student  and  correct  the  student  behavior  though 
grades  dictated  upon  how  well  students  retrieve  information  placed  in  their  long-term  storage.  Freire  claims  in  the 
banking  concept  of  education  that  the  teacher  deposits  knowledge,  much  like  a capitalist  would,  in  order  to  retrieve 
his  or  her  funds  at  a later  date,  in  this  case  from  the  student^ank,  with  interest.  The  accrued  interest,  on  top  of  the 
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correct  response  to  the  answer,  is  the  student’s  mindset  that  he  or  she  is  essentially  powerless  in  this  exchange.  The 
dividend  for  the  capitalist  is  proletariat  passivity. 

Eisner’s  Three  Curricula 

Using  Eisner’s  (1994)  three  curricula  - explicit,  implicit,  and  null,  - we  can  deconstruct  what  Tech  intends 
to  promote:  explicit,  the  agenda  illustrates  desire  for  responsible,  self-sufficient,  active  learners,  who  proactively 
contribute  much  to  their  own  learning  goals  and  methods;  implicit,  the  methods  are  designed  to  make  students  react 
to  external  stimuli  in  a proscribed  manner  (e.g.  fill-in-the-blanks  and  multiple -choice),  creating  passive  students 
given  precious  little  room  for  critique  and  analysis,  two  keys  for  active  learning;  null,  the  content  is  predetermined 
and  predominate,  so  little  freeplay  exists  for  student  discovery — the  assumption  here  is  that  knowledge  is  finite, 
fixed,  and  ultimately  determinable  to  an  absolute  value.  Tech  has  pronounced  a knowledge-in -action  agenda,  yet  has 
promulgated  a knowledge-out-of-context  methodology.  The  strength  of  this  instructional  design  is  that  students  tend 
to  do  better  on  conduit  model  testing,  yet  their  critical  analytical  skills  suffer:  “Such  a curriculum  of  knowledge-out- 
of  context  may  enable  students  to  do  well  on  multiple  -choice  items.  It  does  not  enable  them  to  enter  on  their  own 
into  our  vital  academic  traditions  of  knowing  and  doing.  They  lack  the  skills  to  develop  interpretation,  to  analyze  a 
new  situation,  or  to  muster  evidence  in  support  of  new  arguments  and  unexpected  opinions”  (Applebee,  1996,  p.33). 
The  underlying  problem  resides  in  the  privileged  status  of  the  content  as  the  origin,  ends,  and  fixed  meaning  of 
knowledge.  We  are  carrying  the  baggage  from  Plato’s  “myth  of  the  cave”  where  knowledge,  episteme,  is  ultimately 
and  permanently  definable  to  a fixed  point — a monad.  This,  in  turn,  leads  to  suspect  pedagogical  methodologies  that 
emphasize  knowledge-out-of-context.  Applebee  (1996)  describes  how  this  mindset  affects  methods: 

Educators  have  relied  on  classroom  practices  that  focus  almost  exclusively  on  memory,  allowing  goals  of 
active  reasoning  and  participation  to  fall  by  the  wayside.  Instead  of  the  knowledge-in-action  that  both 
allows  and  develops  through  participation  in  culturally  significant  traditions  of  discourse,  we  have 
emphasized  the  knowledge-out-of-context  that  comes  from  studying  its  characteristics  (p.  26). 

This  reliance  on  a contextualized  knowledge  may  well  enable  students  to  do  well  on  multiple  -choice  and 
fill-in-the-blank  tests,  but  does  precious  little  to  prepare  them  for  a world  that  does  not  function  in  such  a reductive 
manner.  Subsequently,  this  method  instills  a dichotomous  world-view  in  which  students  learn  that  real  world 
decisions  can  be  distilled  to  either/or  solutions  that  reduce  complexity  at  the  expense  of  creativity. 

The  task  at  hand  is  to  find  ways  to  salvage  the  goals  of  the  academic  agenda  from  the  myopic  and  ill- 
conceived  methods  adopted  from  information  technology.  In  this  decade  scholars  from  various  disciplines  have 
offered  warnings  about  assumptions  inculcated  within  this  transformation  (I  use  this  term  generously  for  now 
because  pedagogical  praxis  has  undergone  precious  little  change  while  the  medium  has)  and  propositions  for 
offering  students  to  become  more  participatory  and  active  learners  in  the  environment.  If  we  give  heed  to  and  adopt 
humanist,  post-structuralist,  and  pragmatic  misgivings  and  sensibilities  respectively,  we  may  actually  take  some 
meaningful  steps  towards  skilling  active  participants  in  a multivocal  and  participatory  democracy-  a much  more 
preferable  locus  in  a public  university  than  jumping  so  readily  into  bed  with  market  place  positivistic  assumptions. 
Specifically,  I intend  to  look  at  Garrison  and  Burton’s  (1995)  warnings  voiced  in  “Power,  Knowledge,  and 
Hypermedia”  and  George  Landow’s  (1994)  call  to  move  hypertext  towards  post-modernism  delineated  in  Hypertext; 
The  Convergence  of  Contemporary  Critical  Theory  and  Technology. 

Garrison  and  Burton’s  Skepticism 

Garrison  and  Burton  (1995),  in  “Knowledge,  Power,  and  Hypermedia”  cite  Nelson’s  critiques  of  scientific 
learning  theories  harking  back  to  Taylor  and  Bobbitt’s  scientific  management  models  that  have  resurfaced  in  current 
conduct  and  accountability  centered  educational  models  demanding  that  education  emulate  the  market  place.  Nelson 
balks  at  the  oppressive  nature  of  bureaucratic  scienticism  that  often  fails  to  take  learner  relevance  and  educational 
context  into  consideration.  In  contrast.  Nelson  offers  his  Xanadu  concept  focusing  on  open  hypertext  as  opposed  to 
universal  hypertext.  Open  hypertext,  simply  put,  allows  users  to  create  their  own  links  and  add  information  to  a 
naturally  evolving  matrix,  whereas  a universal  hypertext,  much  like  Vannevar  Bush  outlined  in  “As  we  may  think,” 
is  constructed  by  specialists  bound  by  fixed  hierarchies  and  standardized  rules.  In  the  former  case,  we  have 
“computer-text -system  people”  who  value  everyone’s  contribution;  in  the  latter  case,  we  have  “information  Lords” 
controlling  content  and  access  by  “information  Peons”  (p.  71). 

Landow  and  Hypertextuality 
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Landow  holds  forth  hope  that  hypertext,  hypermedia,  and  on-line  learning  environments  may  accomplish 
some  of  the  post  structuralism  goals:  “we  must  abandon  conceptual  systems  founded  upon  ideas  of  center,  margin, 
hierarchy,  and  linearity  and  replace  them  with  ones  of  multilinearity,  nodes,  links,  and  networks”  (p.  752).  Landow 
emphasizes  that  electronic  links  create  more  easily  accessible  “lexias”  to  external  links  increasing  the  viability  for 
intertextuality.  This  intertextuality,  in  turn,  helps  reduce  the  status  of  the  author  at  the  expense  of  the  reader: 
“hypertext  blurs  the  boundaries  between  reader  and  writer”  (p.  755).  Barthes’  “readerly  text”  comes  to  the  fore 
seeking  to  create  a text  for  active  readership  and  disestablish  the  “pitiless  divorce”  between  producer/user, 
owner/customer,  and  author/reader  to  which  one  may  readily  add  teacher/pupil  (p.  755).  The  non-linear  links  in  a 
hypertext  and  the  reader’s  ability  to  add  to  the  text,  involving  feedback  from  other  writers,  offers  a more  active  role 
for  the  traditionally  passive  reader.  The  reader/student  becomes  an  active  participant  in  making  meaning  thereby 
increasing  the  relevance  and  the  links  to  the  reality  of  the  transactional,  lived  experience. 

Derrida’s  Deconstructing  of  the  Transcendent  Signified 

With  respect  to  Derrida  (2000),  hypertext  offers  a text  more  closely  aligned  to  our  lived  experiences  in 
which  context  as  the  center  of  meaning  takes  the  place  of  a contextualized  truths  or  structural  centers.  A living 
hypertext  is  constantly  restructured  and  recentered  as  the  context  shifts  creating  an  infinity  of  new  contexts. 
Hyperpedagogy  uses  a similar  paradigm  in  which  the  class  - here  defined  as  participants,  content,  and  context  in  a 
transactional  environment  - becomes  an  assemblage  ora  constantly  mediated  montage  of  meanings.  Derrida  quotes, 
“I  believe  that  the  center  is  a function,  not  a being — a reality,  but  a function.  And  this  function  is  absolutely 
indispensable”  (p.  495).  By  moving  the  locus  of  significance  from  essence  to  function,  Derrida  effectively 
deconstructs  the  viability  of  fixed  meaning  that  examination  standardization  strategies,  conduit-teaching  models,  and 
panoptic  pedagogies  that  rely  heavily  on  prerequisites  and  like  -minded  philosophical  assumptions. 

Derrida  (2000),  in  “Structure,  Sign,  and  Play  in  the  Discourse  of  Human  Sciences,”  questions  the  function 
of  structuralism  by  deconstructing  certain  Platonic  and  Cartesian  assumptions  regarding  the  privileged  status  of 
structural  centers.  After  Derrida  concludes  his  argument,  the  validity  of  the  “myth  of  the  cave”  from  Plato’s 
Republic  (1985)  and  the  cogito  ergo  sum  from  Descartes’  Meditating  on  First  Philosophy  (1993)  lay  in  shards.  He 
stipulates  that  Western  philosophy’s  foundational  assumptions,  so  deeply  intertwined  within  the  structure  of 
episteme,  need  to  be  seriously  revaluated.  The  center  of  traditional  philosophical  structures,  at  once  part  of  the 
structure  and  simultaneously  existing  transcendency  beyond  its  grasp,  are  not  centers  at  all.  Transcendence  is  a 
central  tenet  in  the  metaphysics  of  presence,  what  Derrida  labels  the  transcendent  signified.  When  we  remove  the 
concept  of  transcendent  signified  and  allow  for  freeplay,  we  extend  the  domain  and  interplay  of  signification 
infinitely  (p.  496).  As  he  stipulates,  “Freeplay  is  the  disruption  of  presence”  (p.  508).  Recognizing  that  structures  are 
flexible  and  adaptive  to  the  demands  of  place  and  time,  ruptures  the  eschatological  belief  in  epistemology  inexorably 
linked  to  the  ideologies  of  ultimate  knowledge  (episteme),  origins  (arche),  and  ends  (tools).  Reality  is  no  longer  a 
discovered  monad  or  essence  confined  by  the  alpha  of  arche  and  the  omega  of  tools.  By  admitting  freeplay  room  in 
our  concepts  of  reality,  we  can  deny  the  dualities  inculcated  within  the  metaphysics  of  presence:  phvsis/nomo  s and 
phvsis/techne.  As  Derrida  writes,  “the  whole  historical  chain  which  opposes  ‘nature’  to  the  law,  to  education,  to  art, 
to  techniques — and  also  to  liberty,  to  the  arbitrary,  to  history,  to  society,  to  the  mind”  deconstructs  the  limitations 
placed  on  pedagogues  to  reconstruct  reality  and  teaching  models  (p.  499).  Derrida,  furthermore,  writes  how 
deconstruction  of  utilitarian  empiricism  will  expose  the  limitations  of  ideologies  invested  in  fixed,  timeless,  a 
contextual  truths;  seen  as  tools,  however,  we  can  reconstruct  reality  as  a function.  By  breaking  down  these  dualistic 
barriers  - the  supposedly  inherent  tensions  between  culture  and  nature,  mind  and  body,  education  and  nature  - fade 
into  irrelevancy. 

Regarding  the  prevalence  of  dualisms/binaries,  Bowker  and  Star  (1999)  argue  that  tensions  often  arise  from 
globalization  of  categories.  Local  categories,  meaningful  segmentation  of  the  overwhelming  myriad  of  reality,  upon 
becoming  universal  standards,  codified  and  global,  lose  relevance  to  immediate  tasks  at  hand  and  act  as  barriers  to 
understanding.  Moreover,  those  who  codify  global  standards  take  on  the  mantle  of  authority  and  usurp  the  power 
away  from  the  recipients  of  their  seemingly  arbitrary  categories,  much  like  the  relationship  described  by  Nelson 
between  information  lords  and  peons. 

When  we  accept  that  knowledge  is  neither  teleological  nor  eschatological,  we  can  question  the  privileged 
status  of  the  text  and  teacher,  the  quintessential  classroom  authorities.  In  order  to  questions  the  myths  surrounding 
the  author,  we  need  to  investigate  textual  possessions.  Simply  put,  textual  possessions  refers  to  the  fact  that  most 
effective  (and  the  most  affective)  on-line  collaboration  and  instruction  occurs  as  on-line  textual  communication  not 
as  multimedia  lectures  that  emphasize  the  privileged  status  of  the  content  and  the  content  specialist.  Therefore, 
poststructuralist  theories  concerning  text,  particularly  those  promulgated  by  Derrida,  Foucault,  and  Barthes,  are 
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especially  relevant.  Moreover,  collaborative  work  in  digital  culture  should  more  closely  resemble  renaissance 
coteries  than  the  authorial  work  models  predominate  in  print  culture.  In  coteries  the  primary  method  for  sharing 
knowledge  was  not  conduit  model  - the  textbook  and  professor  as  the  source  of  static  knowledge  to  be  recited  much 
like  a litany  - but  a dialogic  one  in  which  the  contributors  engaged  in  dialectic  disputation  (Downs-Gamble,  1995). 
The  renaissance  manuscript  chapbook  wias  mutable,  emergent,  and  co-produced  text  neither  definable  to  a fixed 
value  nor  attributed  to  a single  author.  The  latter  term  itself  is  problematic  because  its  etymology  resides  in  the  Latin 
root  auctores . meaning  the  authority  on  a subject  typically  referring  to  a religious  subject  (Pask,  1996). 

Foucault,  Barthes,  and  Authority 

Foucault  (1997)  and  Barthes  (1998a)  deconstruct  the  concept  of  author  in  respectively,  “What  is  an 
Author”  and  “Death  of  the  Author.”  For  Foucault  the  modem  conception  of  author  “constitutes  the  privileged 
moment  of  individualization”  (p.  890).  He  writes  that  writing  is  a ]^(game/freeplay)  of  a writer  as  part  of  a matrix: 
“it  is  a question  of  creating  a space  into  which  the  writing  subject  constantly  disappears”  (p.  890).  This  claim  denies 
the  privileged  place  of  author/professor/content  specialist/instmctional  designer  as  the  authority  on  a particular,  set 
subject  or  method  of  teaching.  He  pits  the  historical  function  of  author  against  the  modem  ideal  of  author.  He  writes 
that  historically  the  author-function  exists  as  four  primary  characteristics:  (1)  the  author-function  is  linked  to  the 
juridical  and  institutional  system  that  encompasses,  determines,  and  articulates  the  universe  of  discourses;  (2)  it  does 
not  affect  all  discourses  in  the  same  way  at  all  times  and  in  all  types  of  civilization;  (3)  it  is  not  defined  by  the 
spontaneous  attribution  of  a discourse  to  its  producer,  but  rather  a series  of  specific  and  complex  operations;  and  (4), 
it  does  not  refer  purely  and  simply  to  a real  individual,  since  it  can  give  rise  simultaneously  to  several  selves,  to 
several  subjects — positions  that  can  be  occupied  by  different  classes  of  individuals  (p.  896). 

Barthes  (1998a)  proclaims  the  death  of  the  monolithic  author  occurs  in  the  act  of  writing  (poesis).  Modem 
concepts  of  author  hinge  upon  beliefs  of  univocality  and  singularity  of  purpose  and  knowledge,  yet  Barthes  writes 
“as  soon  as  a fact  is  narrated  no  longer  with  a view  to  acting  directly  on  reality  but  intransitively,  that  is  to  say 
finally  outside  of  any  function  other  than  that  of  the  very  practice  of  the  symbol  itself,  this  disconnection  occurs,  the 
voice  loses  its  origin,  the  author  enters  into  his  own  death,  writing  begins”  (p.  253).  In  one  fell  swoop,  Barthes  tears 
down  the  monolithic  structure  of  autonomous  author  to  reveal  the  character  of  writer  as  practicing  a craft  within  a 
broad  social  milieu.  The  author  for  Barthes  is  a production  of  modem  capitalist  notions  of  liability  and  ownership 
and  a positivist  tyranny.  The  scripter  and  the  text  do  not  exist  timelessly  but  in  the  here  and  now  during  and  within 
the  acts  (praxis)  of  production  and  reading.  The  performance  occurs  in  the  moment  of  production  and  meaning  takes 
shape  in  the  process  of  reading,  never  decipherable  to  an  exact  essence  of  the  text.  Texts  have  no  ultimate  meaning 
tied  to  “God  and  his  hypostases — reason,  science,  law”  (p.  256).  He  claims  that  “Classic  criticism  has  never  paid 
any  attention  to  the  reader,  for  it,  the  writer  is  the  only  person  in  literature”  (p.  257).  The  death  of  the  modem 
concept  of  uni  vocal  and  authoritative  instructor/author  gives  birth  to  the  active  student/reader  as  both  recipient  and 
interpreter.  The  modem  authorial,  hierarchical  stance  also  predicates  discourse  surrounding  gender  and  space. 

Boler,  Massey,  and  Power  Geometries 

At  an  AERA  symposium  last  spring,  Boler  (2001)  spoke  on  “Real  and  Virtual  Gendered  Identities  in 
Educational  Landscapes.”  She  writes,  “The  apparent  ‘disembodiment’  created  in  cyber  culture  poses  a genuine 
dilemma  for  feminist  and  socially -progressive  educators”  (p.  1).  She  declares  that  the  phallocentric  conception  that 
the  body  is  central  to  the  production  of  knowledge  and  the  Platonic/Cartesian  stipulation  that  the  body  needs  to  be 
transcended  as  an  unclean  and  feminine  entity  corrupting  knowledge  and  tmth  dominates  discourses  surrounding 
hypertextuality.  She  juxtaposes  her  skepticism  of  cyber  culture’s  claim  to  be  a non-gendered,  non-racial,  anti- 
chauvinistic  space  with  Massey’s  (1 993)  critique  of  space  anxiety  and  power  geometries. 

Massey  (1993)  in  her  “Power  Geometry  and  a Progressive  Sense  of  Place”  refers  to  how  localities  are  not 
as  homogenous  and  local  as  they  appear  but  are  affected  by  power  geometries  of  local  heterogeneous  values  and 
global  (extra-local)  agencies: 

The  uniqueness  of  a place,  or  a locality,  in  other  words  is  constructed  out  of  particular  interactions  and 
mutual  articulations  of  social  relations,  social  processes,  experiences  and  understandings,  in  a situation  of 
co-presence,  but  where  a large  proportion  of  those  relations,  experiences  and  understandings  are  actually 
constmcted  on  a far  larger  scale  than  what  we  happen  to  define  for  that  moment  as  the  place  itself,  whether 
that  be  a street,  a region  or  a continent.  Instead  then,  of  thinking  of  places  as  areas  with  boundaries  around, 
they  can  be  imagined  as  articulated  moments  in  networks  of  social  relations  and  understandings  (p.  66). 


531 


42 


The  preponderance  of  nostalgic  spatial  language  is  an  aspect  of  power  geometries  in  which  hegemonic 
influences  attempt  to  contain  and  limit  the  chaos  supposedly  non-spatial  and  extemporal  cyber  culture  represents. 
Hegemonic  groups  use  the  time/space  compression  of  cyber  culture  to  further  entrench  the  digital  divide  - women, 
non-whites,  and  poor  people  rarely  find  access  to  social  and  economic  power-geometries  that  white,  middle  to  upper 
class  males  do.  Internet  access  alone  does  not  guarantee  access  to  power  manifested  within  cyber  culture  space. 
While  more  women  are  accessing  online  spaces,  they  are  often  corralled  into  places  that  define  them  as  feminine  and 
marginalized  from  power.  The  most  popular  sites,  such  as  girl  chat  rooms,  traditionally  gendered  spaces  like 
seventeen  and  cosmo  -girl,  and  online  shopping  in  gendered  specified  places,  reify  stereotypical  feminine  roles. 

Boler  (2001)  concludes  that  “the  nostalgia  for  place,  authenticity,  and  stable  identity  which  Massey  recognizes  as  a 
masculine  nostalgic  reaction  in  relation  to  time -space  compression  accurately  explains  the  reinscription  of  space  in 
digital  culture”  (p.  4). 

Traditional  instructional  design  models  tend  to  see  classrooms  (localities)  as  isolated  places  that  exist 
somehow  beyond  the  confines  of  a larger  reality.  Moreover,  they  regulate  the  social  relationships  among  participants 
into  strict  hierarchies  of  power  and  limit  networks  to  homogenous  and  hierarchical  panopticons  of  power  through 
knowledge  transfer  from  information  lords  to  information  peons. 

Systematic  Instructional  Design 

Often  instructional  designers  and  instructional  technologists  trained  in  the  use  of  various  modernist  learning 
models,  particularly  Smith  and  Ragan  (1993),  Mager  (1997),  and  Dick  and  Carey  (1996),  engineer  course 
transformations  from  proximal  to  online.  During  these  transformations,  designers  often  imbrue  the  course  with 
modernist  pedagogical  assumptions  by  implementing  one  of  the  popular  instructional  design  models.  Traditional 
instructional  design’s  reliance  on  the  privileged  position  of  goals  creates  superordinate  structures  that  circumscribe 
student  activity  and  reinforce  fixed  domains  of  knowledge.  Smith  and  Ragan  (1993)  write  that 

Instruction  is  the  delivery  of  information  and  activities  that  facilitate  learners’  attainment  of  intended, 
specific  goals.  In  other  words,  instruction  in  the  conduct  of  activities  that  are  focused  on  learners  learning 
specific  things. . . . Every  learning  experience  that  is  developed  is  focused  toward  a particular  goal.  (p.  2-3) 
The  student  is  passive  and  secondary  to  attainment  of  a goal  he  or  she  has  no  voice  in  choosing  or 
manipulating  to  meet  his  or  her  needs  and  desires.  The  learner  described  in  this  quotation  is  a presumptive 
automaton  ready  for  normalization  that  leads  inexorably  to  a standardized  product  ready  for  the  economic  machine. 
We  can  easily  see  Bobbitt  and  Tyler’s  philosophical  assumptions  playing  in  this  statement.  Moreover,  teleological 
structures  that  emphasize  regulation  and  particularization  of  fixed  goals  reify  the  power  geometry  of  the  designer’s 
privileged  status  at  the  expense  of  both  the  professor  (denigrated  to  a content  specialist)  and  the  student  (now  little 
more  than  content  assimilator).  This  dissemination  into  fixed  roles,  additionally,  dehumanizes  and  regulates  the 
process  of  learning. 

Mager  (1997)  in  his  Preparing  Instructional  Objectives  also  designates  objectives  superordinate  to  the 
learner  and  methods  as  beyond  the  learner’s  reach: 

you  must  clearly  specify  outcomes  or  objectives  you  intend  your  instruction  to  accomplish.  You  must  then 
select  and  arrange  learning  experiences  for  your  students  in  accordance  with  the  principles  of  learning  and 
must  evaluate  student  performance  according  to  the  objectives  originally  selected  (p.  1). 

The  outcomes  and  methods  belong  to  the  instructional  designer;  Mager  assumes  student  as  recipient  of 
content  he  or  she  has  no  choice  and  by  methods  in  which  he  or  she  has  no  voice.  Furthermore,  only  one  set  of 
learning  principles  seems  to  exist  - in  this  case  a form  of  reductive  behaviorism.  One  can  also  note  the  frequent  use 
of  the  imperative  of  his  own  instructional  design.  No  room  is  given  for  any  emergence,  transaction,  or  adaptation  to 
change  that  frequently  happens  in  the  emerging  reality  of  the  classroom:  “instruction  is  only  successful  to  the  degree 
that  it  succeeds  in  changing  students  in  desired  ways”  (p.  13).  The  presumption  of  student  as  automaton  is  naked 
here;  moreover,  the  instructional  designer  defines  success  for  the  learner. 

Arguably,  the  most  popular  instructional  design  model,  often  unquestioned  as  the  instructional  design 
model,  is  Dick  and  Carey’s  (1996)  The  Systematic  Design  of  Instruction.  With  its  emphasis  on  being  systematic, 
such  hierarchical  statements  should  not  surprise  one:  “The  first  step  in  the  model  is  to  determine  what  it  is  that  you 
want  learners  to  be  able  to  do  when  they  have  completed  your  instruction”  (p.  5).  While  their  belief  in  pedagogical 
ownership  is  not  nearly  as  blatant  as  Mager’ s (notably  one  of  the  theorists  informing  the  design),  the  next  quotation 
is  telling  in  how  little  pedagogical  freedom  they  afford  the  learner:  “you  will  determine  step-by-step  what  people  are 
doing”  (p.  5).  Here  the  modem,  mechanistic  nature  of  systematic  design  is  laid  bare.  Traditional  instructional  design 
clearly  follows  in  the  footsteps  of  Bobbitt  and  Tyler. 
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Conclusion 


To  return  to  the  plea  for  finding  a means  between  extremes,  we  should  consult  Garrison’s  (1997)  Dewey 
and  Eros,  Garrison,  harking  back  to  both  classical  Greek  and  Deweyian  concepts  of  education,  argues  that  modem 
education  lacks  eros,  defined  here  as  the  passionate  desire  to  achieve  an  ends.  Clearly,  the  emphasis  resides  in 
relevance,  but  whose  is  a seemingly  unsolvable  conundrum  in  most  modem,  bureaucratic  educational  theorizing. 
Often  we  rely  on  dichotomies  such  as  who  comes  first  in  choosing  the  curriculum:  the  student  or  the  teacher?  Do 
we  pass  off  one  person’s  desires  as  the  only  appropriate  ones , which  are  typically  cloaked  as  objective,  value-neutral 
standards,  or  do  we  pander  to  students’  desire  without  teacher  guidance,  much  less  supervision?  If  we,  however, 
look  for  the  common  good,  what  we  often  call  the  teachable  moment  that  is  emergent  and  co-constructed,  we  can 
avoid  this  false  dichotomy,  this  destructive  either  ...  or  logic.  In  doing  this,  we  must  pay  more  than  lip  service  to 
this  noble  goal. 

In  Curriculum  as  Conversation^  Applebee  (1996)  states  that  often  a discrepancy  between  “grand  goals  of 
exploration  and  discovery”  unfolds  and  how  the  class  is  administered  (p.21).  If  the  teachable  moment  becomes  a 
didactic  game  of  “guess  what  I’m  thinking”  in  which  the  teacher’s  knowledge  or  answer  is  more  valuable  than  the 
student’s,  then  we  are  lamentably  back  to  a pedagogy  of  the  oppressed  cloaked  by  constructivist  buzzwords.  We  are 
practicing  pure  reasoning,  a deductive  and  self-enclosed  quest  for  certainty,  as  opposed  to  practical  reasoning  that 
seeks  ends  we  desire  to  obtain.  The  means,  a constantly  negotiated  center  within  a fluid  structure,  exists  somewhere 
between  these  extremes.  The  most  appropriate  way  to  accomplish  this  shift  away  from  pedantic  pedagogy  is  to 
accept  students’  voices  as  relevant  within  their  educational  trajectories.  Regarding  curriculum  as  conversation, 
Applebee  writes, 

Schooling  should  be  organized  to  help  students  enter  into  culturally  significant  domains  for  conversation, 

themselves  representative  of  broader  cultural  traditions  of  knowing  and  doing.  By  placing  the  emphasis  on 

entry  into  such  conversations,  I seek  to  ensure  that  students  will  emerge  with  knowledge-in-action  rather 

than  knowledge-out-of-context  (p.  49) 

To  do  this  we  need  to  accept  that  knowledge  is  dynamic  rather  than  static  and  that  a means  between  student 
and  teacher  desires  discerned  through  an  emergent  and  mediated  transaction  will  yield  fluid  and  adaptive  hyper- 
pedagogies. 
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Designing  Web  Resource  Learning  Activities 


Dr.  David  Pedersen 
Embry  Riddle  Aeronautical  University 

Web  based  instruction  often  fails  to  utilize  its  most  unique  capability,  resources  found  on  the  web  itself. 

All  too  often,  lists  of  URLs  are  posted  with  the  hope  that  somehow  they  will  promote  learning.  Web  resources  are 
best  utilized  when  appropriate  learning  activities  are  designed  to  make  them  an  integral  part  of  the  learning  process. 
Designing  appropriate  web  resource  learning  activities  requires  an  understanding  of  web  resources  and  an 
application  of  basic  instructional  design  principles. 

The  web  offers  an  extensive  collection  of  sites  on  nearly  every  topic.  Some  sites  are  outstanding,  others 
mediocre  at  best.  Some  are  interesting  and  engaging  others  dull  and  boring.  Some  sites  are  authoritative,  others  just 
plain  inaccurate. 

The  first  step  in  utilizing  web  resources  is  to  find  appropriate  sites.  An  appropriate  site  should  contain 
accurate  information  related  to  course  content  that  can  be  incorporated  into  a learning  activity.  Finding  good  sites 
requires  good  searching  techniques,  discernment,  and  persistence. 

One  of  the  simplest  web  resource  learning  activities  to  help  an  instructor  begin  using  web  resources  is  a 
student  site  search  activity.  The  goal  of  the  activity  is  to  have  students  find,  describe,  and  share  websites  related  to 
specified  topics.  Students  will  need  to  know  web  searching  skills  and  how  to  evaluate  websites.  The  assignment 
requires  students  to  post  an  annotated  link  to  the  websites  they  find  on  an  electronic  discussion  board.  The  activity 
will  provide  students  with  relevant  course  information  and  instructors  with  a list  of  potential  websites  for  future  web 
resource  learning  activities  development. 

Once  a collection  of  appropriate  web  sites  has  been  located,  learning  activities  can  be  developed  that  utilize 
the  sites  effectively.  Some  sites  provide  context  for  course  topics.  Others  provide  good  factual  information.  Some 
help  with  concept  development,  and  others  provide  opportunities  for  students  to  analyze  and  synthesize  information. 
From  an  instructional  design  perspective,  websites  can  be  thought  of  as  existing  materials  that  can  be  utilized  to 
meet  course  outcomes.  The  key  is  to  match  the  inherent  characteristics  of  the  site(s)  to  the  types  of  learning 
activities  needed  to  meet  course  outcomes. 

In  keeping  with  good  instructional  design,  all  web  resource  learning  activities  should  have  an  introduction, 
an  assignment,  and  links  to  the  web  sites  for  the  activity.  While  each  of  these  components  is  specifically  tailored  to 
each  activity,  there  are  some  general  characteristics.  The  introduction  should  provide  context  for  the  activity, 
motivational  elements,  and  guidance  for  accomplishing  the  outcomes  targeted  by  the  activity.  The  assignment 
should  be  a natural  by-product  of  utilizing  the  site  that  verifies  participation  in  the  activity.  The  links  should  be 
appropriate  and  durable. 

A collection  of  interesting  sites  with  a broad  coverage  of  course  topics  can  be  used  to  create  an  information 
exploration  activity  that  provides  context  for  course  topics.  The  introduction  should  explain  how  that  the  activity 
provides  background  and  context  for  the  main  topics  in  the  course.  The  assignment  provides  a means  to  insure  that 
students  complete  the  activity.  It  may  be  as  simple  as  having  them  report  on  their  findings,  or  it  may  require  them  to 
relate  what  they  discovered  to  a key  topic  in  the  course.  The  web  sites  need  to  be  engaging.  They  should  appeal  to 
student  curiosity  and  be  visually  pleasing.  Include  sites  on  a wide  range  of  supplemental  topics.  Each  site  should 
also  have  a broad  coverage  of  topics  on  the  site. 

A knowledge  acquisition  activity  is  a good  way  to  help  students  find  important  factual  information  for  the 
course.  The  activity  introduction  should  motivate  students  by  explaining  how  the  information  is  related  to  course 
outcomes,  and  that  it  is  not  just  "busy  work."  Knowledge  acquisition  activities  should  include  questions  that  focus 
on  specific  facts  relevant  to  the  topic  at  hand.  The  goal  is  to  direct  students  toward  the  information  they  need  to 
know  about  the  topic.  The  assignment  should  require  the  students  not  only  to  have  visited  the  sites  to  gather  the 
facts,  but  also  to  utilize  those  facts  to  accomplish  the  assigned  task.  The  sites  for  the  activity  must  contain  the 
factual  information  necessary  to  answer  the  topic-related  questions.  Look  for  sites  that  focus  on  the  topic  and 
provide  information  not  included  on  other  sites.  Try  to  cover  the  topic  with  a minimal  number  of  sites. 

Web  sites  often  provide  the  examples  necessary  for  a concept  development  activity.  The  introduction  to 
the  activity  should  focus  on  the  key  elements  of  the  concept  being  learned.  The  assignment  should  provide  the 
students  with  an  opportunity  to  demonstrate  that  they  have  a clear  grasp  of  the  concept.  Several  sites  that  clearly 
demonstrate  the  attributes  of  the  concept  or  that  are  obvious  non-examples  of  the  concept  should  be  used. 

A knowledge  generation  exercise  directs  students  to  sites  containing  information  related  to  a specific 
course  topic.  The  assignment  in  the  exercise  requires  that  the  students  analyze  the  information  they  find.  They  are 
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then  required  to  synthes  ize  that  information  and  generate  a new  representation  of  that  knowledge.  Knowledge 
generation  forces  students  into  inquiry  and  higher  order  thinking.  It  is  often  appropriate  to  utilize  collaboration 
between  students  to  provide  multiple  points  of  view.  The  activity  introduction  should  arouse  the  natural  curiosity  of 
students  by  focusing  on  the  controversial  or  challenging  aspects  of  the  activity.  The  assignment  should  provide  the 
students  with  the  opportunity  to  create  a document,  presentation,  or  other  means  of  communicating  the  ideas  they 
have  generated  from  the  information  they  have  learned.  Look  for  sites  with  divergent  opinions  about  the  topic  that 
force  students  to  think  critically.  Provide  enough  sites  to  present  all  of  the  major  aspects  of  the  topic. 

Web  resources  can  provide  effective  learning  activities.  Appropriate  selection,  design,  and  utilization  will 
optimize  the  use  of  those  resources  and  make  them  an  important  part  of  your  course. 
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LEARNING  AT  A DISTANCE  in  South  Dakota:  Description  and  Evaluation  of  the 

Diffusion  of  a Distance  Education 


Michael  Simonson 
Nova  Southeastern  University 
Tamara  Bauck 

Department  of  Education  and  Cultural  Affairs 

Background:  Distance  Education  in  South  Dakota 

The  fall  of  2000  has  brought  broad  new  opportunities  to  K-12  education  in  South  Dakota.  Beginning  on 
August  1 5,  2000  interactive  videoconferencing  classes  began  to  be  sent  over  the  Digital  Dakota  Network  (DDN),  a 
state-wide  telecommunications  network  connecting  all  176  school  districts  in  the  state.  The  DDN  provides  schools 
with  free  Internet,  videoconferencing  and  e-mail.  Although  the  DDN  and  statewide  videoconferencing  is  new  to 
South  Dakota,  regional  video  conferencing  within  the  state  is  not. 

As  early  as  1994  classes  were  offered  over  the  North  Central  Area  Interconnect  (NCAI),  a video 
consortium  of  eight  schools  in  the  northeast  part  of  South  Dakota.  An  advisory  committee  made  up  of  one  principal 
from  each  district  advises  the  governing  board  of  the  NCAI.  The  governing  board  who  actually  makes  the  decisions 
is  composed  of  a superintendent  from  each  district.  It  is  estimated  that  the  original  development  of  the  NCAI  system 
cost  $1 .3  million.  A portion  of  the  funding  came  from  a Rural  Electrification  Administration  grant  with  the  member 
districts  providing  the  remainder  of  the  startup  costs.  Member  schools  presently  pay  $1 1 ,000  a year  membership  fee 
and  educators  teaching  over  the  system  are  given  a $450  stipend  a semester. 

Early  class  offerings  on  the  NCAI  included  Spanish  and  Dakota  Indian  Art.  NCAI  has  continued  to  expand 
their  course  offerings  to  students  and  by  the  spring  of  2000,  NCAI  was  in  a situation  where  they  were  contemplating 
adding  an  additional  videoconferencing  classroom  to  meet  their  student  curricular  needs. 

Lloyd  Trautman,  a physics  teacher  on  the  NCAI,  and  his  wife,  a Spanish  teacher  over  the  network,  have 
been  teaching  on  the  system  since  the  first  year.  Lloyd  was  among  distance  education’s’  harshest  critics  when  the 
NCAI  started.  Today  Lloyd  is  one  of  distance  education’s  strongest  advocates  in  the  state  because  of  his  experience 
and  the  benefits  of  distance  education  that  he  has  seen  firsthand.  In  a school  with  an  enrollment  of  100  students  in 
grades  K-1 2,  distance  education  has  met  a variety  of  needs  that  would  never  have  been  possible.  The  NCAI  schools 
have  experienced  the  benefits  of  distance  education 

A second  video  consortium,  the  Sanborn  Interactive  Video  Network  (SIVN),  began  offering  classes  in 
January  of  1 996.  This  consortium  of  six  K- 1 2 schools,  a private  university,  and  a technical  institute,  utilized  a Rural 
Utilities  Services  (RUS)  grant  to  fund  a portion  of  the  startup  costs  to  establish  their  network.  Member  schools  pay 
$3,000  a year  for  maintenance  and  administration  of  the  system.  A $400  stipend  is  paid  to  individuals  teaching  a 
course  over  the  network.  The  SIVN  presently  offers  seven  classes  over  the  system  and  has  a seen  an  increase  in 
participation  each  year.  More  information  on  the  SIVN  can  be  found  at:  http://mti.tcc.sd.us/teleport/sivn.htm. 

The  Southeast  Interactive  Long  Distance  Learning  (SILDL)  started  offering  classes  in  the  fall  of  1 998  to  its 
1 1 member  schools  in  the  southeast  comer  of  the  state.  A portion  of  this  million-dollar  system  was  paid  for  by  a 
RUS  grant.  Members  pay  $3,000  a year  to  cover  administration  of  the  system  and  pay  a $500  stipend  to  educators 
that  teach  over  the  system. 

The  SILDL  started  with  9 classes  and  100  students  and  within  two  years  is  offering  15  classes  to  225 
students.  This  consortium  has  also  been  in  a position  of  needing  to  add  a second  video  classroom  to  cover  the 
demands  of  the  member  schools.  The  principals  and  superintendents  of  the  member  schools  have  regular  meetings 
where  they  indicate  which  classes  they  are  able  to  offer  and  which  classes  they  need.  Additional  information  on  the 
SILDL  can  be  found  at  their  website:  www.usd.edu/sildl. 

A fourth  video  consortium,  the  East  Central  Interconnect  (ECI),  started  classes  in  the  fall  of  1999.  The  ECl 
received  a RUS  grant  which  provided  a strong  start  for  the  funding  of  this  million-dollar  system.  Teachers  on  the 
ECI  are  paid  a $600  stipend  per  semester  of  teaching.  A school  board  member  from  each  of  the  10  member  schools 
governs  the  ECI.  They  receive  advice  from  two  advisory  groups  of  representative  superintendents  and  principals. 
The  ECI  offered  nine  classes  to  1 1 9 students  its’  first  year  and  eight  classes  to  1 3 1 students  during  the  2000-2001 
school  year. 

All  four  of  these  consortiums  provide  their  teachers  with  training  on  equipment  and  strategies  on  teaching 
at  a distance.  Each  has  developed  policies  that  govern  the  activities  of  each  consortium.  All  have  felt  that  their 
choice  to  try  videoconferencing  as  a means  of  distance  education  for  their  students  was  a wise  decision  and 
investment. 
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Additional  initiatives  by  other  districts  have  also  occurred.  Two  consortia  of  K-1 2 schools  are  in  the 
planning  stages  of  implementing  a video  network.  Two  independent  school  districts  have  also  purchased  video 

conferencing  equipment  which  they  have  used  over  the  last  two  years  to  connect  their  students  with  resources  inside 
and  outside  the  state. 

In  1994  a satellite  network,  the  Rural  Development  Telecommunications  Network  (RDTN),  was 
established  across  the  state.  It  consists  of  1 8 two-way  audio/video  studios  located  throughout  the  state  at 
universities  and  technical  institutes.  In  addition,  80  downlink  sites  located  primarily  at  school  districts  were 
connected  to  the  RDTN.  These  sites  have  one  way  video/two-way  audio.  Use  of  the  RDTN  network  is  on  a fee  per 
use  basis.  The  RDTN  has  provided  South  Dakota  citizens  with  the  opportunity  to  experience  the  opportunities  that  a 
two-way  audio/video  system  can  provide  a large  geographic  region  like  a state. 

This  network  has  been  used  widely  by  government,  education,  and  health  organizations  to  provide 
information  and  training.  The  RDTN  satellite  downlinks  have  also  provided  a vehicle  for  several  rural  schools  in  the 
state  to  receive  high  school  Spanish  and  Chemistry.  Many  districts  have  taken  advantage  of  these  classes  which 
they  would  otherwise  not  been  able  to  offer  their  students. 

Distance  education  at  the  higher  education  level  in  the  state  began  in  1 9 1 4 at  South  Dakota  State  University 
with  extension  activities,  and  in  1915  the  University  of  South  Dakota  began  offering  correspondence  courses.  The 
six  state  universities  have  taken  advantage  of  the  two-way  audio/  one-way  video  capabilities  of  the  RDTN  to  offer 
courses  to  college  level  students  and  to  offer  dual  credit  to  high  school  students. 

More  recently.  Governor  William  Janklow  allocated  funds  to  each  of  the  five  state  universities  to  establish 
a smart  classroom”  or  a Governor’s  Electronic  Classroom  (GEC).  Each  classroom  contained  individual  computer 
workstations  and  video  conferencing  equipment.  The  GECs  provided  high  tech  classrooms  for  universities  to 
provide  instruction  at  distance. 

On  July  1 , 2000,  the  Electronic  University  Consortium  (EUC)  of  South  Dakota  officially  began  operation. 
This  consortium  is  intended  to  leverage  the  state’s  technology  investments  and  make  effective  use  of  the  unique 
strengths  of  each  public  university  to  better  serve  the  people  of  South  Dakota  by  coordinating  off-campus  distance 
education  across  the  South  Dakota  System  of  Public  Higher  Education.  This  consortium  was  made  possible  through 

legislative  action  during  the  2000  session  which  made  funding  available  for  staff  to  coordinate  the  consortium’s 
activities. 

In  the  fall  of  2000,  the  higher  education  institutions  in  the  state  are  offering  distance  learning  opportunities 
via  the  internet,  satellite,  videocassette,  public  television,  and  correspondence.  With  over  700  South  Dakota  college 
students  enrolled  in  multiple  institutions  within  the  state  taking  dis  tance  education  courses  during  the  fall  2000 
semester,  the  face  of  higher  education  in  the  state  is  also  changing. 

Since  1995  Governor  William  Janklow  has  slowly  and  very  carefully  built  a robust  technical  infrastructure 
across  the  state  of  South  Dakota.  In  1995  he  initiated  the  Wiring  the  Schools  (WTS)  project  which  put  three 
computer  drops  for  every  four  students  in  every  classroom,  pulled  Cat5  and  fiber  optic  wiring  throughout  the 
schools,  and  upgraded  the  electrical  wiring  to  manage  the  greater  electrical  demands  of  numerous  computers.  (See 
accompanying  article  regarding  WTS.)  The  work  of  WTS  included  all  public  school  classrooms,  private  schools, 
public  libraries,  and  both  public  and  private  university  classrooms  and  dormitories. 

In  the  spring  of  1 999  Governor  Janklow  announced  a second  statewide  initiative,  Connecting  the  Schools 
(CTS).  This  initiative  built  a statewide  intranet  among  all  176  school  districts  bringing  T1  access  into  every  public 
school  building  K-12.  High-end  two-way  audio/video  systems  were  put  into  almost  every  public  high  school  and 
freestanding  middle  school.  (See  related  article  on  CTS.)  The  Connecting  the  Schools  project  built  upon  the  efforts 
of  Wiring  the  Schools  project,  establishing  a statewide  network  called  the  Digital  Dakota  Network  (DDN).  The  ’ 
Internet  access  and  e-mail  services  provided  via  the  DDN  are  free  to  all  public  schools.  Video  conferencing 
connections  made  within  the  state  are  also  provided  free  to  all  schools. 

Even  though  a huge  investment  has  been  made  in  the  infrastructure  of  the  DDN,  Governor  Janklow  has 
always  recognized  the  necessity  of  developing  human  infrastructure  as  well.  During  the  2000  session,  the  South 
Dakota  Legislature  created  a new  office  within  Department  of  Education  and  Cultural  Affairs  called  the  Office  of 
Educational  Technology.  According  to  Section  4 of  House  Bill  1257,  the  Office  of  Educational  Technology’s 
"exclusive  role  shall  be  assisting  local  school  districts  in  using  educational  technology.  Its  purpose  shall  include 
researching,  analyzing,  procuring,  and  distributing  programs  and  methods  using  educational  technology  in  South 
Dakota  K-12  schools  and  classrooms."  The  office  consists  of  a program  manager,  four  technology  integration 
specialists,  the  Department  webmaster,  and  clerical  support.  The  technology  integration  specialists  provide  direct 
assistance  to  districts  and  individual  teachers  on  the  use  of  the  videoconferencing  equipment  and  instructional 
strategies.  The  assistance  these  specialists  have  and  will  provide  districts  will  be  one  of  the  keys  to  the  successful 
utilization  of  the  DDN. 
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In  addition  to  the  technical  assistance  provided  to  schools,  quality,  long-term  professional  development  was 
also  provided.  In  the  summer  of  2000  a Distance  Teaching  and  Learning  (DTL)  Academy  was  made  available  to 
South  Dakota  educators.  This  academy  was  modeled  after  the  Technology  for  Teaching  and  Learning  (TTL) 
Academies  already  implemented  in  the  state  (see  related  TTL  article). 

Two  sessions  of  this  three-week  academy  were  held  at  a state  university  with  about  130  teachers  in 
attendance.  Participants  focused  on  the  basics  of  operating  the  video  conferencing  equipment  and  adapting 
curriculum  for  distance  delivery.  (See  related  article  on  the  DTL  Academies.)  With  six  of  the  VTEL  LCSOOOs 
bridged  together  during  the  academy,  participants  were  able  to  practice  in  a “live”  situation.  This  academy  has 
follow-up  opportunities  for  participants  throughout  the  school  year  utilizing  the  DDN’s  videoconferencing 
capabilities.  Subsequent  DTL  Academies  are  being  planned  so  that  they  are  delivered  during  the  summer  at  school 
district  locations  utilizing  the  powerful  capabilities  of  the  DDN.  Daily  virtual  guest  speakers  who  are  experts  in  the 
field  of  distance  education  will  be  brought  to  all  the  academy  locations  via  videoconferencing  on  the  network. 

Recognizing  that  others  across  the  nation  have  developed  expertise  in  distance  education,  Governor 
Janklow  wanted  to  capitalize  on  those  learnings.  During  July  of  2000  the  Governor  convened  his  first  “Governor 
Janklow’s  Capital  City  Conclave  on  Distance  Education”  for  which  he  invited  12  national  leaders  in  distance 
education  to  the  state  capitol  for  two  days  to  discuss  the  issues  and  potential  of  distance  education  for  a rural  state 
like  South  Dakota.  (See  accompanying  Conclave  article  for  agenda  and  participant  information.)  United  State’s 
Senator  Tom  Daschle  was  a virtual  guest  of  the  Conclave,  addressing  participants  via  videoconference  from 
Washington  D.C. 

In  addition  to  the  national  guests,  the  Governor  invited  40  state  leaders  to  be  a part  of  the  Conclave 
discussion.  It  was  important  that  leaders  from  various  state  constituencies  understand  the  advantages  and  potential  of 
a statewide  videoconferencing  network.  The  Governor  gathered  input  from  these  ambassadors  on  possible  next  steps 
for  the  state.  Follow-up  with  these  ambassadors  is  planned. 

Recognizing  that  the  DDN  will  cause  a greater  demand  for  technically  skilled  people  to  maintain  the 
network,  the  Governor  has  campaigned  heavily  to  make  Cisco  Networking  Academies  available  in  most  school 
districts.  During  the  first  year  of  this  effort,  32  Cisco  labs  have  begun  to  offer  networking  coursework  to  junior  and 
senior  high  school  students.  Funding  for  some  of  these  labs  has  been  made  possible  through  the  State’s  Department 
of  Labor.  Additional  school  districts  are  being  encouraged  to  apply  for  funding  to  obtain  their  own  Cisco  labs 
building  capacity  throughout  the  state. 

Through  the  efforts  of  many,  led  by  a very  committed  Governor,  South  Dakota  is  in  a position  to  provide 
distance  education  opportunities  to  every  community  and  citizen  in  the  state.  The  robust  Internet  connections  and 
video  conferencing  capabilities  provided  by  the  DDN  make  many  distance  learning  options  available  to  South 
Dakota  learners.  Quality  professional  development  and  technical  assistance  to  schools  will  further  encourage  the 
appropriate  and  effective  uses  of  the  DDN  to  expand  learning  opportunities  across  the  state. 

Evaluation:  Distance  Education  and  South  Dakota 

“In  order  to  plow  straight  rows,  the  farmer  does  not  look  down  at  the  ground  but  at  the  end  of  the  field.” 

One  major  component  of  the  efforts  to  promote  the  use  of  technology  and  distance  education  in  South 
Dakota  and  specifically  of  Phase  III  of  the  Connecting  the  Schools  Proyec/ was  a comprehensive  evaluation  activity. 
The  process  of  evaluation  included  active  participation  in  the  project  by  evaluators,  collection  of  quantitative  and 
qualitative  data  and  submission  of  several  comprehensive  evaluation  reports.  A baseline  report  was  completed  and 
published  in  March  of  2000.  This  baseline  report  established  a foundation  for  subsequent  evaluation  activities.  A 
second  report  included  data  collected  since  the  baseline  report. 

The  overall  evaluation  plan  was  built  around  the  AEIOU  approach  (Fortune  & Keith,  1992;  Sweeney  1995; 
Sorensen,  1996).  The  effectiveness  of  this  approach  has  been  demonstrated  during  its  use  evaluating  the  activities  of 
the  Iowa  Distance  Education  Alliance,  Iowa’s  Star  Schools  Project  (Simonson,  1995;  Sorensen  & Sweeney,  1994),  a 
multi-year  state-wide  distance  education  activity.  Additionally,  the  model  has  been  used  to  evaluate  a number  of 
other  innovative  projects  such  as  the  Iowa  Chemistry  Education  Alliance  (1995),  the  Iowa  General  Chemistry 
Network  (1994),  and  the  DaVinci  Project:  Interactive  Multimedia  for  Art  and  Chemistry  (Simonson  & Schlosser, 
1995). 

The  AEIOU  evaluation  process  provides  a framework  for  identifying  key  questions  related  to  the  project’s 
implementation.  The  AEIOU  model  is  a dynamic  one  that  permits  the  professional  to  tailor  the  process  of 
evaluation  to  the  specific  situation  being  studied.  This  approach  has  five  components  that  permit  examination  of  the 
Phase  III  of  the  Connecting  the  Schools  Project  from  a number  of  different  perspectives. 
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Component  1 - Accountability  - Did  the  project  planners  do  what  they  said  they  were  going  to  do? 

This  is  the  first  step  in  determining  the  effectiveness  of  the  project  and  is  targeted  at  determining  if  the 
project’s  objectives  and  activities  were  completed.  Evaluation  questions  center  on  the  completion  of  specific 
activities.  Additionally,  counts  of  numbers  of  people,  things,  and  activities  are  collected. 

Methods  Used:  Accountability  information  was  collected  from  project  administrative  records.  Project 
leaders  were  asked  to  provide  documentation  of  the  level  of  completion  of  each  of  the  projects  goals,  objectives,  and 
activities. 

Component  2 - Effectiveness  - How  well  done  was  the  project? 

This  component  of  the  evaluation  process  attempts  to  place  some  value  on  the  project's  activities. 
Effectiveness  questions  focused  on  participant  attitudes  and  knowledge.  Evaluations  were  used  to  collect  reactions 
from  participants  of  workshops,  academies,  and  other  project  activities. 

Methods  Used:  Standardized  measures  are  used  to  determine  program  effectiveness.  Teachers  are  asked 
questions  related  perceptions  about  the  appropriateness  of  the  CTS  Project.  Focus  groups  were  conducted  and 
participants  were  systematically  asked  to  respond  to  questions  about  the  project. 

Component  3 - Impact  - Did  the  project  make  a difference? 

During  this  phase  of  the  evaluation,  questions  focused  on  identifying  the  changes  that  resulted  from  the 
project's  activities,  and  were  tied  to  the  stated  outcomes  of  the  project.  In  other  words,  if  the  project  had  not 
happened  what  of  importance  would  not  have  occurred?  A key  element  will  be  the  collection  of  longitudinal  data  at 
the  beginning,  middle,  and  end  of  the  project. 

Impact  is  extremely  difficult  to  determine  because  determinants  of  impact  vary.  Data  were  collected  at  the 
beginning  of  the  project,  during  its  implementation,  and  at  the  end  of  the  first  full  year  of  activity. 

Methods  Used:  Qualitative  measures  such  as  interviews,  focus  groups,  and  direct  observations  will  be  used 
to  identify  the  project’s  impact. 

Component  4 - Organizational  Context  - What  structures,  policies,  or  events  in  the  organization  or 
environment  helped  or  hindered  the  project  in  accomplishing  its  goals? 

The  focus  of  this  component  of  the  evaluation  was  on  identifying  those  contextual  or  environmental  factors 
that  contributed  to,  or  detracted  from,  the  project. 

Methods  Used:  Organizational  context  evaluation  used  interviews  of  key  personnel,  focus  groups  made  up 
of  those  impacted  by  the  program,  and  document  analysis  that  identified  policies  and  procedures  that  influenced  the 
program.  Direct  participation  in  program  actives  by  the  evaluator  also  permitted  direct  observation  of  events. 

Component  5 - Unanticipated  Consequences  - What  changes  or  consequences  of  importance  happened  as 
a result  of  the  project  that  were  not  expected? 

This  component  of  the  AEIOU  approach  identifies  unexpected  changes  of  either  a positive  or  negative 
nature  that  occurred  as  a direct  or  indirect  result  of  the  project.  Unanticipated  consequences  are  a rich  source  of 
information  about  why  some  projects  are  successful  and  others  are  not.  Central  to  the  measurement  of  unanticipated 
outcomes  is  the  collection  of  ex  post  facto  data. 

Methods  Used:  Interviews,  focus  groups,  journals,  and  surveys  that  asked  for  narrative  information  were 
used  to  identify  interesting  and  potentially  important  consequences  of  implementing  the  CTS  Project.  Evaluators 
interacted  with  project  participants  on  a regular  basis  to  learn  about  the  little  successes  and  failures  that  less  sensitive 
procedures  overlook.  Active  and  continuous  involvement  by  evaluators  permitted  them  to  learn  about  the  project  as 
it  occurs 

DIFFUSION  OF  INNOVATIONS 

Distance  education  is  a new  idea  in  South  Dakota.  More  accurately,  distance  education  is  an  innovation  in 
South  Dakota.  Distance  education  is  defined  as: 

Institution-based  formal  education  where  the  learning  group  is  separated  and  where  telecommunications 
technologies  are  used  to  connect  learners,  resources,  and  instructors  (Simonson,  et.  al.,  2000,  p.  10) 

Specifically,  in  South  Dakota  distance  education  uses  a technology--  compressed  video  - that  permits  two 
or  more  sites  to  connect  to  one  another  for  the  synchronous  sharing  of  video  and  audio.  Compressed  video  is  a 
television  technology  that  has  traditionally  been  used  in  corporate  training,  but  increasingly  is  being  used  in  K-12 
education.  Live,  two-way  video  based  instruction  is  a main  strength  of  compressed  video.  In  2000,  compressed 
video  is  considered  by  most  to  be  an  innovation. 

An  innovation  is  an  idea,  practice  or  object  that  is  perceived  as  new.  Innovations  are  introduced  into 
organizations  and  either  are  adopted  or  rejected  (Rogers,  1995).  This  process  is  called  diffusion.  Diffusion  of  an 
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innovation  is  the  process  of  communication  through  certain  channels  overtime  among  the  members  of  a social 
system.  There  are  four  main  elements  of  diffusion: 

Innovation 

An  idea,  practice  or  object  that  is  perceived  as  new  by  an  individual  or  other  unit  of  adoption. 
Communication  Channels 

Process  by  which  participants  create  and  share  information  with  on  another  in  order  to  reach  a mutual 
understanding. 

Time 


Dimension  in  the  innovation-decision  process  by  which  an  individual  passes  from  first  knowledge  of  an 
innovation  through  its  adoption  or  rejection. 

Relative  earliness/lateness  with  which  an  innovation  is  adopted. 

Rate  of  adoption  - number  of  members  of  the  system  that  adopt  the  innovation  in  a given  time  period. 
Social  system 

A set  of  interrelated  units  that  are  engaged  in  joint  problem  solving  to  accomplish  a common  goal. 

With  a technology  innovation,  such  as  distance  education,  there  are  two  components:  (1)  Hardware: 
defined  as  the  tool  that  embodies  the  technology  as  a physical  object  and,  (2)  software,  consisting  of  the  knowledge 
base  for  the  tool.  The  characteristics  of  an  innovation,  as  perceived  by  the  members  of  a social  system,  determine  its 
rate  of  adoption  (Rogers,  1995).  The  five  attributes  of  an  innovation  are: 

1.  Relative  advantage  is  the  degree  to  which  an  innovation  is  perceived  as  better  than  the  idea  it 
supersedes.  The  degree  of  relative  advantage  may  be  measured  in  economic  terms,  but  social 
prestige,  convenience,  and  satisfaction  are  also  important  factors.  It  does  not  matter  so  much  if  an 
innovation  has  a great  deal  of  objective  advantage.  What  does  matter  is  whether  an  individual 
perceives  the  innovation  as  advantageous.  The  greater  the  perceived  relative  advantage  of  an 
innovation,  the  more  rapid  its  rate  of  adoption  will  be. 

2.  Compatibility  is  the  degree  to  which  an  innovation  is  perceived  as  being  consistent  with  the  existing 
values,  past  experiences,  and  needs  of  potential  adopters.  An  idea  that  is  incompatible  with  the 
values  and  norms  of  a social  system  will  not  be  adopted  as  rapidly  as  an  innovation  that  is 
compatible.  The  adoption  of  an  incompatible  innovation  often  requires  the  prior  adoption  of  a new 
value  system,  and  this  is  a relatively  slow  process. 

3.  Complexity  is  the  degree  to  which  an  innovation  is  perceived  as  difficult  to  understand  and  use. 

Most  members  of  a social  system  readily  understand  some  innovations;  others  are  more 
complicated  and  will  be  adopted  more  slowly.  New  ideas  that  are  simpler  to  understand  are  adopted 
more  rapidly  than  innovations  that  require  the  adopter  to  develop  new  skills  and  understandings. 

4.  Trialability  is  the  degree  to  which  an  innovation  may  be  experimented  with  on  a limited  basis.  New 
ideas  that  can  be  tried  on  the  installment  plan  will  generally  be  adopted  more  quickly  than 
innovations  that  are  not  divisible.  Ryan  and  Gross  (1943)  found  that  every  one  of  their  Iowa  farmer 
respondents  adopted  hybrid  seed  corn  by  first  trying  it  on  a partial  basis.  If  the  new  seed  could  not 
have  been  sampled  experimentally,  its  rate  of  adoption  would  have  been  much  slower.  An 
innovation  that  is  trialable  represents  less  uncertainty  to  the  individual  who  is  considering  it  for 
adoption,  as  it  is  possible  to  learn  by  doing. 

5.  Observability  is  the  degree  to  which  the  results  of  an  innovation  are  visible  to  others.  The  easier  it 
is  for  individuals  to  see  the  results  of  an  innovation,  the  more  likely  they  are  to  adopt  it.  Such 
visibility  stimulates  peer  discussion  of  a new  idea,  as  friends  and  neighbors  of  an  adopter  often 
request  innovation-evaluation  information  about  it.  Solar  panel  adopters  often  are  found  in 
neighborhoods  in  California,  with  three  or  four  adopters  located  on  the  same  block.  Other  consumer 
innovations  like  home  computers  are  relatively  less  observable,  and  thus  diffuse  more  slowly. 

In  South  Dakota,  distance  education  using  compressed  video  is  an  innovation  that  is  being  introduced  into 
the  state.  The  success  or  failure  of  this  innovation  will  largely  depend  on  the  process  of  diffusion  that  occurs. 
Diffusion  of  innovations  sometimes  occurs  naturally,  without  the  need  for  outside  intervention.  Often  however, 
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diffusion  is  managed  and  facilitated  by  change  agents  who  involve  opinion  leaders  who  provide  the  necessary 
impetus  for  an  innovation  to  reach  what  is  called  critical  mass.  Critical  mass  is  the  point  at  which  an  innovation 
spreads  and  expands  without  the  need  for  outside  intervention. 

In  South  Dakota,  the  change  agents  who  are  guiding  the  spread  of  distance  education  are  a group  of 
educational  leaders  and  state  department  staff.  This  group  identified  opinion  leaders  and  trained  and  oriented  them 
through  the  use  of  workshops  and  academies.  The  group  of  opinion  leaders  has  and  will  provide  instruction  and 
support  to  South  Dakota  teachers  who  in  turn  will  guide  children  and  young  adults  to  successfully  use  the  innovation 
of  distance  education. 

In  summary,  the  diffusion  of  distance  education  in  South  Dakota  schools  is  a process  involving  an 
innovation  (DDN;  compressed  video),  using  communication  channels  (Workshops  and  Distance  Teaching  and 
Learning  Academies),  over  time,  within  a social  system  (the  educational  community  of  South  Dakota).  The 
evaluation  activities  described  next  were  conducted  to  provide  educational  leaders  (change  agents)  with  information 
to  assist  them  in  reaching  critical  mass,  the  point  at  which  distance  education  is  widely  accepted. 

EVALUATION  ACTIVITIES  AND  RESULTS 

The  goal  of  this  evaluation  effort  is  to  provide  leaders  in  the  state  of  South  Dakota  with  quantitative  and 
qualitative  evaluation  information  about  the  implementation  of  the  distance  learning  infrastructure  phase  of  the 
Connecting  the  Schools  project.  Evaluation  focused  on  the  analysis  of  the  relationship  between  the  distance 
education  training  and  diffusion  of  the  distance  education  innovation  into  the  K-12  environment.  Factors  that  were 
of  special  interest  to  the  evaluation  included  information  about  the  innovativeness  of  those  individuals  involved  in 
distance  education  in  South  Dakota,  and  the  innovativeness  of  educational  organizations  in  the  state.  This 
information  helped  to  explain  why  educators  and  organizations  were  accepting  or  rejecting  distance  education,  and 
provided  insight  into  how  the  process  of  diffusion  of  this  innovation  can  be  facilitated. 

Evaluation  Activities 

The  following  activities  were  part  of  the  South  Dakota  evaluation  plan.  First,  a standardized  questionnaire, 
called  the  Connecting  the  Schools  Questionnaire  (CSQ)  was  systematically  developed.  This  questionnaire  included 
four  sections  - demographic  information,  a measure  of  personal  innovativeness,  and  measure  of  organizational 
innovativeness,  and  questions  related  to  distance  education. 

Second,  the  CSQ  was  administered  to  participants  in  six  workshops  for  opinion  leaders  that  were  held  for 
educators  to  provide  them  with  skills  and  knowledge  related  to  distance  education.  Staff  from  the  Department  of 
Education  and  Cultural  Affairs  (DECA)  conducted  these  workshops.  In  addition  to  completing  the  CSQ,  participants 
at  these  workshops  had  the  opportunity  to  respond  to  open-ended  questions.  These  questions  were  included  in  the 
evaluation  to  give  workshop  participants  the  opportunity  to  express  their  opinions  about  distance  education. 

Third,  participants  at  Technology  Teaching  and  Learning  (TTL)  Academies  and  Distance  Teaching  and 
Learning  (DTL)  Academies  were  asked  to  complete  the  CSQ.  For  these  groups  the  CSQ  was  made  available  in  an 
on-line  version,  so  they  could  respond  via  the  Internet.  The  participants  at  the  TTL  and  DTL  Academies  also  had  the 
opportunity  to  give  open-ended  statements.  Additionally,  a modified  focus  group  activity  was  conducted  with 
participants  of  one  of  the  DTL  academies.  Attendees  were  asked  to  respond  to  three  questions  about  distance 
education  in  the  state. 

Finally,  a random  selection  of  South  Dakota  teachers  was  sent  the  CSQ.  Their  responses  provide  a basis  of 
comparison.  They  represent  the  “average”  South  Dakota  educator.  This  group  also  provided  open-ended  comments 
about  technology  and  distance  education. 

Conclusions 

Based  on  the  results  collected  from  the  CSQ  and  from  focus  group  activities  the  following  conclusions 
about  South  Dakota’s  Connecting  the  Schools  - Phase  III  Project  can  be  made. 

Component  1 : Accountability  - Did  the  project  planners  do  what  they  said  they  were  going  to  do? 

It  is  obvious  from  data  collected,  meetings  attended,  and  reports  submitted  that  the  Connecting  the  Schools 
Project  - Phase  III  is  being  conducted  effectively.  Distance  Teaching  and  Learning  (DTL)  Academies  were  held 
during  the  summer,  as  were  six  specialized  training  sessions  for  opinion  leaders.  Additionally,  a number  of 
meetings,  short  sessions,  and  orientations  were  held,  all  designed  to  explain  the  potential  of  distance  education  and 
the  utilization  of  the  Digital  Dakota  Network. 
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Component  2 - Effectiveness  - Will  the  project  be  done  well? 

The  effectiveness  of  the  project  was  measured  by  the  responses  from  teachers,  administrators,  network 
specialists,  grant  administrators  and  DECA  employees  to  the  training  offered,  utilizing  standardized  measures. 
Questionnaires  were  given  to  participants  in  workshops,  and  focus  groups  were  conducted  with  participants. 

It  is  apparent  from  observations  made,  data  collected,  and  discussions  held  that  the  initial  activities  of  the 
CTS- Phase  III  project  were  considered  to  be  effective  by  participants  and  leaders.  It  is  obvious  that  the  vast 
majority  of  those  involved  believed  that  what  they  were  participating  in  was  effective.  There  has  been  an  interesting 
lack  of  criticism  of  the  CTS  project^  even  though  there  have  been  numerous  concerns  expressed  how  distance 
educadon  will  impact  on  the  South  Dakota  educational  community  and  how  overworked  educators  will  be  able  to 
effectively  adopt  this  innovation.  Apparently,  most  think  the  CTS  is  an  effective  plan  to  alleviate  concerns  about 
distance  education. 

Component  3 — Impact  — Will  the  project  make  a difference? 

By  far  the  most  difficult  evaluation  concept  to  ascertain,  especially  in  the  short  term,  is  the  impact  of  any 
project,  program,  or  innovation.  Impact  can  be  determined  when  baseline  data  are  compared  to  data  collected  in  a 
year,  three  years,  and  five  years.  Data  collected  and  reported  early  in  a project  such  as  this  one  establish  a 
framework  for  determining  the  intermediate  and  long-term  impact  of  distance  education  generally,  and  more 
specifically,  the  DDN  with  its  compressed  video  classrooms.  Of  critical  importance  to  determining  impact  is  the 
evaluation  of  the  diffusion  process  followed  by  the  educational  leaders  who  manage  distance  education  in  the  state. 

It  appears  that  the  impact  of  the  Connecting  the  Schools  Project  - Phase  III  has  been  positive.  There  is  a 
notable  lack  criticism  of  the  project;  most  likely  because  of  the  carefully  planned,  systematic  process  followed  by 
project  leaders.  It  is  important  to  call  attention  to  the  success  of  process  used  to  diffuse  distance  education  into  the 
educational  system  of  South  Dakota.  First,  opinion  leaders  were  identified  and  specialized  training  in  workshops 
was  developed  for  them.  Second,  a large  number  of  teachers  were  invited  to  the  DTL  Academies  where  they 
participated  in  instruction  that  was  comprehensive,  lengthy,  and  conducted  by  opinion  leaders.  DTL  Academy 
graduates  became  highly  knowledgeable  and  many  became  committed  to  the  potential  positive  impact  of  distance 
education.  Finally,  stakeholders  outside  of  the  educational  community  were  targeted  for  information  about  distance 
education  and  the  DDN.  These  efforts  broadened  understanding  in  the  general  community  and  helped  diffuse  the 
innovation. 

Component  4 - Organizational  Context  - are  the  structures,  policies,  or  events  in  place  that  will  help  the 
project  in  accomplishing  its  goals? 

Preliminary  evaluation  results  clearly  show  that  the  concept  of  organizational  context  is  critical  to  the 
success  of  the  CTS -Phase  III  project.  Specifically,  these  organizational  topics  have  been  identified: 

Teachers  must  have  staff  development  to  overcome  their  fear  of  the  innovation. 

Teachers  need  continued  support  after  staff  development. 

Educational  policies  that  guide  school  policy  will  need  to  be  studied  and  changed  where  necessary. 

Teacher  (and  by  implication,  parent  and  student)  fears  and  concerns  will  need  to  be  addressed  in  a 
systematic  and  planned  manner  in  order  for  distance  education  and  the  DDN  to  be  widely  accepted. 

Information  collected  in  focus  group  sessions  provided  considerable  information  concerning  organizational 
context.  First,  there  was  reported  a high  level  of  general  support  for  distance  education,  specifically  because  of  the 
potential  for  curriculum  enhancement.  Second,  many  South  Dakota  educators  said  they  were  fearful  of  the 
consequences  of  distance  education.  Many  groups  reported  these  fears  over  a number  of  focus  group  activities.  The 
fear  or  apprehension  felt  by  those  who  must  ultimately  adopt  an  innovation  such  as  distance  education  must  be  dealt 
with.  It  appears  that  the  lengthy  training  provided  in  the  DTL  Academies  is  an  excellent  technique  that  alleviates 
apprehension.  Continued  follow-up  activities  with  educators  are  necessary. 

Component  5 - Unanticipated  Consequences 

A number  of  unanticipated  activities  occurred  - the  most  significant  was  a special  meeting  held  in  Pierre, 

SD  in  July.  This  meeting.  Governor  Janklow’s  Capital  City  Conclave  on  Distance  Education,  brought  a dozen 
national  and  international  experts  in  distance  education  to  South  Dakota  to  interact  with  state  leaders.  Also,  there 
seems  to  be  a level  of  leadership  being  provided  by  staff  (change  agents)  from  the  Department  of  Education  and 
Cultural  Affairs  (DECA)  that  was  not  apparent  when  this  evaluation  began.  This  leadership  seems  to  transcend  the 
CTS  Project.  While  leadership  is  difficult  to  document,  many  comments  from  South  Dakota  educators  summarized 
in  this  report  refer  to  the  positive  influence  of  DECA  staff 

It  is  obvious  that  the  CTS -Phase  III  project’s  activities  have  been  accountable,  and  that  early  indications 
are  that  they  have  been  effective  in  meeting  stated  objectives.  The  impact  of  the  project’s  initial  activities  will  be 
determined  over  time.  Baseline  information  has  been  collected  and  will  permit  a clearer  determination  of  impact  in 
subsequent  reports  and  in  future  years. 
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It  is  clear  from  information  collected  that  the  organizational  context  in  which  distance  education  exists  in 
South  Dakota  is  evolving,  as  it  should,  and  that  additional  changes  will  need  to  be  considered.  Finally,  it  seems  that 
a number  of  new  and  potentially  important  activities  are  being  considered  that  were  not  anticipated  when  the  CTS 
Project 'WdiS  planned.  This  is  to  be  expected  and  is  considered  by  most  evaluators  as  an  indication  of  effectiveness. 

Significant  suggestions  identified  as  a result  of  evaluation  of  the  diffusion  of  distance  education  in  South 
Dakota  include  the  following: 

• Many  of  the  participants  of  the  six  special  workshops  held  during  the  spring  and  summer  of  2000 
were  selected  because  they  were  considered  to  be  opinion  leaders.  The  data  support  that  many 
opinion  leaders  did  attend  these  workshops. 

• A large  number  of  the  participants  of  the  Distance  Teaching  and  Learning  (DTL)  and  Technology 
for  Teaching  and  Learning  (TTL)  Academies  had  relatively  low  levels  of  personal  innovativeness. 

• Classroom  teachers  are  the  group  most  emulated  by  workshop  attendees,  TTL/DTL  attendees  and 
South  Dakota  teachers,  in  general.  Classroom  teachers  are  a group  that  increasingly  should  be  used 
as  opinion  leaders.  This  will  speed  the  diffusion  of  distance  education  in  South  Dakota. 

• The  process  of  using  change  agents  to  orient  opinion  leaders  who  in  turn  work  with  teachers  and 
other  adopters  is  a good  one  that  seems  to  be  working  effectively  in  the  state. 

• Trialability  and  observability  have  been  recognized  as  critical  to  the  adoption  of  distance 
education  in  South  Dakota.  Trialability  and  observability  opportunities  should  be  increased  in 
number  and  location  to  speed  the  adoption  of  distance  education. 

• An  increased  number  of  TTL/DTL  attendees  are  likely  to  have  relatively  low  levels  of  personal 
innovativeness  indicating  a reluctance  to  adopt  distance  education.  A different  strategy  for 
providing  staff  development  to  these  educators  should  be  considered.  Specifically,  individuals 
with  strong  reluctance  to  adopt  an  innovation  can  not  be  easily  influenced  by  normal  diffusion 
strategies  (Rogers,  1995).  Rather,  most  efforts  should  be  directed  at  educators  with  higher  levels 
of  innovativeness  towards  distance  education.  This  is  not  to  suggest  that  those  with  low  levels  of 
innovativeness  should  be  ignored.  Instead,  efforts  should  be  focused  on  other,  more  innovative 
educators. 

• According  to  the  data  collected,  younger,  highly  educated  South  Dakota  teachers  are  a group  that 
should  be  used  to  provide  leadership  for  the  adoption  of  distance  education  in  the  state. 

• Continued  support  of  teachers  who  adopt  or  are  considering  the  adoption  of  distance  education 
after  the  DTL  Academy  experience  should  be  available.  Mini-grants,  short  workshops,  visits  by 
Department  of  Education  and  Cultural  Affairs  staff,  and  publicity  for  projects  are  examples  of 
ways  to  provide  continued  support  for  educators  who  are  using  the  DDN  and  distance  education. 

In  summary,  the  strategy  developed  to  diffuse  distance  education  into  the  educational  community  of  South 
Dakota  appears  to  be  an  approach  to  be  modeled  by  other  states  and  regions  that  are  interested  in  developing  a large 
distance  education  system. 

First,  change  agents  working  for  the  Department  of  Education  and  Cultural  Affairs  were  used  to  provide 
training  and  support  for  educators  emulated  by  their  peers.  These  groups  received  special  attention  in  workshops 
designed  expressly  for  them.  These  educators  were  considered  to  be  opinion  leaders;  individuals  looked  to  by  their 
peers  as  persons  to  respect.  Next,  a large  number  of  educators  were  given  in-depth  staff  development  in  lengthy 
Distance  Teaching  and  Learning  Academies.  Finally,  the  trained  educators  returned  to  their  local  schools  and 
communities  where  they  were  supported  by  staff  from  the  state  department  of  education  in  efforts  to  use  the  Digital 
Dakota  Network  for  distance  education.  The  process  of  diffusion  of  the  innovation  of  distance  education  was 
research  and  theory  based  (Rogers,  1995),  evaluated  systematically  (AEIOU  Approach),  and  dynamic  so  changes 
could  be  quickly  implemented  to  respond  to  unanticipated  events.  In  South  Dakota,  Daniel  Burnham's  famous 
statement  is  the  watchword. 

“Make  no  little  plans;  they  have  no  magic  to  stir  the  blood!” 
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Abstract 


At  the  University  ofTwente  a new  curriculum  on  educational  science  and  technology  has  been  introduced. 
That  occasion  was  used  to  try  to  develop  an  apprenticeship  model  in  which  the  students  are  regarded  as 
young  professionals  from  the  very  beginning.  In  that  model  the  students  are  expected  to  govern  their 
professional  growth  by  actively  collecting  evidence  of  acquired  competencies  in  electronic  portfolios.  This 
activity  should  stimulate  teachers  to  adapt  their  teaching  style  to  the  requests  from  students  for  feedback  on 
products  that  the  students  would  like  to  put  into  their  portfolios.  After  three  iterations  of  development  in 
three  consecutive  academic  years,  however,  the  use  of  portfolios  is  still  not  successful.  The  reasons  why  are 
discussed  and  steps  to  be  taken  are  suggested. 


The  Faculty  of  Educational  Science  and  Technology  at  the  University  of  Twente  in  the  Netherlands  is 
educating  educational  designers  with  specializations  in  seven  directions:  curriculum  development,  instructional 
technology,  instrumentation  (media,  computers,  and  internet  in  education),  educational  organization  and 
management.  Human  Resources  Development  (HRD),  educational  testing,  and  social  science  research.  The  Faculty 
intends  to  switch  to  a competency-oriented  curriculum  for  its  three-year  Bachelor  program. 

The  term  'competency'  is  partly  to  be  understood  in  terms  of  the  knowledge  and  skills  that  comprise  the 
competency  profile  of  a profession  and  partly  as  the  ability  “...to  operate  in  ill-defined  and  ever-changing 
environments,  to  deal  with  non-routine  and  abstract  work  processes,  to  handle  decisions  and  responsibilities,  to  work 
in  groups,  to  understand  dynamic  systems,  and  to  operate  within  expanding  geographical  and  time  horizons”  (Keen, 
1992).  Knowledge  and  skills  can  be  assessed  in  traditional  ways.  The  kind  of  behavior  as  described  by  Keen  is  more 
difficult  to  observe.  This  at  the  same  time  causes  a problem  in  terms  of  the  extent  to  which  competency  in  Keen's 
terms  should  dominate  the  whole  philosophy  and  organization  of  the  program.  To  get  around  this  problem, 
competency  development  was  seen  as  an  activity  that  can  take  place  in  concert  with  doing  traditional  coursework,  as 
long  as  the  students  would  be  able  to  handle  the  related  learning  experiences  from  a meta-cognitive  level  that 
complies  with  Keen's  definition  of  competent  behavior. 

The  use  of  portfolios  offers  a possibility  to  build  evidence  of  the  development  of  competencies.  The  main 
attraction  of  portfolios  is  formed  by  its  potential  to  assess  progress  and  process  as  essentials  for  learning  (Saywer, 
1994).  In  the  new  curriculum  portfolios  are  introduced  for  self-assessment  by  the  students  as  well  as  for  monitoring 
their  progress. 

Starting  in  the  academic  year  1999,  electronic  portfolios  were  introduced  to  provide  students  with  a tool  for 
actively  working  on  competency  development.  The  introduction  was  monitored  and  the  use  of  the  portfolios  was 
developed  in  a few  steps  aiming  at  answering  the  following  question: 

"Do  student  portfolios  yield  sufficient  support  to  help  students  to  direct  their  own  development  in  a 

competency-oriented  curriculum,  and  if  so,  under  what  conditions?" 

The  literature  shows  growing  consensus  that  educational  reform  efforts  are  doomed  to  fail  unless  the 
teachers'  cognitions,  including  their  beliefs,  intentions,  and  attitudes,  are  taken  into  account  (Haney,  Czemiak,  & 
Lumpe,  1996).  The  introduction  of  student  portfolios  is  not  only  planned  to  serve  the  purpose  of  the  students,  but 
also  as  a tool  for  the  implementation  of  the  new  curriculum.  The  latter  relates  to  the  readiness  and  willingness  of 
teachers  to  adopt  a new  approach,  for  which  the  following  question  should  be  answered: 

"Does  student-directed  collection  of  evidence  of  professional  growth  influence  teaching  style  of  academic 
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staff?" 


The  educational  concept  ''initiation  in  the  academic  profession" 

The  new  curriculum  approach  is  based  on  a concept  that  is  labeled:  "initiation  in  the  academic  profession" 
(Verhagen,  2000).  It  is  an  apprenticeship  model  in  which  the  students  who  enter  the  university  directly  from 
secondary  school  are  regarded  as  young  colleagues  from  the  very  beginning.  A central  principle  of  the  concept  is 
that  the  interaction  between  students  and  teachers  should  take  place  in  a professional  context  as  much  as  possible  in 
stead  of  interaction  in  an  instructional  context.  Assignments  should  as  much  as  possible  mirror  professional  practice 
to  help  students  to  develop  professional  behavior.  The  result  of  a literature  assignment  should  take  the  form  of  the 
literature  part  of  a scientific  article;  communication  products  of  design  assignments  such  as  proposals,  budget 
estimates,  blueprints  or  evaluation  reports  should  take  a form  that  would  be  appropriate  to  present  to  clients;  and  so 
forth.  Teachers  should  primarily  be  regarded  as  experts  who  enable  the  development  of  academic  insights  by 
scientific  discourse. 

To  make  this  work,  a new  approach  of  mentorship  has  been  introduced  together  with  the  new  curriculum. 
Each  student  becomes  a member  of  a mentor  group  that  is  chaired  by  a staff  member  of  the  faculty.  A mentor  group 
consists  of  about  1 2 students  from  all  three  years  of  the  Bachelor  program,  about  four  students  from  each  generation. 
The  mentor  groups  provide  a social  structure  in  which  the  master  (the  mentor/staff  member)  and  the  experienced 
students  from  the  second  and  third  year  help  the  first  year  students  (the  learners)  to  be  initiated  in  the  professional 
culture  of  academic  professionals.  In  these  groups  the  discussion  of  personal  development  towards  becoming  a 
competent  professional  is  a standing  issue.  The  group  members  are  required  to  collect  evidence  of  professional 
growth  that  lends  itself  for  discussion  with  the  mentor  and  in  the  mentor  groups. 

The  students  collect  the  evidence  of  their  professional  development  in  their  electronic  portfolios.  Each 
portfolio  consists  of  four  parts:  (a)  An  introduction  of  the  owner  (student),  (b)  a text  based  curriculum  vitae,  where 
the  student  is  expected  to  put  personal  information,  information  about  his  or  her  school  career,  information 
concerning  professional  development  in  or  outside  of  the  official  program  (like  having  a job  in  the  field)  and  other 
information  such  as  involvement  in  sports  and  hobbies;  (c)  an  archive,  where  the  student  puts  evidence  of  his  or  her 
professional  development  during  the  study;  and  (d)  a showcase  where  the  student  chooses  to  present  a selection  of 
his  or  her  best  work.  The  archive  is  the  central  tool  for  the  student  to  perform  self-assessments  about  specific 
accomplishments  as  well  as  to  reflect  on  personal  development  in  general.  Self  assessment  is  considered  to  be  a 
form  of  metacognition  that  is  essential  for  self-regulation  (Simon  & Forgette-Giroux,  2000). 

Additional  features  of  the  new  curriculum  are  that  the  development  of  information  and  communication 
skills  is  integrated  into  courses  and  that  all  courses  are  organized  using  a Web-based  course  management  system 
(TeleTOP,  a home-made  Lotus  Domino  application). 

The  implementation  of  the  new  approach  requires  a substantial  change  of  teacher  behavior.  Most  teachers 
are  used  to  teacher-controlled  instructional  formats.  Some  teachers,  however,  share  the  philosophy  of  the  new 
curriculum  concept.  They  should  act  as  the  pioneers  and  early  adopters  who  provide  a critical  mass  of  authentic 
professional  tasks  that  allow  student  to  develop  the  necessary  skills  for  self-regulation  of  their  academic  education. 
These  tasks  and  the  interaction  in  the  mentor  groups  are  expected  to  shape  the  attitude  and  abilities  of  the  student 
into  characteristics  of  self-reliant  young  professionals.  The  extent  to  which  this  will  appear  to  be  true  will  answer  the 
first  research  question. 

Approaching  the  students  in  such  a way  that  they  perceive  the  need  to  adopt  a professional  attitude  is 
e^q^ected  to  cause  students  to  work  conscientiously  on  compiling  evidence  of  their  professional  development  in  the 
form  of  portfolio  products.  They  then  will  seek  feedback  of  the  teachers  on  their  work  and  ask  for  comments  on  the 
added  value  of  assignments  for  the  objectives  of  a course  and  in  the  framework  of  the  competency  profile  of  the 
Bachelor  program  as  a whole.  It  is  expected  that  this  behavior  will  influence  the  teaching  style  of  the  academic  staff 
and  will  help  teachers  who  are  reluctant  to  invest  in  the  new  educational  concept  to  move  in  the  desired  direction. 

The  extent  to  which  this  effect  occurs,  will  answer  the  second  research  question. 

The  first  experiences 

Preparing  for  the  academic  year  1999-2000 


The  principles  of  the  new  approach  have  been  presented  to  the  teaching  staff  on  several  occasions,  at  first 
to  estimate  whether  the  concept  was  appealing  to  them.  The  overall  impression  was  sufficiently  positive  to  start  the 
preparation  of  the  introduction  of  the  new  approach  in  the  program.  A few  months  before  the  academic  year  1999- 
2000,  a group  of  student-friendly  staff  members  was  invited  as  mentors.  Together  with  the  faculty  management  they 
developed  procedures  and  a related  manual  to  start  the  new  mentor  groups.  Involving  the  mentors  at  this  s tage 
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resulted  in  their  ownership  of  the  concept  for  the  new  mentor  groups  and  the  way  of  working  in  those  groups. 
Elements  of  the  approach  are  that  competency  development  was  related  to  three  major  roles  of  professionals  in  our 
field:  designer,  researcher,  and  consultant;  and  that  explicit  attention  should  be  paid  to  generic  competencies  such  as 
planning,  self  management,  interpersonal  skills,  communication,  and  academic  reflection.  In  respect  to  the 
individual  development  of  the  students,  a list  of  products  that  should  be  collected  in  the  portfolios  was  specified. 
They  concern  results  from  assignments  in  courses  that  may  be  considered  as  evidence  of  acquired  knowledge  and 
skills,  thus  contributing  to  the  competency  profile  of  the  student.  Monthly  professional  meetings  of  the  mentor 
groups  were  planned  to  discuss  progress.  During  these  meetings  also  attention  was  paid  to  the  quality  of  the  program 
as  experienced  by  the  students.  It  was  expected  that  the  students  in  their  role  as  beginning  educational  designer 
should  be  interested  in  strengths  and  weaknesses  of  the  courses  in  which  they  participate.  The  input  from  the  mentor 
groups  was  also  considered  as  valuable  for  the  formal  evaluation  of  running  courses.  The  teaching  staff  was 
informed  about  the  intended  approach  and  invited  to  work  accordingly. 

Outside  the  mentor  groups,  information  and  communication  specialists  developed  their  curricula  in  close 
cooperation  with  teachers  from  selected  courses  to  arrive  at  the  integration  of  relevant  tasks  and  assignments  in  the 
different  courses. 

Results  from  the  academic  year  1999-2000 

The  mentor  groups  appeared  to  be  handicapped  by  the  fact  that  it  was  the  first  year  and  thus  only  first-year 
students  were  members.  The  monthly  meetings  failed  also  because  the  students  had  such  a close  contact  with  each 
other  throughout  the  week,  that  no  substance  to  discuss  remained  for  the  meetings.  Course  evaluations  became  a 
formal  ritual  with  no  real  impact. 

Teachers  and  students  appeared  to  behave  more  traditional  than  expected,  leading  to  much  interaction  in  an 
instructional  context  and  little  in  a professional  context.  The  instructivistic  teaching  style  in  many  courses  appeared 
a dominant  factor  in  shaping  student  behavior.  In  stead  of  working  on  a professional  attitude  that  complies  with  the 
model  of  the  students  as  young  colleagues,  the  students  felt  that  they  went  to  school  to  take  lessons  and  make  tests. 

Moreover,  due  to  technical  problems  portfolio  software  was  introduced  to  the  students  at  a late  stage  (the 
end  of  the  first  semester).  Students  had  then  to  go  back  to  already  completed  courses  to  find  the  required  products 
for  their  portfolios.  It  was  unlucky  that  for  unclear  reasons  they  appeared  not  to  be  informed  about  the  list  of 
required  products  that  existed  from  the  beginning  of  the  academic  year.  And  when  they  learned  about  the  list, 
several  students  became  annoyed  because  they  consider  that  list  as  contradictory  to  the  concept.  If  the  portfolios  are 
tools  for  governing  one’s  own  learning  process,  they  should  be  able  to  decide  by  themselves  what  to  put  into  it.  At 
that  time,  so  much  was  unclear,  that  most  students  failed  to  work  with  the  portfolios  in  a proper  way.  In  conclusion, 
the  electronic  portfolios  were  hardly  used. 

The  only  thing  that  really  worked  was  the  integration  of  information  and  communication  skills  into  courses. 
The  carefully  developed  set  of  tasks  on  information  and  communication  skills  made  the  students  acquire  the  related 
skills  every  time  they  needed  them  for  the  assignments  in  the  courses.  A literature  assignment  in  a course  on 
pedagogy  was  used  to  teach  them  how  to  find  literature,  an  assignment  to  write  a paper  for  an  other  course  was  used 
to  explicitly  pay  attention  to  writing  skills,  and  so  on. 

2000-2001:  Some  changes 

The  insight  was  developed  that  students  should  not  be  forced  to  put  products  in  a portfolio,  because  this  is 
contradictory  to  self  management.  The  students,  who  were  interested  in  the  desired  approach,  told  that  to  us  more 
than  once  and  they  were  right  at  this  point.  The  mentors  were  asked  to  guide  the  students  in  developing  self- 
management skills,  using  the  electronic  portfolio  as  a discussion  platform. 

Now  that  students  of  two  study  years  were  member  of  the  mentor  groups,  activities  were  specified  that 
could  bring  the  concept  to  life.  The  elderly  students  could  now  introduce  the  new  students  to  all  kinds  of  procedures 
and  habits  in  the  faculty.  And  group  discussion  could  now  aim  at  points  of  interest  for  which  the  vision  of  both  the 
first  year  and  the  second-year  students  was  relevant.  The  number  of  official  meetings,  however,  was  reduced  to 
seven.  This  measure  was  taken  to  avoid  the  problem  of  too  few  subjects  for  discussion  that  came  with  the  monthly 
meetings  in  the  first  year. 

Changing  the  teaching  style  of  teachers  towards  competence  development  seemed  not  to  be  possible 
directly.  So  a major  role  for  the  mentor  was  envisioned  given  the  character  of  the  guild  model  in  the  mentor  groups. 
The  mentors  were  asked  to  work  with  the  students  on  helping  them  to  use  their  portfolios  for  self-governance. 
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Results  from  the  academic  year  2000-2001 

This  time  the  mentors  started  to  resist  to  the  idea  that  they  should  work  with  the  students  in  such  a way  that 
the  students  would  develop  the  met  cognitive  skills  to  monitor  their  own  professional  growth  with  the  electronic 
portfolios  as  the  basic  tool.  They  argued  that  the  curriculum  and  the  way  in  which  the  courses  are  taught,  should 
have  this  effect. 

Again,  the  portfolios  were  hardly  used  although  the  software  was  now  available  almost  from  the  beginning. 
But  the  early  start  had  also  a disadvantage.  The  students  were  introduced  to  the  portfolio  software  in  a workshop 
where  to  technical  skills  were  practiced  without  real  products  to  put  into  the  portfolio.  The  first  products  that  would 
be  suitable  had  still  to  be  produced  in  the  courses  that  just  were  started.  By  the  time  that  the  portfolios  could  be  used, 
most  students  had  forgotten  how  to  do  that.  As  there  was  this  time  no  list  of  required  products,  only  few  students 
appeared  to  motivate  to  start  filling  their  portfolios.  Many  other  students,  however,  appeared  not  to  be  able  to  decide 
what  they  could  put  into  the  portfolio.  Partly  the  reason  is  that  they  appeared  to  be  very  critical  of  their  first-year 
products,  considering  their  own  work  as  real  beginner’s  work  that  is  not  worth  to  be  put  into  a portfolio. 

The  cooperation  between  students  from  different  years  appeared  to  be  one-sided.  Only  in  the  beginning  of 
the  academic  year  were  the  older  students  active  when  introducing  the  new  students.  Subjects  of  mutual  interest  to 
old  and  new  students  were  not  identified.  The  new  students  could  also  not  bring  anything  of  relevance  for  the  older 
student.  The  question:  “What  are  my  benefits?”  was  hard  to  answer  for  the  older  students. 

Still,  the  students  were  positive  about  the  meetings  of  the  mentor  groups.  During  the  meetings  the  students 
discussed  general  information  with  their  mentor  and  with  each  other  and  they  used  the  meetings  as  a platform  to 
complain  about  organizational  or  educational  problems  in  the  faculty.  They  could  speak  freely  about  anything, 
which  gave  the  meetings  an  “I  am  not  alone  with  this”  function. 

Putting  students  in  control 

2001-2002:  A last  chance  for  portfolios? 

Gradually  it  becomes  clear  that  the  basic  philosophy  of  the  concept  “Initiation  in  the  Academic  Profession” 
does  not  really  settle  in  the  faculty.  They  believe  that  the  concept  is  worthwhile,  is  reason  for  a third  attempt.  This  is 
where  the  approach  was  developed  that  is  the  reason  for  this  paper.  Again  the  idea  is  that  the  students  should 
develop  initiative  in  using  their  portfolios  as  a tool  for  collecting  evidence  of  their  professional  growth.  In  the  first 
week  of  the  academic  year,  they  were  trained  to  use  their  portfolios  in  two  ways:  technically  to  learn  how  to  put 
elements  in  the  portfolio  and  how  the  manage  the  portfolio;  and  conceptually  on  how  to  use  the  portfolio  for 
monitoring  and  managing  professional  progress.  An  adapted  guide  for  the  mentor  groups  explained  the  purpose  and 
the  philosophy:  It  was  recognized  that  not  all  students  are  ready  for  this  kind  of  metacognitive  activity.  They  are 
therefore  allowed  to  use  or  not  use  their  portfolios  for  self  management.  Filling  the  archive  with  products  of  courses, 
however,  is  this  time  required  to  maintain  basic  portfolio  skills  until  the  moment  that  the  student  is  ready  and  willing 
for  the  intended  use. 

The  results  so  far 

The  number  of  students  that  works  with  the  portfolios  in  the  intended  way  is  neglectible.  There  are  just  a 
few  students  who  work  with  their  portfolios.  These  students  are  mostly  using  the  archive  function  just  for  their  own 
purposes  and  not  to  reflect  on  what  they  have  done  in  past  periods.  Also  most  mentors  still  don’t  use  the  portfolios 
for  the  individual  meetings  with  the  student.  Some  of  them  do,  but  they  leave  no  room  for  self-directedness  by  the 
students  because  they  require  the  students  to  fill  the  archive. 

In  the  meantime,  the  discussion  about  the  usefulness  of  portfolios  has  become  an  issue  in  a broader 
perspective.  Students  who  are  following  the  old  curriculum  (from  before  the  introduction  of  the  new  educational 
concept)  start  asking  for  their  own  portfolios  for  making  overviews  of  products  that  they  collect  in  courses  during 
their  study.  So  these  oldest  students  see  the  purpose  and  the  advantages  of  portfolios  from  a need  for  systematically 
archiving  products.  Regrettably,  however,  when  providing  them  with  the  portfolio  software,  they  don’t  find  the  time 
to  really  do  it.  It  seems  a similar  phenomenon  as  with  the  staff  members  who  like  the  idea  of  the  new  educational 
concept,  but  do  not  really  change  their  methods  to  comply  with  it. 

When  student  and  staff  members  are  asked  what  they  think  about  the  portfolio  idea,  they  are  almost  all  very 
positive.  But  still,  it  did  not  work  out.  The  project  fa  ils  and  we  have  to  find  out  what  we  may  learn  from  it. 
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Discussion 


Why  portfolios  can  be  a success 

When  looking  for  a field  where  portfolios  do  work,  the  field  of  Human  Recourse  Development  (HRD)  is  an 
obvious  one.  Self-responsible  adults,  who  have  a job  and  related  responsibilities,  benefit  from  individual  learning 
arrangements  that  are  reported  by  collecting  evidence  of  achievements  in  a portfolio.  The  use  of  (electronic) 
portfolios  is  in  that  context  appropriate  because  the  learner  has  sufficient  metacognitive  (and  computer)  skills  to  use 
the  portfolio  tool  properly.  An  educating  agency  (training  department,  external  course  provider,  etc.)  shares  the 
philosophy  of  competency-based  education  and  is  therefore  open  to  assessment  on  the  basis  of  individual  portfolios. 
This  context  is  essentially  different  from  the  university  situation  where  young  students  who  enter  the  university 
directly  from  secondary  school,  do  not  have  the  maturity,  the  experience  and  the  interest  to  work  along  these  lines. 
This  is  the  starting  point  for  discussing  why  portfolios  may  fail. 

Why  portfolios  fail 

Portfolios  fail  when  the  students  don’t  see  the  value.  Portfolio  proponents  tend  to  deny  the  psychological 
developmental  stage  of  the  students.  But  many  (young)  students  are  not  prepared  or  willing  to  look  at  themselves  in 
the  metacognitive  way  that  is  required  for  proper  dealing  with  portfolios.  Further  it  seems  that  the  spontaneous  fun 
in  studying  theory  is  hampered  by  precise  questions  about  the  requirements  that  have  to  be  fulfilled  for  a 
competency-based  curriculum.  A student  who  is  really  involved  in  a subject,  has  to  make  a severe  mental  switch 
when  he  or  she  has  to  step  outside  that  subject  to  analyze  on  a metacognitive  level  whether  what  he  or  she  is  doing  is 
a contribution  to  the  development  of  competence. 

And  even  when  students  see  some  value,  for  instance  for  building  a comprehensive  archive  of  their  work 
during  their  study,  they  may  misinterpret  the  function  of  the  archive  by  denying  products  of  which  they  are  not  very 
proud,  like  the  first-year  products  when  they  feel  themselves  still  beginners.  To  reflect  on  professional  growth, 
however,  these  products  are  needed  for  reasons  of  comparison  with  later  accomplishments.  Proper  guidance  of  the 
students  by  mentors  could  help,  but  this  requires  that  the  mentors  are  convinced  of  the  value  of  portfolios  as  a tool 
for  monitoring  progress.  In  our  case,  we  were  clearly  not  able  to  motivate  the  mentors  in  this  sense. 

And  there  is  also  a very  practical  reason  why  portfolios  may  fail.  That  is  when  the  software  causes 
problems.  The  system  may  need  too  many  steps  for  simple  tasks;  the  server  may  too  often  be  too  busy,  and  so  on.  In 
our  case,  several  technical  limitations  did  for  certain  not  stimulate  the  use  of  the  portfolios. 

Portfolios  also  fail  when  the  teachers  fail  to  adjust  their  teaching  accordingly.  Teachers  are  mainly  prepared 
to  carry  out  a well-defined  course.  When  they  have  to  step  beyond  the  concrete  patterns  to  adjust  themselves  to 
individual  trajectories,  many  teachers  fail  to  comply  with  that  fact.  The  idea.that  a concept  such  as  “initiation  in  the 
academic  profession”  can  be  put  into  practice  outside  the  courses  by  regarding  the  traditional  courses  as  occasions 
for  gathering  portfolio  products  that  are  used  by  the  students  and  mentors,  does  not  work.  The  concept  and  the  use  of 
the  portfolios  have  to  be  operationalized  within  the  courses. 

Where  to  go  from  here? 

Who  wants  to  succeed  in  an  effective  learning  process,  ought  to  be  able  to  coordinate  his  own  learning 
process  (McCombs,  1988).  In  order  to  make  the  concept  “initiation  in  the  academic  profession”  successful,  we  will 
have  to  arrange  a situation  in  which  the  students  will  be  helped  to  get  ready  to  do  so.  And  this  has  primarily  to 
happen  in  courses,  while  the  new  mentor  groups  may  have  a support  function.  The  developmental  readiness  of  the 
students  has  to  be  taken  into  account.  Alexander  (1995),  for  example,  mentions  three  stages  for  the  evolution  for  the 
learner.  In  the  habituated  stage,  the  student  has  a diminutive  knowledge  level.  Having  just  a little  domain -specific 
knowledge  the  student  appeals  on  common  strategies.  The  second  stage  is  the  ability  or  competency  level.  The 
students  get  more  comprehensive  and  coherent  knowledge  of  the  subject  and  there  is  a change  the  student  will  select 
the  correct  strategy  for  the  specific  situation.  The  third  stage  is  the  expert  level.  At  this  stage  the  students  have  ample 
knowledge  of  domain  specifics,  are  ready  to  regulated  themselves,  and  are  able  to  add  new  knowledge  to  the 
domain.  These  stages  ask  for  a curriculum  line  in  which  each  stage  has  a logical  place.  This  leads  in  our  case  to  a 
choice  for  courses  in  which  the  three  stages  have  a natural  place:  the  series  of  courses  about  design  methodology 
that  runs  form  the  first  to  the  third  year.  This  choice  is  inspired  by  the  Design  Studio  as  it  works  at  the  Master’s  level 
at  the  University  of  Georgia  at  Athens  (Rieber,  2001).  The  principle  as  it  will  be  tried  in  our  program  is  that  third- 
year  students  will  take  responsibility  for  design  assignments  while  second-year  students  will  act  as  helpers  for 
specific  tasks  that  need  already  proper  workmanship  (like  carrying  out  a literature  study  or  an  evaluation),  and  first 


year  students  will  be  used  for  very  concrete  tasks  for  which  it  is  not  necessary  to  be  very  knowledgeable  about  the 
specific  domain.  In  this  stream,  the  principles  of  the  concept  “initiation  in  the  academic  profession”  may  be  fully 
exploited,  together  with  the  use  of  portfolios.  Next  to  and  in  balance  with  this  stream,  theory  courses  may  still  be 
taught  in  more  traditional  ways  as  long  as  all  teaching  complies  with  the  seven  principles  for  good  practice  in 
undergraduate  education  as  listed  by  Chickering  en  Gamson  (1987,  quoted  by  Chickering  en  Ehermann, 
http://www.aahe.org/technology/ehrmann.htm):  (a)  good  practice  encourages  contacts  between  students  and  faculty, 
(b)  good  practice  develops  reciprocity  and  cooperation  among  students,  (c)  good  practice  uses  active  learning 
techniques,  (d)  good  practice  gives  prompt  feedback,  (e)  good  practice  emphasizes  time  on  task,  (f)  good  practice 
communicates  high  expectations,  and  (g)  good  practice  respects  diverse  talents  and  ways  of  learning. 

After  the  lessons  learned  with  our  attempts  to  introduce  electronic  portfolios  faculty  wide,  we  hope  that  the 
more  modest  approach  for  introducing  portfolios  and  competency-based  learning  in  the  design  stream  of  our 
program,  will  appear  to  be  the  right  step  to  help  to  initiate  our  students  into  the  academic  profession. 
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ABSTRACT 

E-portfolios  have  an  ease  of  use  that  former  paper-based  portfolios  do  not  Virginia  Polytechnic  Institute 
and  State  University  uses  ^portfolios  as  a method  to  assess  students  in  the  Instructional  Technology 
graduate  program.  A core  of  volunteer  graduate  students  assumed  the  responsibility  of  designing  a model 
for  using  e^portfolios  based  on  the  new  AECT  standards.  This  paper  explores  the  process  and  decisions 
involved  in  developing  a specification  for  an  easy-to-use  e-portfolio  system  to  assess,  evaluate  and  display 
graduate  students*  work.  Issues  discussed  include  the  nature  of  the  students'  role  in  the  development  of  the 
model  and  their  subsequent  recommendations  for  preserving  a well-defined  structure  for  the  e-portfolio  to 
meet  the  needs  of  the  faculty  without  sacrificing  the  ability  of  the  students  to  be  creative  with  the  display  of 
the  content.  Implemented  items  from  this  investigation  included  a template-based  system  with  a checklist  of 
required  components  for  the  e-portfolio.  Included  in  the  recommendations  is  a data-driven  approach  that 
separates  the  content  from  the  structure  of  the  e -portfolio,  making  it  possible  to  easily  enter  the  information 
and  re-purpose  it  for  a variety  of  formats  and  uses. 

Background 

The  professional  development  portfolio  was  conceptualized  for  use  by  the  Instructional  Technology  (IT) 
Program  at  Virginia  Tech  in  1993  as  a means  for  faculty  in  the  department  to  assess  student  work  at  different  stages 
of  matriculation.  The  portfolio  was  also  a way  for  the  program  to  gauge  its  current  practices  of  teaching  and 
learning.  For  students,  the  portfolio  served  as  an  excellent  tool  to  archive  and  showcase  their  work.  The  early 
portfolios  were  displayed  in  non-electronic  form.  Students  would  present  the  work  they  completed  in  class  during 
program-wide  examinations  (qualifying  exam,  preliminary  exam,  final  defense).  As  technologies  advanced,  the 
requirement  was  modified  to  give  students  the  option  of  creating  an  electronic  portfolio.  This  was  again  modified 
later  to  make  an  electronic  portfolio  mandatory.  The  faculty  also  adopted  the  original  AECT  standards  that 
consisted  of  seven  primary  competency  areas.  Although  all  of  the  portfolios  had  the  same  seven  topical  headings, 
the  design  for  these  portfolios  was  left  to  the  discretion  of  each  individual  student. 

When  the  new  AECT  standards  came  out  in  2000,  the  faculty  adopted  them  for  use  with  the  electronic 
portfolios.  The  electronic  portfolio,  however,  continued  to  have  several  shortcomings.  The  lack  of  guidelines  led  to  a 
great  deal  of  variation  in  interpreting  the  AECT  standards  and  in  formatting  the  portfolios.  In  addition,  students  had 
many  questions  about  how  to  develop  their  portfolios  and  faculty  members  had  a difficult  time  comparing  the 
portfolios  across  students.  As  a result,  the  Professional  Development  Portfolio  (PDP)  committee  was  formed  to 
attempt  to  incorporate  the  new  AECT  standards  into  the  departmental  PDP  and  to  find  solutions  for  these  problems. 

Beginning  of  PDP  Committee 

The  Professional  Seminar  committee,  which  plans  seminars  for  IT  graduate  students,  developed  a seminar 
on  the  professional  development  portfolio  anticipating  the  changes  in  the  AECT  standards.  During  this  seminar,  the 
new  AECT  standards  were  presented  and  a number  of  students  who  had  already  created  electronic  portfolios 
volunteered  to  show  their  work  to  new  students  in  the  program  who  were  going  to  develop  portfolios  in  the  near 
future.  Based  on  the  comments  from  both  students  and  faculty  during  this  seminar,  it  became  obvious  that  the 
portfolios  were  very  different  from  student  to  student,  which  made  it  difficult  for  faculty  to  compare  the  PDPs.  As  a 
result,  a student-led  committee,  which  included  a faculty  advisor,  was  created  to  propose  a more  systematic  PDP 
model  for  use  by  future  students.  Although  it  still  had  to  be  approved  by  the  faculty,  this  “bottom-up”  approach 
allowed  students  to  have  a voice  in  making  changes  to  enhance  the  program. 

The  PDP  Committee  subsequently  went  to  work  on  researching  electronic  portfolios.  Researching  the  use 
of  portfolios,  both  electronic  and  non-electronic,  was  undertaken  through  three  methods:  reviewing  literature, 
evaluating  other  IT  programs,  and  interviewing  IT  faculty  and  students. 
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Literature  Review 


In  reviewing  the  literature,  the  committee  investigated  portfolios  and  their  definitions,  their  uses  and 
advantages,  and  their  development.  The  committee  also  followed  this  initial  review  with  an  investigation  of  the 
AECT  standards  and  their  suitability  as  the  assessment  measure  for  the  professional  development  portfolio. 

Much  of  the  literature  found  on  the  topic  of  portfolios  was  in  support  of  their  use  in  education.  Portfolios 
are  defined  as  selective  and  purposeful  collections  of  student  work  made  available  in  either  electronic  or  non- 
electronic formats  (Adams  & Hamm,  1992).  Portfolios  provide  meaningful  documentation  of  an  individual’s 
abilities  and  represent  a “learning  history”  over  a period  of  time  (Meisels,  1994).  The  use  of  portfolios  in  education 
leads  to  learning  environments  that  are  more  student-centered  because  students  accept  more  responsibility  and 
become  the  agents  of  their  own  education  (Paris,  1 992) 

Barrett  (2000)  explores  the  steps  necessary  for  creating  portfolios.  These  steps  include  deciding  on  an  area 
of  assessment,  selecting  assessment  measures,  collecting  and  selecting  content,  reflecting  on  and  organizing  the 
content,  and  presenting  the  content.  The  committee  decided  to  adopt  this  process  as  a method  to  guide  the 
subsequent  development  of  the  PDP  model. 

In  the  investigation  of  the  suitability  of  using  the  AECT  standards  as  the  assessment  measure  for  the 
portfolios,  a primary  question  arose:  why  did  the  department  of  Instructional  Technology  at  Virginia  Tech  choose  to 
use  the  competencies  and  what  have  they  to  gain  from  including  them  in  the  professional  development  portfolio? 
The  faculty  at  Virginia  Tech  chose  to  use  the  AECT  competencies  because  of  the  reputation  AECT  has  in  the 
instructional  technology  field  and  its  leadership  in  promoting  professionalism  in  the  field  through  the  National 
Council  for  Accreditation  of  Teacher  Education  (NCATE).  In  addition,  the  IT  faculty  at  Virginia  Tech  felt  that  they 
gained  a measure  of  professional  accountability  through  the  use  of  established  standards. 

Further  investigation  of  the  AECT  Standards  addressed  several  areas.  These  areas  included  a 
developmental  history  of  the  AECT  standards,  the  theory  behind  the  standards,  and  the  standards  themselves.  AECT 
standards  came  into  existence  in  the  1 970s  when  then  president  Robert  Heinich  appointed  task  forces  to  look  into 
accreditation  and  certification  issues  addressing  “concerns  for  the  place  of  instructional  technology  in  teacher 
education  and  for  the  professional  preparation  of  media  personnel”  (Association  of  Educational  Communications 
and  Technology,  2000).  As  the  years  passed,  standards,  or  guidelines  as  they  were  then  called,  were  approved  and 
revised.  Finally  in  1996,  NCATE  requested  a change  to  performance-based  accreditation,  thus  requiring  AECT  to 
make  revisions  to  the  standards  once  again.  AECT  revised  their  1994  standards  to  reflect  NCATE’s  request.  In  order 
to  do  so,  AECT  used  the  major  domains  of  the  field  as  defined  by  Instructional  Technology:  The  Definitions  and 
Domains  of  the  Field,  written  by  Seels  and  Richey  in  1 994  (see  Figure  1 ).  The  new  standards,  which  were  used  for 
this  project,  were  approved  by  both  AECT  and  NCATE  in  2000  (Association  of  Educational  Communications  and 
Technology,  2000). 

The  AECT  standards  were  based  primarily  on  the  theoretical  framework  contained  in  the  work  found  in 
two  books:  Instructional  Technology:  The  Definition  and  Domains  of  the  Field  (Seels  & Richey,  1994)  and  The 
Knowledge  Base  of  Instructional  Technology:  A Critical  Examination  (Richey,  Caffarella,  Ely,  Molenda,  Seels,  & 
Simmonson,  1 993).  The  first  book  offers  a definition  of  the  field,  or  assessment  area  for  the  portfolio,  and  a 
description  of  the  domains  and  sub-domains  of  the  field.  The  second  book  provides  the  theoretical  underpinnings  of 
each  domain.  Seels  & Richey’s  definition  of  “instructional  technology”  is  as  follows: 

Instructional  Technology  is  the  theory  and  practice  of  design,  development,  utilization,  management  and 
evaluation  of  processes  and  resources  form  learning...  The  words  Instructional  Technology  in  the  definition 
and  a discipline  devoted  to  techniques  or  ways  to  make  learning  more  efficient  based  on  theory  but  theory 
in  its  broadest  sense,  not  just  scientific  theory...  Theory  consists  of  concepts,  constructs,  principles,  and 
propositions  that  serve  as  the  body  of  knowledge.  Practice  is  the  application  of  that  knowledge  to  solve 
problems.  Practice  can  also  contribute  to  the  knowledge  base  through  information  gained  from  experience... 
Of  design,  development,  utilization,  management,  and  evaluation  refer  to  both  areas  of  the  knowledge  base 
and  to  functions  performed  by  professionals  in  the  field...  Processes  are  a series  of  operations  or  activities 
directed  towards  a particular  result...  Resources  are  sources  of  support  for  learning,  including  support 
systems  and  instructional  materials  and  environments...  The  purpose  of  instructional  technology  is  to  affect 
and  effect  learning  (Seels  & Richey,  1994,  pp.  1 -9). 

Figure  I illustrates  how  the  field's  theoretical  underpinnings  are  divided  into  five  domains  (Seels  & Richey,  1994). 
The  related  sub-domains  are  listed  under  each  domain. 
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In  terms  of  the  standards  themselves, 
there  are  two  categories:  the  initial  programs  in 
Educational  Communications  and  Instructional 
Technology  (ECIT)  and  the  advanced 
programs  in  Educational  Communications  and 
Instructional  Technology.  The  initial  ECIT 
programs  are  the  basic  entry  into  the  field  and 
are  typically  masters  level  programs  that  lead 
to  certification.  The  advanced  ECIT  programs 
focus  on  knowledge  and  skills  that  go  beyond 
the  initial  program.  There  are  five  domains  in 
each  program:  development,  utilization, 
design,  management,  and  evaluation.  The 
definitions  of  the  domains  are  as  follows: 
Design  refers  to  the  process  of 
specifying  conditions  for  learning. 
Development  refers  to  the  process  of 
translating  the  design  specifications  into 
physical  form...  Utilization  refers  to  the  use  of 

processes  and  resources  for  learning...  Management  refers  to  processes  for  controlling  instructional  technology... 
Evaluation  is  the  process  for  determining  the  adequacy  of  instruction.  (Seels  & Richey,  1994,  pp.  24-43). 

The  standards  are  the  same  for  both  programs;  it  is  in  the  implementation  or  performance  of  the  standards 
that  the  programs  differ.  The  difference  between  the  two  is  of  quantity  and  depth.  The  advanced  program  requires 
more  activities  and  higher  quality  performance  than  the  initial  program. 

Once  the  literature  review  was  complete.  The  PDF  committee  was  convinced  of  the  usefulness  of  portfolios 
in  education.  In  addition,  the  committee  agreed  that  the  AECT  standards  justified  their  use  with  a well-defined  area 
of  assessment  (the  instructional  technology  field),  a solid  theoretical  base  and  a stringent  review  and  accreditation 
process.  The  committee  recommended  the  use  of  the  initial  ECIT  standards  as  the  framework  for  the  master’s  level 
Instructional  Technology  students’  portfolios  and  the  advanced  ECIT  standards  as  the  framework  for  the  doctoral 
students’  portfolios  at  Virginia  Tech. 


Figure  1.  Domains  of  the  Instructional  Technology  field. 
(Seels  <Sl  Richey,  1994) 


Other  IT  Programs 


In  order  to  complement  the  literature  review,  the  committee  attempted  to  evaluate  the  use  of  electronic 
portfolios  by  reviewing  websites  of  eight  well  known  IT  programs.  However,  the  use  of  portfolios  was  not  well 
documented  on  the  web,  so  the  committee  contacted  representatives  of  each  program  directly.  Members  of  the 
committee  e-mailed  a contact  person  from  each  of  the  IT  programs  requesting  information  about  their  use  of 
portfolios.  The  results  from  evaluating  other  programs  did  not  provide  any  consistent  approach.  Other  programs 
indicated  that  they:  1 .)  did  not  use  portfolios;  2.)  had  an  optional  portfolio;  3.)  had  an  analog/electronic  portfolio 
option;  4.)  created  their  own  standards;  and  5.)  used  AECT  or  other  organizational  standards.  These  findings  only 
served  to  substantiate  the  committee’s  purpose  and  commitment  to  developing  a more  consistent  set  of  PDP 
standards. 


Facuity/Student  Interviews 

Interviewing  several  members  of  the  IT  program’s  faculty  provided  valuable  insight  as  to  the  rationale 
behind  the  use  of  portfolios  and  their  needs  in  regards  to  the  portfolios.  Interviews  with  students  in  the  program  also 
provide  information  about  their  needs  as  well.  Although  the  interviews  were  done  in  an  informal  manner,  the 
anecdotal  information  the  faculty  and  students  provided  helped  the  committee  to  develop  guidelines  for  use  with  the 
portfolios.  Much  of  the  information  gathered  from  those  interviews  was  used  in  the  writing  of  this  document  and  in 
the  subsequent  implementations  and  recommendations  that  came  from  this  investigation. 

Needs  Analysis 

Once  a framework  was  adopted  for  the  electronic  professional  development  portfolio  model,  the  committee 
began  to  investigate  the  specific  needs  of  the  primary  stakeholders,  the  faculty  and  the  students.  The  primary  goal  of 
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the  team  working  on  the  model  was  to  identify  these  needs  and  to  propose  a portfolio  solution  that  would 
accommodate  each  set  of  needs  as  effectively  as  possible. 

Many  of  the  faculty  noted  the  growth  they  witnessed  from  the  time  they  first  saw  a student’s  portfolio  at  the 
qualifying  stage  to  the  final  defense.  Students  also  recognized  their  own  growth  throughout  the  process  and  felt  as  if 
they  had  a way  to  show  their  work.  Moreover,  students  discussed  how  employers  often  appreciated  their  level  of 
preparedness.  Nevertheless,  although  both  the  faculty  and  students  had  good  things  to  say  about  the  use  of 
portfolios  in  the  IT  program,  it  soon  became  obvious  that  the  portfolios,  as  they  existed,  did  not  completely  meet 
their  needs. 

Faculty  Needs 

The  faculty  analysis  resulted  in  three  specific  areas  in  which  needs  were  expressed  with  regard  to  the  PDP. 
These  areas  include:  1 .)  Assessment,  2)  Accessibility,  and  3.)  Ease  of  Use. 

Assessment 

For  the  faculty,  the  primary  area  of  importance  in  terms  of  the  PDP  is  the  ability  to  assess  the  work  that 
students  placed  in  the  portfolio.  Faculty  members  decided  that  a portfolio  based  on  the  AECT  standards  would  be  a 
useful  method  for  assessing  the  overall  achievements  of  students  in  the  program  both  on  their  own  merits  and  in 
comparison  to  other  students.  The  faculty  put  forth  general  requirements  for  items  to  include  in  the  portfolio  for 
assessment,  consisting  of  courses  taken,  activities  performed,  and  products  produced.  However,  as  the  PDP 
currently  exists,  the  faculty  has  not  provided  specific  requirements  as  to  the  manner  in  which  the  material  is  to  be 
presented.  This  was  done,  in  part,  to  provide  the  students  with  freedom  to  be  creative  with  the  design  of  their  PDPs. 
In  many  respects,  the  PDP  became,  from  the  student’s  perspective,  the  culminating  design  project  of  the  degree 
program.  The  lack  of  a defined  structure,  however,  has  made  it  difficult  for  the  faculty  to  be  able  look  at  the 
portfolios  and  assess  the  quantity  and  quality  of  work  from  one  student  to  another. 

The  ability  to  easily  assess  the  quality  of  a student’s  work  in  comparison  to  other  students  using  the  PDP 
necessitates  that  the  portfolios  be  as  consistent  as  possible.  This  consistency  can  be  achieved  through  both  content 
and  format  guidelines.  Presently,  however,  such  guidelines  have  not  been  created  due  to  the  differing  nature  of 
student  and  faculty  needs  with  regard  to  the  PDP.  To  establish  stringent  guidelines  with  regard  to  content  and 
format  may  hinder  student  creativity  and  also  their  ability  to  customize  the  PDP  for  specific  job  searches.  Taking 
some  of  the  control  of  the  development  of  the  PDP  away  from  the  students  may  force  them  to  double  their  work  by 
creating  two  portfolios  - one  for  the  instructional  technology  faculty  and  one  for  prospective  employers. 

Accessibility 

The  faculty  also  indicated  that  the  portfolios  provided  by  the  students  needed  to  be  easily  accessible.  When 
the  portfolios  were  integrated  into  the  instructional  technology  department,  they  were  primarily  paper-based. 
Eventually,  they  were  moved  to  their  current  electronic  form  in  order  to  make  them  easily  accessble  to  the  faculty. 
However,  current  lack  of  specific  electronic  guidelines  with  regard  to  how  the  PDPs  are  packaged  does  not  ensure 
easy  access  from  one  computer  to  another  or  one  application  to  another,  such  as  a browser. 

Ease  of  Use 

Finally,  the  faculty  indicated  that  it  would  be  desirable  if  the  material  in  the  student  PDPs  could  be  easily 
used  for  data  purposes.  An  example  included  the  ability  to  track  classes,  experiences  and  projects  through  a 
database.  In  such  an  environment,  theses  variables  can  be  related  and  compared  across  students  over  periods  of 
time. 

Student  Needs 

The  student  analysis  resulted  in  four  specific  areas  in  which  needs  were  expressed  with  regard  to  the  PDP. 
These  areas  include:  1.)  Clear  definition  of  requirements,  2.)  Flexibility  for  creativity,  3.)  Usability  for  both 
academic  and  job  related  activities  and  4.)  Easy  modification  of  content. 

Defined  requirements 
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Students  used  the  PDP  primarily  as  a means  to  fulfill  the  requirement  set  out  in  the  series  of  oral 
departmental  exams.  The  main  requirement  set  forth  by  the  faculty  was  that  the  PDP  be  web-based  and  that  all 
classes,  projects  and  professional  experiences  be  represented.  The  required  structure  of  the  document  was  only  that 
these  items  be  represented  so  that  it  was  apparent  how  each  related  to  the  original  seven  competencies  detailed  by 
AECT.  This  limited  set  of  requirements  allowed  for  a great  deal  of  variety  in  the  design  of  PDPs  and  promoted 
confusion  among  the  students  as  to  what  was  required.  Students’  inaccurate  understanding  of  the  PDP  made  it  clear 
that  it  would  be  helpful  to  make  the  requirements  more  comprehensive.  In  addition,  well-defined  structural 
requirements  would  reduce  the  amount  of  time  spent  by  the  students  in  development  of  their  own  PDP.  This  is 
especially  the  case  for  students  who  are  not  as  experienced  in  web  or  graphic  design. 

Flexibility  to  be  creative 

The  original  setup  allowed  for  a great  deal  of  personal  freedom  in  structure  and  displays.  Although  the 
PDPs  had  a similar  outline-based  structure  that  consisted  of  an  introduction  with  a definition  of  the  PDP  followed  by 
seven  sets  of  lists,  the  overall  design  was  unique  from  student  to  student.  Flexibility  was  exercised  in  visual  aspects 
of  the  site  such  as  images  and  color  scheme.  The  students  wanted  guidelines,  but  also  desired  the  flexibility  to 
design  the  “look  and  feel”  of  their  portfolios. 

Usability  for  both  academic  and  job  related  activities 

It  was  important  for  students  to  have  a PDP  that  would  also  work  as  a non-academic  portfolio.  Often  the 
structure  used  to  meet  the  needs  of  the  IT  faculty  at  Virginia  Tech  was  not  appropriate  for  prospective  employers. 

As  a result,  students  often  had  to  re-design  their  portfolios  for  use  in  job  searches.  Since  the  content  and  format 
were  tied  together  in  the  electronic  portfolios,  this  required  much  time  and  effort. 

Easily  modifiable 

The  PDP  is  continuously  modified  throughout  its  development.  A well-designed  PDP  should  allow  for 
easy  addition  of  content,  as  well  as  resorting  of  items  already  included,  without  the  need  to  recreate  the  document 
format  or  navigation  systems. 

Solutions 

The  recommendations  of  the  group  came  in  three  forms.  The  first  piece  was  a prototype  PDP  to  be  used  as 
a model  for  future  PDPs.  A “how-to”  manual  was  developed  for  students  to  guide  them  through  creating  their  own 
PDPs.  The  last  piece  consisted  of  a set  of  recommendations  for  future  investigation. 

Prototype 

The  prototype  was  a website  developed  from  content  in  an  already  existing  PDP.  A template-based  system 
was  recommended  in  order  to  provide  clearer  guidelines  for  the  students  as  they  develop  their  PDPs.  Though  this 
recommendation  removes  the  students’  flexibility  in  the  aesthetic  aspects  of  their  PDP,  a more  standardized  model 
had  the  benefit  of  reducing  ambiguity  in  the  development  of  the  PDP.  The  imposition  of  these  content  and  format 
guidelines  also  meets  the  need  of  the  faculty  to  easily  locate,  analyze,  and  compare  the  work  of  students  across 
PDPs.  Through  the  use  of  the  template,  the  content  is  more  consistent,  comparable,  and  thus,  more  assessable. 

In  the  prototype,  each  page  on  the  site  contains  a side  bar  navigation  system  that  allows  the  viewer  to  link 
to  any  of  the  other  major  sections  within  the  site.  As  shown  in  Figure  2,  the  site  contained  5 major  sections:  (1) 
Entry  page,  (2)  Vita,  (3)  Checklist,  (4)  Competency  listing,  and  (5)  Referenced  items. 
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Figure  2.  PDF  Site  Map 
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The  entry  page  was  set  up  as  an  introduction  to  the  individual  student’s  PDP.  A personal  statement  is  the 
most  prominent  content  of  the  page.  The  student  has  the  ability  to  add  links  to  outside  personal  information  from 
the  entry  page.  (Faculty  felt  that  sensitive  personal  information  should  be  removed  from  the  entry  page,  as  the  PDP 
site  is  not  secure.) 

Faculty  and  students  both  felt  that  a vita  should  be  included  as  a part  of  the  PDP  as  it  is  considered  to  be  a 
standard  element  in  the  professional  development  of  an  Instructional  Technology  student.  Since  there  are  already 
established  guidelines  for  vitas,  the  prototype  simply  focused  on  a location  in  the  site  to  place  the  vita,  not  on  how  to 
develop  one. 

The  checklist  identifies  all  of  the  required  components  in  each  of  the  major  areas  of  the  PDP  in  a table  (see 
Figure  3).  The  checklist  allows  the  students  to  identify  which  components  have  been  included  in  the  PDP  by 
inserting  a checkmark  graphic  into  the  appropriate  cell  of  the  table.  This  chart  is  sorted  by  competency  and  type  of 
referenced  item  (classes,  professional  activities,  projects).  This  method  allows  the  faculty  to  quickly  judge  the 
breadth  of  the  student’s  experiences  as  defined  in  the  portfolio.  In  addition,  it  provides  the  students  with  a clear  set 
of  guidelines  indicating  what  is  required  in  the  portfolio. 


Figure  3.  PDP  Checklist 
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The  competency  section  of  the  site  details  each  of  the  5 categories  of  the  AECT  standards.  Like  the 
checklist,  this  section  is  included  to  assist  the  students  in  defining  their  experiences  as  they  relate  to  the  profession. 
General  as  well  as  itemized  definitions  as  put  forth  by  AECT  for  each  competency  are  listed  on  the  pages.  At  the 
end  of  each  page,  there  are  links  to  classes,  projects,  and  professional  activities  that  relate  to  that  competency. 

In  addition  to  linking  to  the  referenced  items  through  the  competency  pages,  the  PDP  contains  three  pages 
that  list  out  all  items  by  category.  The  categories  for  the  referenced  items  are  classes,  projects,  and  professional 
activities.  This  will  allow  the  viewer  to  isolate  types  of  work  (e.g.,  class  projects,  and  professional  activities)  across 
the  competencies.  This  also  allows  for  alternate  groupings  and  classifications  to  be  set  up  if  needed.  Creating  a new 
table  of  contents,  and  linking  the  appropriate  items  to  the  contents  page,  will  allow  for  a completely  new  structure 
for  a portfolio  for  viewers  other  than  the  IT  faculty.  If  the  student  wishes  to  modify  the  appearance  of  the  page,  only 
the  templates  for  the  sections  need  to  be  modified  and  the  whole  site  will  reflect  the  changes. 
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“How-to”  document 


The  “how-to”  document  was  created  as  a guide  for  students  to  creating  their  own  PDFs.  It  contains 
explanations  of  competencies,  descriptions  of  the  required  elements,  and  recommendations  for  development  and 
maintenance  of  the  PDF.  The  competencies  describe  in  detail  the  AECT  areas  that  the  students  are  expected  to  use 
as  a framework  for  populating  the  content  in  the  PDP.  This  section  also  describes  how  the  Instructional  Technology 
department  will  work  with  students  if  they  wish  to  emphasize  specific  components  of  these  competencies.  The 
description  of  required  elements  outlines  what  the  faculty  expects  from  the  students  in  terms  of  competencies  and 
other  activities.  These  other  activities  include  the  development  of  a vita,  consulting,  teaching,  researching,  and 
mentoring  newcomers  to  the  profession.  The  development  and  maintenance  section  includes  recommendations  for 
setting  up  a website  to  house  the  PDP  and  techniques  for  modifying  and  updating  the  content  in  the  PDP.  This 
document  provides  clear  guidelines  and  developmental  procedures  for  students.  In  addition,  by  clearly  laying  out 
these  guidelines,  the  faculty  has  a reference  point  with  which  to  evaluate  the  portfolios. 

Future  Recommendations 

In  the  discussion  of  how  to  develop  a PDP,  the  group  made  recommendations  for  future  exploration  of 
several  concepts.  The  committee  began  to  explore  standard  formatting  structures  for  the  PDP,  such  as  HTML  or 
XML  and  the  use  of  cascading  style  sheets  for  formatting.  These  structures  could  ensure  consistency  of  PDFs  across 
computer  platforms  and  browsers.  The  committee  also  investigated  the  possibility  of  the  PDP  structure  existing  in  a 
database-driven  environment  that  separates  the  content  of  the  PDP  from  the  recommended  template.  By  having  all 
of  the  content  entered  into  the  database  and  then  automatically  populated  into  the  template,  the  faculty  would  be  able 
to  isolate  and/or  compare  specific  pieces  of  content  within  a database  system.  This  would  facilitate  management 
and  organization  of  large  amounts  of  data  from  the  PDFs  from  a variety  of  students  across  many  years.  In  such  a 
system,  students  adding  content  would  not  need  to  worry  about  formatting.  The  recommended  template  would 
automatically  be  populated  with  the  content  maintained  in  the  database.  In  order  to  take  into  account  the  needs  of 
different  students,  multiple  templates  could  be  offered  for  the  students  to  use  in  the  data  driven  system.  Pre  - 
designed  as  well  as  customizable  templates  could  be  chosen  as  options  for  PDP  display.  This  would  begin  to 
accommodate  the  individual  design  needs  of  the  students  without  forcing  them  to  re-enter  their  content. 

Summary 

In  the  development  of  the  PDP  model,  both  the  process  and  the  outcomes  provided  positive  benefits  to  the 
students  and  the  faculty.  The  students,  through  the  committee,  took  ownership  of  the  development  of  the  PDP 
model.  In  addition,  both  the  students  and  the  faculty  had  their  needs  met  through  the  analyses  and  recommendations 
developed  by  the  committee.  More  work  in  needed  in  the  development  of  this  model,  but  the  committee  has 
developed  a solid  framework  to  guide  future  exploration. 
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Abstract 

Leveraging  Internet  technologies,  distance  education  is  enjoying  a renaissance  of  sorts.  With  its  newfound 
popularity  come  greater  resources  as  well  as  higher  expectations  and  greater  scrutiny.  If  distance 
education  programs  are  to  support  their  dramatic  growth  and  outlive  the  hype,  they  must  demonstrate 
performance  results.  Performance,  however,  does  not  just  happen.  High  quality  organizations  actively 
support  performance  through  processes,  structures,  and  feedback  systems  that  are  aligned  with 
organizational  goals.  In  presenting  recommendations  for  performance-oriented  approaches,  an  established 
model  of  organizational  alignment  is  applied  to  distance  education  in  postsecondary  institutions.  The 
model  facilitates  the  analysis  of  goals,  structure,  and  management  practices  across  the  organization,  its 
processes,  as  well  as  the  work  and  workers  involved.  Also  presented  are  performance  improvement 
strategies  such  as  benchmarking  and  documenting  workflows,  setting  clear  expectations,  and  developing 
feedback  systems.  Distance  education  practitioners  can  use  the  organizational  alignment  model  and  he 
strategies  discussed  to  design  a new  program  or  transform  an  existing  one. 

Introduction 
Onto  the  Web  we  go 

Though  distance  education  has  been  around  in  various  forms  for  many  years,  it  is  only  recently  — with  the 
emergence  of  web-based  delivery  platforms  — that  it  has  become  the  center  of  so  much  attention  (Foshay  & Moller, 
in  press).  “Where  distance  education  used  to  suffer  from  benign  neglect,  now  everyone  wants  to  have  a say  in  how 
(it)  is  used,  controlled,  and  managed”  (Sachs,  1999:  p.75).  Many  institutions  that  previously  ignored  distance 
education  altogether,  are  now  rushing  to  deploy  their  own  online  programs  in  an  effort  to  curb  damage  to  their 
enrollments  and  possibly  their  reputations. 

New  entries 

Post-secondary  institutions  face  leveling  student  enrollment  and  rising  competition  from  several  directions. 
Previously  viewed  as  relatively  insignificant  outreach  projects,  distance  education  programs  (often  from  smaller 
schools)  are  taking  more  and  more  students  away  from  large,  regionally  dominant  schools.  Institutions  can  no 
longer  take  for  granted  the  geographic  boundaries,  or  “turfs,”  that  enabled  them  to  virtually  monopolize  their 
regions.  For-profit  educational  institutions  (e.g..  University  of  Phoenix)  are  becoming  more  commonplace  and  are 
beginning  to  gain  accreditation.  Also,  there  is  an  increasing  number  of  professional  certification  programs,  which 
more  and  more  “free-agent”  learners  are  using  in  lieu  of  matriculated  programs  to  gain  entry  into  the  workforce  and 
to  increase  their  marketability  (Foshay  & Moller,  in  press). 

Veterans  transitioning  to  the  Web 

Those  colleges  and  universities  that  long  ago  embraced  distance  education  (through  print,  audio,  and  video 
technologies)  face  critical  challenges  also.  Stay  within  their  niche  of  experience  and  face  possible  obsolescence? 

Or,  “take  arms  against  a sea  of  troubles”  and  embrace  the  Internet  (with  computer  mediated  communication  tools)  as 
the  vehicle  of  choice  for  distance  education?  The  latter  presents  significant  restructuring  and  reengineering 
challenges. 

Current  online  programs 

Though  many  institutions  have  decided  to  take  the  plunge  into  web-based  distance  learning,  few  have 
found  a winning  recipe  for  success.  In  fact,  in  the  rush  to  move  courses  online,  front-end  analysis  and  planning  are 
often  first  to  be  discarded  for  the  sake  of  expedience.  It  seems  that  in  at  least  some  cases,  distance  education 
administrators  are  taking  the  ‘hare’  approach,  figuring  that  Just  this  once,  they  will  win  the  race  against  the  tortoise. 
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Distance  education  in  the  spotlight 


In  a Department  of  Education  survey  (1997*98),  20%  of  the  respondents  - 990  post-secondary  institutions 
- reported  that  within  three  years  they  planned  to  join  the  1,680  schools  offering  online  distance  education  courses 
(NCES,  2000).  With  about  54,000  courses  available  over  the  web  in  1998,  serving  1.6  million  students,  and  with 
relatively  low  economic  barriers  to  entry,  instead  of  asking  ‘Why  distance  learning?’  many  university  presidents  are 
asking  ‘Why  not?’ 

With  increased  attention  from  university  presidents  comes  more  resources  and  higher  status  as  well  as 
higher  expectations  and  greater  scrutiny.  Expectations  typically  take  the  form  of  business  results:  increased 
enrollments,  lower  operating  costs,  and  (in  some  organizations)  higher  profits.  Greater  administration  commitment 
and  scrutiny  ensures  that  in  order  to  thrive  in  today’s  more  businesslike  post-secondary  institutions,  distance 
education  programs  must  demonstrate  business  results.  Otherwise,  they  risk  eventual  outsourcing,  re-structuring, 
and  (perhaps  worst  of  all)  divestiture.  A new,  results -oriented  approach  is  needed  to  assure  that  distance  education 
efforts  are  focused  on  goals  that  are  in  alignment  with  the  institution’s  mission. 

Towards  business  results;  A human  performance  approach 

Administrators  who  recognize  the  need  to  instill  a results-oriented  mindset  in  their  distance  education 
departments  may  find  it  helpful  to  examine  proven  strategies  from  the  field  of  human  performance  technology 
(“HPT”).  HPT  draws  from  a variety  of  other  fields,  including  management  sciences,  organizational  development, 
instructional  systems  design,  and  quantitative  analysis.  Figure  1 lists  several  of  HPT’s  core  principles.  The 
strategies  that  have  evolved  from  these  p rinciples  are  as  diverse  as  HPT’s  origins.  Harmon  (1984)  provides  a 
hierarchical  structure  for  organizing  HPT  strategies.  These  encompass  high  level,  organizational  solutions  such  as 
total  quality  management  ("TQM")  and  balanced  scorecard  evaluation  systems  as  well  as  more  grass  roots, 
individual-oriented  applications  such  as  job  re-engineering,  performance  support  tools,  and  360-degree  performance 
evaluations.  In  general,  HPT  advocates  measuring  actual  performance;  comparing  results  against  pre-set  goals;  and 
implementing  systems,  processes,  and  practices  that  are  vertically  aligned  with  those  goals.  Considering  the 
seeming  lack  of  clear  goals,  long-term  planning,  and  feedback  systems  that  characterize  distance  education  today, 
applying  an  HPT  approach  may  shed  some  light  on  best  practices  for  establishing  a successful  program  or 
transforming  an  existing  program  into  a results-oriented  organization. 

Figure  7.  Human  Performance  Technology  core  principles 

T.  ftWd^  between  behavior  (what  someone  does)  and  performance  (measurable  results  achieved),  and  j 

focuses  primarily  on  performance.  | 

2.  Diagnosing  performance  problems  involves  measuring  the  gap  between  current  and  expected  performance. 

3.  All  performance  problems  stem  from  the  worker,  the  environment,  or  a combination  of  both. 

4.  Processes  depend  on  people  to  make  them  work,  but  people  are  not  productive  when  they  are  in  bad  processes.  | 

5.  Organizations  are  open  systems  with  inputs,  outputs,  processes,  and  feedback  systems.  Management, 
development,  and  systems  functions  affect  all  organizational  performance. 

• Management  functions  guide  and  control  the  organization’s  processes  and  people  through  | 

feedback,  coaching,  reprimands,  and  incentives.  | 

• Development  functions  improve  the  abilities  and  capabilities  of  workers  through  training,  | 

mentoring,  and  other  developmental  activities.  | 

• Systems  components  are  the  variables  in  the  work  environment  that  affect  performance  such  as  i 

work  processes,  standard  operating  procedures,  policies,  practices,  feedback  mechanisms,  reward  | 
structures,  etc.  | 

6.  Exemplary  performance  should  be  used  to  benchmark  workflows,  set  optimal  performance  levels,  etc.  j 

6.  Rather  than  seeing  HPT  solutions  as  expenses,  they  should  be  viewed  as  investments  and  return  on  investment 

(“ROI”)  calculated  for  aU  ^ I 

Adapted  from  Rothweli  (1 996:  pp.^^^^^ 

An  organizational  alignment  model 

Rummler  & Brache’s  Organizational  Alignment  Model  (1990)  provides  a useful  framework  for  analyzing 
and  aligning  the  goal-setting,  structure,  and  management  practices  of  an  institution,  its  distance  education  processes. 
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and  its  staff  members.  An  analysis  matrix  has  been  adapted  from  this  model  and  is  shown  in  Figure  2.  The  nine 
cells  of  this  matrix  form  the  nine  focal  points  for  analysis.  A similar  approach  involving  organizational  alignment 
analysis  has  been  used  to  analyze  parallel  issues  in  training  and  e-leaming  contexts  (Moller,  Benscoter,  Rohrer- 
Murphy,  2000;  Prestera  & Moller,  in  press).  The  model  will  be  applied  here  in  the  context  of  distance  education 
both  to  the  question  of  what  is  and  what  should  be  present  to  support  the  goals,  structure,  and  management  of 
distance  education  programs.  Recommendations,  often  founded  in  human  performance  technology  (HPT)  and 
organizational  behavior  literature,  will  also  be  presented. 


Fi^ure^^^rganizat^^ 


Goals 

Structure 

Management 

Organization 

Cell  1 - What  will 
training  contribute  to  the 
institution’s  business 
goals? 

Cell  4 - How  should  we 
structure  the  distance 
education  group  to  meet 
the  institution’s  business 
goals? 

Cell  7 - How  will  we 
measure  and  improve 
contributions  to  business 
goals? 

Process 

Cell  2 - What  are  the  key 
success  factors  for 
delivering  distance 
education  such  that  they 
meet  the  business  goals? 

Cell  5 - How  should 
workflows  be  structured  in 
order  to  drive  success? 

Cell  8 - How  will  we 
measure  and  improve  the 
process? 

Job/  Performer 

Cell  3 - What  do  we  need 
from  staff  members  in 
order  to  achieve  our 
process  goals? 

Cell  6 - How  should  roles 
and  responsibilities  be 
defined  in  order  to  meet 
expectations  and  deliver 
process  results? 

Cell  9 - How  will  we 
measure  and  improve 
individual  performance  on 
the  job? 

Adapted  from  Rummler  6c  Brache  (1990) 


Goals 


Rothwell  (1996)  defines  a goal  as  '*a  target  for  achievement...  difficult  to  measure  or  to  achieve  in  an 
identifiable  time  span...  developed  directly  from  an  organization’s  mission  or  purpose  statements.”  (p.  1 15).  Goal- 
oriented  by  nature,  human  beings  often  set  goals  in  pursuit  of  loftier  purposes,  reaching  out  towards  visions  of  a 
better  future.  It  should  be  of  no  surprise  then  that  in  the  workplace,  higher  performance  is  possible  when  workers 
know  what  goals  they  should  achieve  and  are  confident  in  their  ability  to  achieve  them  (House,  1971).  Therefore, 
distance  education  administrators  should  set  departmental  goals  that  are  in  alignment  with  the  institution’s  mission, 
perceived  as  achievable,  and  well-communicated  to  staff  members. 

In  practice,  however,  departmental  and  organizational  goals  rarely  seem  to  possess  these  traits.  Distance 
education  initiatives  are  often  undertaken  with  at  best  only  token  consideration  of  institutional,  logistical,  and 
instructional  needs  (Blumenstyk,  1996).  When  program  goals  do  exist  they  are  often  tied  closely  to  misperceptions, 
which  ultimately  undermine  the  success  of  the  initiative.  In  other  instances,  goals  do  not  align  with  the  program’s 
processes  and/or  they  are  not  communicated  properly  within  the  program.  In  this  section,  an  argument  is  presented 
for  establishing  results -oriented  goals,  benchmarking  successful  process  goals,  and  clearly  communicating 
performance  expectations  for  distance  education  staff 

Cell  1:  Organization  goals 

Many  managers  both  in  the  corporate  world  and  in  non-profit  organizations  manage  their  organizations 
with  only  one  or  two  broad  performance  indicators,  which  offer  limited  help  in  diagnosing  performance  gaps,  are 
typically  too  broad  for  individual  workers  to  feel  accountable  for  them,  and  generally  create  a sense  of  tunnel  vision. 

Recommendation  lA:  Use  balanced  approach  to  goal  setting 

For  decades,  managers  have  sought  out  the  “magic  number,”  the  one  indicator  that  accurately  measures  the 
health  of  their  organization.  Net  income,  earnings  per  share,  economic  value  added,  and  the  other  financial 
measures  are  widely  used  in  for-profit  organizations.  Enrollment  figures  are  typically  used  in  academic  settings. 
However,  a new  paradigm  in  the  area  of  performance  measurement  has  arisen  in  the  last  ten  years.  It  asks  the 
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question,  “What  are  your  organization’s  key  success  factors?”  and  “How  can  you  assess  your  organization’s 
performance  in  those  areas?”  Kaplan  and  Norton  (1996)  categorized  these  factors  into  four  dimensions  of 
performance:  learning,  operating  efficiency,  customer  satisfaction,  and  financial  success.  Having  only  one 
performance  measure,  the  authors  assert,  is  like  driving  a car  with  no  other  indicators  but  a fuel  gauge.  In  distance 
education,  organizations  should  move  beyond  enrollment  targets  and  set  goals  that  - as  a group  - paint  as  complete 
a picture  of  performance  as  possible.  Figure  3 lists  some  possible  success  factors  that  could  be  part  of  a distance 
education  balanced  scorecard. 


f'igure  3.  Possible  balanced  scorecard 


Customer 
Satisfaction  Goals 

• Improve  access  - To  imnrove  access  to  instruction  for  workers  whose  schedules  do 
not  permit  enrollment  in  traditional  classroom  programs  (e.g.,  night  shift  crews),  are 
geographically  dispersed,  or  economically  challenged. 

• Individualize  instruction  - Individualize  instruction  through  blended  course  offerings 
(that  combine  delivery  methods)  and  individualize  content  selection. 

• Lifelong  learning  - To  suDDort  continuous  learning,  sustaining  learning  experiences 
beyond  the  time  constraints  of  the  classroom. 

• Emplover  relations  - How  can  we  improve  emplover  perceptions/acceptance  of 
distance  education  degrees  so  that  students  with  such  degrees  are  perceived  as  just  as 
qualified,  if  not  more  qualified,  than  resident  students. 

• Collaboration  - To  provide  collaborative  educational  opportunities  (see  discussion 
groups,  CSILEs,  and  CSCWs). 

Operating 

• Delivery  efficiency  - To  organize,  update,  and  distribute  content  efficiently  and 

Efficiency  Goals 

effectively  through  knowledge  management  systems  (see  reusable  information 
objects  and  reusable  learning  objects). 

• Reducing  downtime  - What  steps  can  be  taken  to  reduce  student/facultv  downtime 
caused  by  technological  constraints  (e.g.,  bandwidth)  and  system  errors. 

• Scheduling  flexibility  - To  improve  flexibility  in  scheduling  for  faculty  and  content 
experts,  enabling  them  to  leverage  their  time. 

Learning  Goals 

• Innovate  instruction  - Distance  education  programs  can  act  as  testing  ground  for 
performance-based  assessments,  new  learning  paradigms,  and  instructional 
strategies. 

• Faculty  development  - As  part  of  the  design  and  delivery  process,  faculty  will 
develop  new  medium-specific  skills  and  may  improve  their  overall  teaching  skills  as 
well. 

Financial  Goals 

• Operating  costs  -To  reduce  costs  for  the  student,  the  school,  and  the  faculty. 

• Profit  - With  for-profit  organizations,  this  will  inevitably  be  a measure. 

Recommendation  IB:  Assess  needs  of  distance  education  stakeholders 

As  part  of  identifying  the  goals  and  measures  to  be  included  in  this  scorecard,  distance  education  programs 
should  assess  the  needs  of  all  stakeholders,  including  potential  and  existing  students,  faculty,  administration,  and  the 
IT  department.  For  example,  though  distance  education  is  gaining  acceptance  among  students,  it  is  by  no  means 
mainstream.  In  fact,  the  non-traditional  students  who  would  typically  enroll  in  distance  education  courses  are  still 
the  in  the  minority  (Young,  2000).  As  marketing  researchers  know  firsthand,  there  is  a significant  difference 
between  a consumer’s  willingness  to  buy  a product  or  service  (acceptance)  and  their  actual  motivation  to  buy  it 
(demand).  Assessing  the  demand  for  distance  education  courses  (through  surveying,  pilot  studies,  etc.)  should 
precede  any  distance  education  enterprises.  Similarly,  student  needs  regarding  content,  interface,  technical  support, 
scheduling,  etc.  should  be  assessed  as  well.  Mclsaac  (1998)  contrasts  the  U.S.  model  of  distance  education  with  the 
needs-based  approach  of  non-U. S.  institutions. 

“Many  countries  have  identified  educational  needs  and  have  designed  distance  learning  delivery  systems  to 
meet  those  needs.  Conversely,  it  appears  that  educators  and  administrators  in  the  U.S.  have  discovered 
broadband  communication  technology  and  are  searching  for  ways  to  use  the  technology  to  compete  for 
students  in  the  higher  education  market  and  to  cut  costs,  (p.24)” 

Recommendation  1C:  Communicate  goals  and  measures 
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Once  the  program’s  goals  and  measures  - which  should  be  aligned  with  the  institution’s  mission  - are  set, 
they  should  be  communicated  to  the  entire  college  or  university  with  support  from  the  President.  Support  from  the 
top  is  critical  in  moving  the  program  past  gridlock.  This  is  especially  true  for  distance  education  programs  since 
their  activities  must  (by  their  very  nature)  cross  geographic/departmental  boundaries  and  consequently  traverse 
many  political  obstacles.  In  describing  the  success  of  the  distance  education  program  at  Northern  Virginia 
Community  College’s,  Sachs  (1999)  singled  out  public  support  from  the  President  as  “a  key  ingredient”  and  wrote 
that  “This  frequently  meant  discussions  were  not  stuck  on  ‘whether’  to  do  something  and  moved  to  ‘how  to  do  it.  (p. 
68).” 

Cell  2:  Process  goals 

“Place  a good  worker  in  a flawed  process  and  the  process  wins...  every  time”  (Rummler  & Brache,  1990). 
This  truism  underscores  the  importance  of  establishing  a sound  operational  process  for  developing  and  delivering 
distance  education  courses.  Identifying  the  goals  of  a distance  learning  program  requires  careful  planning  and  an 
eye  towards  the  future.  Unfortunately,  as  Simerly  (1999)  points  out,  “today’s  organizational  planning  must  occur  in 
an  environment  of  unpredictability  that  usually  does  not  proceed  in  linear  ways...  (with)  key  stakeholders  often 
(having)  conflicting  points  of  view  regarding  the  planning  process  and  its  outcomes”  (p.42).  With  the  rise  in 
acceptance  of  distance  education  comes  greater  internal  conflict,  as  more  stakeholders  are  involved,  each  jockeying 
for  authority  over  online  resources.  Also,  distance  education  programs  are  often  in  such  a hurry  to  move  content 
online  that  workflows  are  rarely  documented,  measured,  or  evaluated.  Without  clear  process  goals,  however, 
consistency,  accountability,  and  collaboration  are  jeopardized.  If  each  member  of  a design  team,  for  example,  has  a 
different  idea  of  how  the  design  process  is  supposed  to  work  --  and  there  is  no  documented  way  of  doing  things - 
time  will  be  wasted  and  quality  may  even  be  compromised.  In  addition,  without  documenting  workflows,  there  is 
no  means  to  retain  ‘lessons  learned’  and  other  valuable  organizational  learning. 

Recommendation  2A:  Benchmark  workflows 

With  the  rise  of  Total  Quality  Management,  many  organizations  are  benchmarking  their  internal  and 
external  processes  (Rothwell,  1996).  Though  quantitative  sources  of  data  for  online  distance  education  are  scarce, 
there  are  some  benchmarking  reports,  case  studies,  and  guideline  recommendations  available.  Carnevale  (2000) 
describes  a benchmarking  study  of  six  institutions  with  strong  distance  education  programs,  conducted  by  the 
Institute  for  Higher  Education  Policy  (an  executive  summary  can  be  downloaded  at 

http://www.ihcp.com/qualitvonline.pdf>.  Twenty-four  benchmarks  for  online  distance  education  are  provided, 
including  the  importance  of  online  interaction  with  instructor  and  other  students.  Another  source,  described  by 
Young  (2000),  is  a faculty  report  written  by  16  professors,  which  takes  a critical  look  at  distance  education.  The 
report,  for  example,  recommends  against  the  implementation  of  complete  undergraduate  degree  programs  for 
traditional  students,  citing  student  need  of  “personal  socialization”  (this  report,  titled  ‘Teaching  at  a distance’  is 
available  online  at  http://www.vpaa.uillinois.edu/tid/report/).  Also,  several  authors  provide  recommendations  and 
guidelines,  often  based  on  practical  experiences  (Trentin,  1998;  Sachs,  1999;  Mclsaac,  1998;  Collis,  1999;  Evans, 
1999;  Prestera  & Moller,  in  press). 

Benchmarking  questions  to  be  answered  include: 

• What  should  be  happening  in  course  development  and  delivery  that  will  improve  how  staff 
members  perform? 

• What  should  be  happening  to  meet  or  exceed  student  expectations  of  their  online  course 
experience? 

• What  resources  will  staff  members  need  in  order  to  perform  optimally? 

Cell  3:  Work/Worker  goals 

An  examination  of  what  should  be  happening  with  work  and  workers  means  identifying  the  competencies 
needed  to  achieve  the  department’s  process  and  organizational  goals.  Competencies  are  often  examined  in  the 
context  of  job  descriptions,  performance  standards,  and  feedback  mechanisms. 

Recommendation  3A:  Write  or  re-write  job  descriptions 
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Job  descriptions  document  work/worker  goals  and  are  an  excellent  means  for  communicating  expectations. 
In  developing  job  descriptions,  distance  education  administrators  should  benchmark  the  roles,  competencies,  and 
performance  outputs;  compare  those  against  the  department’s  current  skill  set;  identify  any  gaps  that  exist;  and 
develop  a strategy  for  acquiring  those  competencies.  This  may  involve  hiring  new  individuals  with  the  right  skills, 
re-engineering  jobs,  and/or  developing  skills  through  training,  mentoring,  practice,  and  other  developmental 
activities.  In  a survey  of  103  distance  education  experts,  Thach  and  Murphy  (1995)  identified  the  principal  roles, 
skills,  and  outputs  required  to  deliver  distance  education.  Administrators  can  use  these  results  as  a basis  for 
assessing  competencies  within  their  departments.  The  study  identifies  eleven  roles,  which  are  listed  along  with  their 
descriptions  in  Figure  4. 

Figure  4.  Summary  J l 


.Role  Description 

"iiistra^  w’o7k'whrin  SMEs  to  design  courses,  revise  existing  courses  to  fit 

distance  learning  environment 

Instructor  Lead  instructional  design  effort,  facilitate  course  delivery,  monitor  and  evaluate 

learner  performance 

Technology  Expert  Advise  in  selection  of  distance  learning  technology,  ensure  reliability  of  technology, 

assess  future  changes  in  technology 

Technician  Keep  equipment  in  running  condition,  respond  to  users’  questions  and  problems 

Administrator  Manage  staff  and  operations,  balance  budget,  market  distance  learning  programs 

I Site  Facilitator  Assist  students  in  learning  at  remote  sites,  distribute/collect  materials/assignments, 

proctor  tests 

i Support  Staff  Register  students,  communicate  course  schedule/descriptions,  coordinate  support 

services 

I Editor  Edit  course  content  for  style,  clarity,  grammar,  and  structure.  Arrange  text  layout  for 

presentation 

I Librarian  Provide  library  assistance  to  students,  assist  with  searches,  delivery  materials  to 

students 

i Evaluation  Specialist  Provide  tools  and  evaluation  instruments,  monitor  program  success/problems,  consult 

instructor  about  evaluation 

j Graphic  Designer  Design  attractive,  clear  layo^  


Adapted  from  Thach  (Si  Murphy  (1995:  p.  67-69) 

The  list  includes  primary  roles  (namely,  instructors,  instructional  designers,  technology  experts,  and 
administrators)  as  well  as  supporting  roles,  such  as  editors,  librarians,  and  graphic  designers.  General  interpersonal 
skills,  such  as  communication,  collaboration,  and  teamwork,  as  well  as  other  more  role -specific  skills  are  listed  in 
conjunction  with  expected  performance  outputs.  These  may  serve  well  as  benchmarks  for  writing  job  descriptions, 
recruitment  profiles,  and  performance  goals.  They  can  also  guide  the  use  of  development  plans  and  the 
implementation  of  certification  programs  for  distance  education  faculty,  instructional  designers,  etc. 

For  veteran  distance  education  programs,  some  additional  issues  include: 

• Do  current  workers  have  the  technical  and  interpersonal  skills  to  work  effectively  with  faculty  in 
developing  online  courses? 

• Do  supervisors  have  the  necessary  skills  to  deal  with  the  complex  political  issues  of  a growing 
distance  education  program? 

Recommendation  3B:  Link  goals  to  reward  system 

“People  will  do  what  they  are  rewarded  for  doing”  (Rothwell,  1 996:  p.257).  Though  this  truism  seems 
relatively  obvious,  there  are  still  managers  around  who  seem  to  believe  that  people  perform  simply  to  keep  their  jobs 
or  out  of  a sense  of  loyalty.  There  are  few  examples  that  better  demonstrate  this  than  faculty  in  distance  education. 
Before  distance  education  became  an  overnight  sensation,  practitioners  were  primarily  faculty  who  volunteered. 
Now,  with  many  faculty  being  asked  to  teach  online,  resistance  is  on  the  rise  (Dillon  & Walsh,  1 992).  At  many 
universities,  and  particularly  land  grant  institutions,  faculty  are  evaluated,  promoted,  and  rewarded  based  primarily 
on  their  research,  and  to  a lesser  degree,  classroom  teaching  and  outreach  services.  Few  schools  offer  rewards  for 
teaching  distance  education  courses  (Wilson,  1998:  Wolcott,  1997).  In  fact,  at  some  universities,  distance  education 
faculty  earn  less  credit  for  teaching  than  their  classroom  counterparts  (Wolcott,  1997).  Expectancy  theory  tells  us 
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that  if  faculty  value  their  rewards,  and  if  they  feel  capable  of  teaching  distance  education  courses,  then  performance 
expectations  and  rewards  will  motivate  them  to  teach  online  (Vroom,  1967).  To  the  extent  that  distance  education  is 
a priority  for  the  institution,  administrators  should  align  distance  education  goals  with  the  university’s  feedback  and 
reward  systems.  An  example  of  this  may  be  to  give  more  weight  to  the  development  and  delivery  of  distance 
education  courses  as  a criteria  for  tenure. 

Structure 

Performance  occurs  within  the  imaginary  boundaries  of  organizational  structures,  standard  operating 
procedures,  and  inter-personal  relationships.  Unlike  goals  and  management  practices,  structure  typically  cannot  be 
changed  quickly  or  easily.  Formal  embodiments  of  structure,  such  as  titles,  operations  manuals,  and  organizational 
charts  can  be  changed  overnight,  however,  informal  structures  (norms  and  values)  are  ingrained  in  the  organization’s 
culture  and  are  therefore  more  difficult  to  transform.  Nevertheless,  sometimes  change  they  must.  With  distance 
education  receiving  more  attention  from  institutions,  governments,  and  students  there  is  greater  pressure  to  increase 
enrollment,  quality,  and  speed.  In  other  words,  distance  education  departments  may  need  to  focus  on  performance 
more  than  ever  before,  even  as  they  expand  to  accommodate  a greater  number  of  courses,  faculty,  and  students. 

Cell  4:  Structuring  the  organization 

The  structure  of  a distance  education  department  relative  to  the  university  or  college  it  serves 
communicates  to  all  members  of  the  institution  the  relative  importance  and  scope  of  distance  education.  Mark 
(1990)  distinguishes  four  types  of  structures,  a distance  learning  program,  unit,  institution,  and  consortium.  With  a 
program,  distance  education  courses  are  developed  entirely  by  the  faculty  of  an  academic  department.  A distance 
learning  unit  is  a separate  entity  operating  inside  of  a college  or  university.  Usually,  the  unit  will  have  its  own  full- 
time staff,  which  is  completely  dedicated  to  developing  distance  education  courses  (e.g.,  Penn  State’s 
WorldCampus).  A distance  learning  institution  is  a wholly  separate  entity  (e.g.,  British  Open  University)  with  its 
own  faculty,  administrative,  and  supporting  units.  Its  sole  purpose  is  the  development  and  delivery  of  distance 
education  courses.  A consortium  is  an  alliance  involving  two  or  more  institutions  or  units,  which  share  design 
and/or  delivery  resources. 

Recommendation  4A:  Align  structure  with  goals  and  resources 

The  structure  selected  depends  on  the  institution’s  goals  and  its  resources.  Wilson  (1998)  writes  that 
“distant  courses  require  three  to  four  times  more  dollars  to  develop  and  three  to  eight  times  more  faculty  (time)  and 
support  personnel  resources”  (p.  3).  If  true,  institutions  that  seek  cost  savings  from  distance  education  should  create 
sufficient  economies  of  scale  to  overcome  the  up-front  fixed  costs  of  developing  and  delivering  distance  education 
courses. 

Sachs  (1999;  p.68)  recommends  the  distance  learning  unit  structure  over  the  program,  citing  several 

benefits: 

• Receives  permanent  status 

• Is  allocated  a budget 

• Has  formal  representation  on  committees 

• Serves  the  entire  college  or  university 

• Pools  distance  education  resources  and  knowledge 

• Scalability  - can  expand  or  contract  as  needed 

• Allows  for  economies  of  scale 

• Allows  for  development  of  complete  degree  programs 

Another  benefit  of  the  unit  structure  is  that  it  enables  distance  education  administrators  the  flexibility  to  set 
goals  that  cross  service  area  boundaries  and  impact  the  entire  institution,  not  just  one  academic  department. 
Downsides  to  this  structure  include  increased  overhead,  possible  jealousy  from  other  service  areas  that  compete  for 
budget  allocations,  and  resentment  from  those  departments  over  their  loss  of  control  over  distance  education 
resources. 

Establishing  a separate  distance  learning  institution  is  one  way  to  avoid  some  of  the  political  and  budgetary 
obstacles  involved  with  internal  units.  An  institution,  acting  as  a subsidiary,  affiliate,  or  independent  of  a resident 
university  or  college  has  its  very  own  budget  and  most  of  its  resources  are  self-contained.  This  enables  it  to  be  more 
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agile,  flexible,  and  responsive  to  changing  student  needs  and  organizational  goals.  Carr  (2000)  reports  that 
universities,  such  as  Cornell,  Temple,  and  NYU,  have  established  for-profit  online  subsidiaries  for  these  same 
reasons. 

Cell  5:  Structuring  the  processes 

Process  structures  are  commonly  operationalized  through  policies,  practices,  and  standard  operating 
procedures.  These  structures  help  a distance  education  unit  maintain  consistency  in  its  workflows  and  outputs, 
document  organizational  learning,  and  accelerate  the  ramp  up  time  for  new  hires.  One  method  for  developing  good 
processes  involves  comparing  current  practices  against  benchmarked  “best  practices”  collected  from  successful 
programs  and  then  identifying  any  “disconnects”  in  the  workflows.  This  process  of  comparison  can  identify 
opportunities  for  improving  the  efficiency  and  effectiveness  of  key  distance  education  processes  such  as 
instructional  design  and  class  facilitation  (Moller,  Benscoter,  & Rohrer-Murphy,  2000). 

Recommendation  5A:  Empower  faculty 

One  benchmark  of  successful  distance  education  processes  is  faculty  involvement  (Camevale,  2000; 

Young,  2000).  Sachs  (1999:  p.68)  points  out  that  by  giving  faculty  a sense  of  ownership  in  the  course  development 
process,  distance  education  units  can: 

• Reduce  faculty  turnover 

• Improve  design  flexibility 

• Encourage  continuous  improvement 

• Increase  the  credibility  of  their  program 

• Create  a direct  pipeline  into  mainstream  activities 

• Eventually  reach  a status  of ‘accepted’  in  the  institutional  culture 

The  challenge  for  distance  education  staff  is  instilling  the  faculty  with  a sense  of  ownership  while  still 
maintaining  some  control  over  the  quality  and  speed  of  the  process.  As  Sachs  (1999)  described,  “Some  faculty  were 
not  adept  at  taking  advantage  of  the  new  technology  tools  at  their  disposal”  (p.70)  and  therefore  require  support 
from  instructional  design  and  technology  experts  to  optimize  their  use  of  computer-mediated  distance  learning 
technologies.  In  order  to  achieve  this  balance  of  faculty  control  and  expert  guidance,  processes  should  be  structured 
in  such  a way  that  faculty,  designers,  and  technologists  work  hand-in-hand  as  a team  to  produce  a course. 
Interdependence  is  critical  in  combating  the  silo  effect,  where  functional  performance  is  maximized  at  the  expense 
of  the  department’s  overall  effectiveness  (Rummler  & Brache,  1990). 

Other  benchmarks  will  raise  similar  issues  of  process  design,  such  as: 

• How  will  decisions  be  made  regarding  the  adoption  of  new  technologies? 

• How  will  current  courses  be  revised  to  work  with  new  technologies? 

• What  supporting  processes  will  be  outsourced? 

• How  will  faculty  be  selected  and  certified? 

• How  will  student  performance  be  assessed? 

• How  are  courses  going  to  be  selected  for  online  delivery? 

Cell  6:  Structuring  the  job 

The  structure  and  organizational  design  of  a distance  education  unit  will  impact  the  performance  of  the 
department  as  a whole  as  well  as  the  effectiveness  of  individual  staff  members  (Moller,  Benscoter,  & Rohrer- 
Murphy,  2000).  Rothwell  (1996)  suggests  that  redesign  efforts  may  be  needed  “if  the  external  environment  becomes 
more  unstable  because  it  is  becoming  more  competitive  or  if  the  internal  environment  becomes  more  unstable  due  to 
changing  work  methods  or  relationships”  (p.228).  Given  the  dynamic  nature  of  external  environments,  distance 
education  administrators  would  do  well  to  design  or  redesign  their  organizations  in  such  a way  as  to  maximize 
flexibility  and  responsiveness. 

Traditionally,  distance  education  was  developed  in  small  teams,  typically  consisting  of  an  author  and  an 
editor.  This  author-editor  model  was  fast  and  inexpensive,  however,  with  neither  of  the  participants  having 
instructional  design  expertise,  courses  often  lacked  instructional  quality  (Moore  & Kearsley,  1996).  With  the  advent 
of  more  complex  technologies  (e.g.,  videoconferencing  and  computer  conferencing)  and  greater  investment  into 
distance  education,  teams  expanded  to  include  producers,  instmctional  designers,  technologists,  graphic  artists, 
librarians,  and  other  specialists.  With  increasing  project  complexity,  higher  stakes,  and  more  specialists  involved. 
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development  time  and  costs  increased  exponentially.  Development  cycles  of  over  one  year  were  not  uncommon 
with  this  course  team  model. 

Recommendation  6A:  Structure  jobs  to  optimize  quality  and  efficiency 

There  are  several  other  types  of  design  models  that  organizations  can  use,  including  functional,  divisional, 
project,  matrix,  and  virtual  design  models.  Traditional  functional  and  divisional  hierarchies  may  lack  the  flexibility 
and  responsiveness  needed  in  today’s  changing  distance  learning  marketplace.  The  project  structure  emphasizes  the 
role  of  project  managers  who  lead  temporary  projects  as  the  need  arises.  This  may  be  suitable  for  fledgling 
programs  that  are  looking  to  broaden  support  for  distance  education,  but  it  does  not  lend  itself  well  to  long-term 
projects  such  as  establishing  entire  online  degree  programs. 

Matrix  Model 

The  matrix  design  offers  some  possibilities  for  organizing  distance  education  efforts.  Distance  education 
workers  would  have  two  immediate  supervisors:  the  lead  for  their  particular  area  of  expertise  (e.g.,  Instructional 
Design  Lead)  and  the  project  managers  they  work  with  to  develop  courses.  This  model  is  frequently  used  by  service 
organizations.  In  fact,  many  courseware  development  firms  are  organized  this  way.  The  design  has  inherent 
flexibility,  responsiveness,  and  attention  to  quality,  which  can  be  critical  in  dynamic,  client-oriented  environments. 
On  the  other  hand,  this  structure  can  often  confuse  staff  members  because  of  its  complexity,  its  often  conflicting 
priorities,  and  added  layer  of  management.  Another  drawback  is  that  by  having  project  managers,  whose  job  it  is  to 
plan,  coordinate,  and  control,  it  may  be  difficult  to  instill  a sense  of  ownership  in  faculty  members. 

Virtual  Model 

The  virtual  design  is  often  seen  as  the  design  of  the  future,  with  its  emphasis  on  temporary  skilled  workers, 
vendors,  external  consultants,  and  suppliers.  This  design  of  tomorrow  is  in  fact  the  design  of  today  for  the  film  and 
television  industries.  Producers  assemble  directors,  writers,  cast  members,  set  designers,  as  well  as  other  support 
resources,  vendors,  and  consultants  as  needed  to  work  together  for  a limited  time.  Often,  there  are  some  full-time 
employees  but  only  as  many  as  absolutely  necessary.  One  of  the  main  drawbacks  of  outsourcing  core  competencies 
is  the  loss  of  benefits  from  organizational  learning.  In  distance  education,  over-use  of  vendors,  external  consultants, 
and  temporary  help  may  not  sit  well  with  faculty  and  certainly  will  do  little  to  build  long-term  relationships  and 
acceptance  within  the  institutional  culture. 

Consultant  Team  Model 

Another  possible  structural  design  is  the  consulting  team  model.  In  this  approach,  faculty  members  lead 
the  development  effort  and  are  assisted  by  instructional  design  and  technology  internal  consultants.  Other  support 
staff  members  are  also  involved  and  may  include  some  vendors,  external  consultants,  and  temporary  workers.  This 
model  can  be  particularly  effective  if  work  processes  are  geared  towards  using  reusable  learning  objects,  templates, 
and  centralized  depositories  of  reference,  content,  and  assessment  materials.  These  would  minimize  the  amount  of 
customization  required  for  each  course  and  reduce  the  need  for  large  full-time  staffs  of  specialists.  Olcott  and 
Wright  (1995)  argue  against  such  faculty -centered  models,  writing  that:  “distance  education  has  both  a responsibility 
and  a unique  opportunity  to  foster  a student -centered  learning  process  responsive  to  the  diversity  of  adult  learners” 
(p.l4).  Striking  a balance  between  student  needs  and  faculty  control  can  be  a difficult  proposition  for  the  distance 
education  staff.  However,  alienating  faculty  by  removing  curriculum  control  and  classroom  autonomy  may,  in  the 
final  analysis,  contribute  little  to  improving  the  learning  experience  for  the  student. 

Management 

The  purpose  of  management  is  to  put  the  right  people,  systems,  and  resources  in  place  to  succeed;  assess 
performance;  provide  feedback;  and  take  action  to  maintain  alignment  with  established  organization,  process,  and 
job  performance  goals.  Key  issues  to  be  addressed  include: 

• Are  there  feedback  systems  that  help  diagnose  performance  deficiencies? 

• Are  the  right  people  in  place  in  sufficient  numbers  and  with  the  right  resources  to  work 
effectively?  In  other  words,  are  staff  members  motivated  to  perform? 

• Are  there  consequences  for  negative  performance  as  well  as  rewards  for  positive  performance 
results? 

• Are  expectations,  roles,  and  standards  clearly  communicated  throughout  the  organization? 
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Cell  7:  Managing  the  organization 

In  managing  the  distance  education  department,  administrators  should  set  milestones,  measure 
performance,  and  ensure  consistent  alignment  with  the  institution’s  mission.  Distance  education’s  contribution  to 
the  organization  should  be  assessed  and  weighed  against  costs  and  organizational  needs.  Currently,  the  proliferation 
of  online  distance  education  courses  is  precariously  perched  on  several  assumptions.  Sachs  (1999)  summarizes 
them,  writing: 

“Distance  education  courses  are  being  developed  because  there  is  a perception  that  there  are  a 
large  number  of  students  to  be  gained,  often  at  the  expense  of  other  institutions;  that  it  will  be 
cheaper  to  serve  students;  or  that  it  will  be  easier  to  teach  because  there  will  not  be  class  meetings. 
(P.77). 

Recommendation  7A:  Conduct  cost  analysis 

Though  the  debate  continues,  there  is  no  conclusive  evidence  to  show  that  online  courses  are  cheaper  or 
easier  to  implement  than  traditional  classroom  instruction.  Therefore,  cost  justifications  for  distance  education 
should  be  used  cautiously,  if  at  all.  Results  in  business  and  academic  settings  indicate  that  distance  learning  efforts 
can  reduce  costs  but  too  often  bring  more  economic  pain  than  gain  (Fomaciari,  1999).  A thorough  cost  analysis  may 
help  administrators  reduce  the  economic  uncertainty  involved  and  should  help  them  set  realistic  goals  for  the 
distance  education  programs.  Whalen  and  Wright  (1999)  offer  a framework  for  cost  analysis  that  can  be  used  to: 
o Predict  the  number  of  students  needed  for  a program  to  break  even 
o Identify  the  fixed  and  variable  costs  involved 

o Identify  realistic  program  life  spans  (time  in  use  before  requiring  revision) 
o Identify  the  opportunity  costs  involved 
o Compare  the  development  costs  for  the  alternatives  available 
Rumble  (1999)  and  Jewett  (2000)  provide  similar  cost  analysis  models. 

Recommendation  7B;  Identify  the  benefits/goals  to  be  achieved 

Distance  education  potentially  offers  significant  benefits  both  to  the  institution  and  its  students.  Figure  2 is 
a partial  list  of  these  benefits.  Though  competition  for  enrollment  and  reduced  costs  are  possible  macro -benefits,  or 
goals,  of  distance  education,  some  schools  are  finding  that  flexibility  is  the  single,  greatest  reason  to  implement  it. 
Schools  in  Australia,  for  example,  have  taken  to  using  the  term  “flexible  learning”  to  describe  the  concurrent  (dual- 
mode) usage  of  distance  learning  and  classroom  instruction.  As  Evans  (1999)  points  out:  “The  viability  of  an 
educational  institution  is  not  just  a matter  of  costs.  It  is  also  a matter  of  making  judgments  about  community  needs 
and  obligations  and  state  or  national  priorities”  (p.41).  Collis  (1999)  describes  the  growing  movement  in  Europe 
towards  improving  flexibility  of  location  as  part  of  a broader  effort  to  individualize  education  through  flexibility  in 
content  selection,  learning  resources,  technology  use,  and  the  amount  and  type  of  communication.  Student-oriented 
goals  such  as  these  can  help  the  university  at-large  to  deal  with  shifting  demographics,  changing  societal  needs  (e.g., 
lifelong  learning),  and  growing  competition  from  non-traditional  educational  and  certifications  programs.  Also, 
making  flexibility  an  organizational  goal  mandates  changes  throughout  the  institution  and  positions  the  distance 
education  program  as  an  important  change  agent  and  resource. 

Cell  8:  Managing  the  processes 

Design  and  delivery  processes  should  be  benchmarked  against  established  Instructional  Systems  Design 
(ISD),  Human  Performance  Technology  (HPT),  and  distance  education  best  practices.  For  example,  the  standards 
developed  by  the  International  Board  of  Standards  for  Training  Performance  and  Instruction  (IBSTPI,  1998)  can 
form  the  basis  for  the  department’s  process  standards  as  well  as  help  in  the  development  of  hiring,  training,  and 
evaluation  programs  for  instructional  designers. 

Recommendation  8A;  Assess  resource  usage 

Managing  the  process  also  means  allocating  the  right  type  and  right  amount  of  resources.  Distance 
education  resources  include  technology  (hardware,  software,  infrastructure),  skills  (ISD  skills,  technical  know-how), 
funding  (for  outsourcing,  updating  systems,  etc.),  material  (e.g.,  not  enough  desks),  and  human  resources  (not 
enough  people  or  time  to  do  the  job,  the  wrong  skill  sets  for  the  job).  In  order  to  make  educated  decisions  about 
resource  allocation,  managers  should  put  systems  into  place  that  assess  resource  usage. 
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Recommendation  8B:  Assess  the  efficiency  of  workflows 

In  a survey  of  faculty  member  attitudes  toward  distance  education,  Rockwell,  Schaer,  Fritz,  and  Marx 
(1999)  found  that  57%  of  faculty  members  viewed  release  time  as  an  incentive  and  ranked  time  requirement  as  their 
number  one  obstacle  (p.6).  With  more  and  more  faculty  being  asked  to  deliver  courses  online,  the  debate  is  under 
way:  does  it  take  more  or  less  effort  and  time  to  develop  and  deliver  a distance  education  course?  A s urvey  of 
distance  education  faculty,  found  that  53%  believed  that  the  distance  courses  required  more  work  than  resident 
classes  (NEA,  2000).  In  a case  study  comparing  a traditional  class  and  an  online  course,  Visser  (2000)  found  that 
development  and  delivery  time  for  the  online  course  significantly  exceeded  those  of  the  traditional  course.  DiBiase 
(2000)  found  contradictory  results  in  a similar  comparison  case  study.  However,  he  did  observe  that  the  frequency 
of  contact  for  online  coursers  is  higher  (5  times  per  week)  than  traditional  courses  (4  times  per  week).  Considering 
the  narrow  margin  of  difference  in  the  NEA  study  (44%  felt  it  takes  less  work)  and  conflicting  results  of  the  case 
studies,  this  may  be  an  issue  that  cannot  be  resolved  in  generic  terms.  Rather,  each  distance  education  department 
should  formalize  processes  for  assessing  the  amount  of  time  needed  to  develop  and  deliver  courses  as  well  as  the 
relative  quality  of  those  courses.  In  light  of  those  results,  a more  accurate  evaluation  can  be  formed  and,  when 
necessary,  steps  can  be  taken  to  reduce  the  amount  of  time  and  effort  required  of  faculty.  Though  faculty  release 
time  is  important  from  the  standpoint  of  faculty  morale,  it  is  even  more  critical  from  the  standpoint  of  quality. 
Faculty  and  the  specialists  who  support  them  need  to  have  sufficient  time  for  design,  development,  and  delivery. 
There  should  also  be  formative  and  summative  evaluation  of  courses  to  help  team  members  adjust  their  outputs  and 
continuously  improve  their  courses.  As  Moore  (2000)  points  out:  “it’s  not  a question  of  having  less  work  or  more 
work...,  but  (rather)  getting  better  quality  out  of  the  same  effort  as  a result  of  that  effort  being  more  effectively 
organized  and  applied”  (p.5). 

Cell  9:  Managing  the  job/workers 

Managing  a distance  education  staff  can  be  a challenging  job.  Not  long  ago,  distance  education  courses 
were  created  by  small  teams  of  two  to  three  people.  Today,  given  the  complexity  of  technology-based  instruction, 
distance  education  programs  rely  on  a variety  of  specialists.  How  will  these  specialized  workers  be  managed?  Who 
is  best  qualified  to  manage  them  and  evaluate  their  performance?  Given  the  specialized  nature  of  their  work,  will 
staff  members  need  to  be  evaluated  against  performance  standards  for  their  given  fields  or  should  they  be  held 
accountable  for  more  general  team  performance  standards  such  as  quality,  speed,  and  costs?  How  will  staff 
members  receive  feedback  for  their  individual  and/or  team  performance?  Thomas  Gilbert  (1978)  points  out  that 
“more  than  half  of  the  problems  of  human  competence  can  be  traced  back  to  inadequate  (feedback)  data”  (p.l  79). 

Recommendation  9A:  Performance  standards 

Performance  standards  act  as  an  anvil  against  which  performance  is  shaped.  Without  standards,  it  is 
difficult,  perhaps  impossible,  to  distinguish  consistently  good  performance  from  bad,  skilled  workers  from  poor 
ones,  and  best  practices  from  sub-optimal  procedures.  Performance  standards  provide  an  objective  basis  for 
performance-oriented  feedback  mechanisms. 

Recommendation  9B:  Feedback  mechanisms 

Rothwell  (1995)  found  that  feedback-related  strategies  make  up  3 of  the  top  4 most  frequently  encountered 
human  performance  improvement  interventions.  This  reflects  the  paramount  importance  of  information  and 
communication  in  an  organization.  Feedback  helps  with  “goal  refinement,  documentation,  determination  of  impact, 
and  program  improvement”  (Hawkes,  1996:  p.31).  Distance  education  administrators  should  establish  mechanisms 
for  clear  and  timely  communication  of  roles,  responsibilities,  performance  expectations,  performance  results,  and 
any  information  that  is  pertinent  to  the  work  of  developing  distance  education  courses.  Without  such  information, 
performance  suffers.  On  the  other  hand,  “significant  performance  gains  can  be  achieved  by  improving  the  flow  of 
information  about  the  work”  (Rothwell,  1996:  p.l  86).  Sources  of  performance  feedback  include  student  comments 
about  their  experiences,  help  desk  logs,  budget-to-actual  reports,  and  peer  evaluations. 

Conclusion 

Distance  education  in  postsecondary  institutions  is  seeing  dramatic  growth.  Is  this  upsurge  an  anomaly  or 
is  distance  education  a force  that  will  stand  the  test  of  time  and  succeed  in  bringing  about  profound  changes  in  the 
way  we  teach  and  learn?  Though  we  cautiously  subscribe  to  the  latter,  much  will  depend  on  distance  education’s 
ability  to  deliver  results,  not  just  in  terms  of  bottom-line  returns  for  institutions  but  also  in  terms  of: 
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• Fruitful  educational  experiences  for  students 

• Perceptions  that  employers  have  of  students  with  online  degrees 

• True  access  across  socio-economical  classes  and  regions 

• Adoption  of  innovative  instructional  approaches 

This  paper  has  examined  the  organizational  alignment  of  distance  education  programs.  Applying 
performance  improvement  strategies,  recommendations  have  been  presented  for  communicating  goals, 
benchmarking  processes,  measuring  results,  providing  timely  feedback  at  all  levels,  and  rewarding  performance.  A 
more  complete  summary  is  provided  in  Figure  5.  Without  a clear  vision  of  performance  goals,  without  plans  for 
making  the  vision  a reality,  and  without  the  structures  and  systems  to  manage  performance,  distance  education 
administrators  compromise  organizational  results  and  may  realize  only  too  late  that  performance  does  not  just 
happen. 
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Goals 

Structure 

Management 

Organization 

Cell  1 - Establish 
balanced  scorecard  with 
goals  that  are  aligned  with 
the  institution’s  mission. 

Cell  4 - Determine  the  size 
and  scope  of  the  distance 
education  organization. 

Cell  7 - Assess  student  and 
organizational  needs. 
Evaluate  results  through 
cost^eneflt  analysis. 

Process 

Cell  2 - Benchmark 
processes  by  identifying 
best  practices  for 
instructional  design, 
development,  and 
delivery  of  online  courses. 

Cell  5 - Use  benchmarks 
to  set  process  standards 
and  develop  policies  and 
practices  to  support 
performance. 

Cell  8 - Evaluate  processes 
to  determine  standard 
resource  usage  levels.  Use 
feedback  to  improve 
efficiency  and  quality. 

Job/  Performer 

Cell  3 - Identify  roles 
needed,  responsibilities, 
and  outputs.  Tie 
performance  goals  to 
reward  system. 

Cell  6 - Develop  a 
workflow  design  that 
supports  best  practices. 

Cell  9 - Set  employee 
performance  standards, 
measure  results,  and  use 
feedback  to  improve 
performance. 
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Abstract 

This  study  investigated  the  effects  of  personalized  instruction  on  the  achievement  and  attitudes  of 
Taiwanese  students  on  two-step  mathematics  word  problems.  A total  of  136  fourth-graders  in  a Taiwanese 
public  school  participated  in  the  study.  Subjects  initially  completed  a Student  Survey  on  which  they  chose 
their  favorite  foods,  sports,  stores,  classmates,  and  other  selections.  The  most  popular  items  were  then 
used  to  create  personalized  math  word  problems  for  the  pretest,  personalized  instructional  program,  and 
posttest.  Subjects  were  blocked  by  ability  based  on  their  pretest  scores  and  were  randomly  assigned  within 
ability  levels  to  either  a personalized  or  non-personalized  version  of  the  print-based  instructional  program. 
After  finishing  the  program,  subjects  completed  a student  attitude  survey  and  the  posttest.  A repeated- 
measures  ANOVA  revealed  that  subjects  in  the  personalized  treatment  made  significantly  greater  pretest- 
to-posttest  gains  than  those  in  the  non-personalized  treatment.  Subjects  also  performed  significantly  better 
on  the  personalized  pretest  and  posttest  problems  than  on  the  non-personalized  problems.  Personalized 
subjects  and  higher-ability  students  both  had  significantly  more  positive  attitudes  toward  the  instructional 
program  than  their  non-personalized  and  lower-ability  counterparts. 

Introduction 

Data  from  the  National  Assessment  of  Educational  Progress  (1992a,  1992b)  indicate  that  mathematics  word 
problems  are  difficult  for  students  at  all  age  levels  in  elementary  and  secondary  schools.  A major  cause  of  the 
difficulty  appears  to  be  the  students’  inability  to  convert  the  problems  into  the  math  operations  that  must  be 
performed  to  solve  them  (Hart,  1996).  Some  researchers  have  also  noted  that  lack  of  familiarity  with  word  problem 
structures  may  also  contribute  to  poor  student  performance  (Mayer,  1982;  Rosen,  1984). 

Personalizing  mathematics  word  problems,  such  as  incorporating  personal  background  information  into  the 
problem  content,  can  lead  to  improvements  in  performance  (Anand  & Ross,  1987;  Davis  -Dorsey,  Ross,  & Morrison, 
1991;  Lopez  & Sullivan,  1991,  1992).  Anand  and  Ross  (1987)  tested  the  effect  of  using  computer-assisted 
instruction  to  personalize  mathematics  instruction  for  elementary  school  children.  Students  who  received 
personalized  instruction  scored  significantly  higher  on  math  word  problems  involving  rule  recognition  and  transfer 
than  those  whose  instruction  was  not  personalized.  The  authors  claimed  that  personalized  contexts  increased 
students’  comprehension  and  motivation  by  helping  them  interpret  important  information  in  the  problem  statement. 
Davis -Dorsey  et  al.  (1991)  found  that  both  second-grade  and  fifth-grade  students  made  significant  achievement 
gains  from  rewording  and  personalization  of  the  context  in  math  word  problems.  In  two  separate  studies  with  rural 
eighth-grade  Hispanic  American  students,  Lopez  and  Sullivan  (1991,  1992)  found  significant  overall  achievement 
effects  for  personalization  on  one-step  and  two-step  mathematics  word  problems. 

Several  studies  have  found  that  student  attitudes  are  more  positive  when  student  interests  and  preferences 
are  incorporated  into  instruction  in  order  to  personalize  it.  Ross  and  his  colleagues  (Ross,  1983;  Ross,  McCormick, 

& Krisak,  1986;  Ross,  McCormick,  Krisak,  & Anand,  1985)  employed  personalization  in  a series  of  adaptive 
instruction  studies.  Favorable  attitude  results  were  obtained  when  preservice  teachers  received  education-related 
materials  and  nursing  students  received  medical-related  materials,  and  poorer  results  were  attained  when  each  group 
received  the  other’s  materials.  Herndon  (1987)  found  that  high  school  students  who  received  instruction  based  on 
common  group  interests  had  significantly  more  favorable  attitudes  and  higher  return -to-task  motivation  than 
students  whose  instruction  was  not  interest-based.  Cordova  and  Lepper  (1996)  and  Hart  (1996)  also  found  more 
favorable  attitudes  or  motivation  toward  personalized  than  toward  non-personalized  instruction. 

Researchers  have  offered  two  theory -based  explanations  for  the  effectiveness  of  personalized  instruction  in 
studies  where  it  has  yielded  better  results  than  non-personalization.  One  is  that  students’  greater  familiarity  with 
personalized  problem  situations  and  content  may  enable  them  to  solve  problems  more  easily  by  reducing  their 
cognitive  load  (Lopez  & Sullivan,  1991,  1992;  Miller  & Kulhavy,  1991).  This  position  is  supported  by  D’Ailly, 
Simpson,  and  MacKinnon’s  (1997)  statement  that  “s  elf-referencing  facilitates  general  encoding  processes  and 
decreases  the  load  on  working  memory  during  problem  solving.” 
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The  second  explanation  is  based  on  interest  theory  (Mayer,  1998).  Mayer  notes  that  students  exert  more 
effort  and  are  more  successful  solving  problems  that  interest  them  than  problems  that  do  not.  Several  researchers 
have  cited  greater  student  interest  and  motivation  as  reasons  for  better  performance  under  personalized  instruction 
(Cordova  & Lepper,  1996;  Lopez  & Sullivan,  1992;  Ross  & Anand,  1987).  The  “reduced  cognitive  load”  and 
“increased  interest”  explanations  appear  to  be  compatible  with  one  another  rather  than  being  alternative  or 
competing  explanations. 

The  present  study  was  designed  to  address  the  unanswered  questions  from  the  Ku  and  Sullivan  (2000) 
research  and  to  investigate  the  stability  of  the  other  findings  from  that  study  and  other  research.  The  likelihood  of  a 
ceiling  effect,  which  confounded  interpretation  of  the  personalization  versus  non -personalization  issue  in  the  earlier 
research,  was  reduced  by  conducting  this  study  with  fourth-grade  students  instead  of  fifth  graders  as  in  the  earlier 
study.  This  study  was  also  conducted  with  136  students,  or  nearly  twice  as  many  as  the  earlier  one,  to  increase  the 
power  of  the  statistical  analyses.  These  changes  were  designed  to  provide  a clearer  answer  to  the  question  of 
whether  instruction  incorporating  personalized  math  word  problems  would  yield  significantly  better  student 
performance  than  instruction  using  only  non-personalized  items. 

The  question  of  whether  merely  stating  test  items  in  personalized  form  would  yield  better  performance  than 
stating  them  in  non-personalized  form,  irrespective  of  the  type  of  instruction  students  receive,  was  addressed  by 
including  an  equal  nurrber  of  personalized  and  non-personalized  items  on  the  pretest,  as  well  as  on  the  posttest. 
Including  personalized  and  non-personalized  items  on  the  pretest,  as  well  as  on  the  posttest,  in  the  present  study  was 
designed  to  permit  more  direct  and  stable  comparisons  of  the  effects  of  the  two  types  of  test  items  alone  and  in 
combination  with  the  instructional  treatments. 

The  present  study  investigated  the  effects  of  two  levels  of  group  personalization  (personalized,  non- 
personalized)  on  the  achievement  of  fourth-grade  Taiwanese  students  on  two-step  mathematics  word  problems. 
Mathematics  ability  level,  as  determined  by  scores  on  the  pretest  administered  prior  to  the  instructional  phase  of  the 
study,  was  also  included  as  a variable  because  it  is  an  important  factor  in  mathematics  achievement  and  because  of 
differential  findings  by  ability  level  in  earlier  research  on  personalization  (Dwyer,  1996;  Ku  & Sullivan,  2000; 

Lopez  & Sullivan,  1991). 

The  primary  research  questions  for  the  study  were  as  follows: 

1.  Does  personalization  of  instruction  increase  the  achievement  of  Taiwanese  students  on  mathematics 
word  problems? 

2.  Does  personalization  of  instruction  have  a differential  effect  on  the  performance  of  higher-ability  and 
lower-ability  students  on  mathematics  word  proble  ms? 

3.  Do  Taiwanese  students  perform  better  on  personalized  word  problems  than  on  non-personalized 
problems  irrespective  of  the  type  of  instruction  (personalized  or  non-personalized)  they  receive? 

4.  Does  personalization  of  mathematics  word  problems  influence  student  attitudes  toward  the  instruction 
on  these  problems? 

Method 

Subjects 


A total  of  1 36  fourth-grade  Taiwanese  students  from  four  classes  taught  by  different  teachers  at  a public 
elementary  school  in  Taiwan  participated  in  this  study.  The  school  is  located  in  a mid-level  income  and 
socioeconomic  area  in  Fengyuan,  a city  with  a population  of  approximately  160,000  people. 

Materials 

Student  Survey.  A 20-item  student  survey  was  used  to  determine  the  personal  backgrounds  and  interests  of 
the  participants.  Topics  included  the  names  of  the  students'  favorite  places,  activities,  sports,  friends,  convenience 
stores,  foods,  and  so  forth.  Students  wrote  in  two  favorite  responses  for  each  survey  item. 

The  survey  was  administered  one  week  prior  to  the  pretest.  Responses  to  each  survey  item  were  tabulated 
by  the  experimenter  and  then  used  to  design  the  personalized  version  of  the  instructional  program  and  the  tests. 

Instructional  Program  Two  parallel  versions  of  an  instructional  program  on 

two-step  math  word  problems  were  developed  in  print  form  in  Chinese.  Taiwanese  students  learn  addition 
and  subtraction  in  the  third  grade  and  multiplication  and  division  in  the  fourth  grade.  The  word  problems  in  the 
instructional  program  and  the  test  items  were  taken  directly  from  the  fourth-grade  and  fifth-grade  mathematics 
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textbooks  used  by  the  participants.  The  program  was  administered  after  the  students  had  studied  one-step 
multiplication  and  division,  and  it  covered  two-step  processes  that  they  had  not  yet  formally  studied. 

The  non-personalized  version  of  the  instructional  program  was  written  first  and  included  standard  problem 
types  from  the  students’  math  textbook.  The  personalized  version  was  then  written  by  incorporating  the  most 
popular  referents  (places,  foods,  sports,  names,  etc.)  from  the  Student  Survey  into  the  previously  non-personalized 
version. 

An  example  of  a practice  item  in  personalized  and  non-personalized  form  is  provided  below.  The  example 
item  requires  using  multiplication  followed  by  division. 

Non-personalized:  The  teacher  has  2 dozen  cans  of  soft  drink  to  be  shared  equally  by  8 students. 

. How  many  cans  of  soft  drink  does  each  student  get? 

Personalized:  The  teacher,  Ms.  Sue,  has  2 dozen  cans  of  milk  tea  to  be  shared  equally  by  8 

students.  How  many  cans  of  milk  tea  does  each  student  get? 

The  instructional  program  covered  procedures  for  solving  two-step  word  problems  involving  four  different 
combinations  of  multiplication  and  division  operations  (multiply-multiply,  multiply-divide,  divide-multiply,  divide- 
divide)  with  whole  numbers.  A four-step  strategy  based  on  the  work  of  Enright  and  Choate  (1993)  was  incorporated 
into  the  instructional  program  for  both  treatments.  The  four  steps,  which  had  not  been  taught  to  the  students 
previously,  were;  1 . Read  the  question.  2.  Think  through  the  problem.  What  must  be  found  out?  What  steps  are 
involved?  3.  Choose  the  steps  and  do  the  math.  4.  Check  your  answer. 

Instruction  for  solving  each  of  the  four  types  of  problems  contained  the  four-part  Enright  and  Choate 
strategy  with  two  worked  examples  for  each  problem  type.  After  the  examples  for  each  problem  type,  the 
instructional  program  contained  two  practice  problems  for  the  students  to  work.  Each  pair  of  practice  problems  was 
placed  on  a page  with  enough  open  space  to  allow  students  to  work  out  the  problems.  When  students  completed  the 
instructional  program,  the  experimenter  collected  the  programs  and  scored  the  practice  problems.  On  the  following 
day,  the  experimenter  conducted  a review  in  which  he  provided  the  answers  to  the  practice  problems  and  the 
explanations  for  them.  All  materials  used  in  the  study  were  in  Chinese. 

Procedures 

The  Student  Survey  was  administered  to  participants  two  weeks  prior  to  the  treatment.  The  most  popular 
choices  (places,  foods,  sports,  and  so  forth)  from  the  survey  were  subsequently  used  to  convert  the  non-personalized 
problems  into  the  personalized  content  for  the  pretest,  the  instructional  program,  and  the  posttest.  The  pretest  was 
administered  one  week  after  students  filled  out  the  Student  Survey. 

After  the  pretests  had  been  scored,  the  subjects  were  blocked  within  each  class  by  their  pretest  scores  into 
higher-ability  and  lower-ability  groups,  and  were  assigned  within  blocks  to  either  the  personalized  or  the  non- 
personalized  version  of  the  instructional  program.  Sixty-eight  subjects  each,  34  higher-ability  and  34  lower-ability, 
were  in  the  personalized  treatment  and  the  non-personalized  treatment.  The  mean  pretest  score  was  9.35  (SD  = 

1 .65)  for  the  68  higher-ability  subjects  and  3.74  (SD  = 2. 1 1 ) for  the  68  lower-ability  subjects. 

The  experimental  part  of  the  study  took  place  over  three  40-minute  class  periods  on  three  different  days  one 
week  after  the  pretest.  The  experimenter  served  as  the  instructor  for  all  treatments  in  regularly  scheduled  math 
classes  in  four  different  classrooms.  The  experimenter  read  instructions  to  all  students  and  told  them  that  they 
would  be  helping  with  the  development  of  a new  math  program  in  Taiwan  and  that  they  should  try  their  best  to  learn 
and  to  solve  the  problems.  On  the  first  day,  subjects  completed  the  instructional  part  of  the  program.  On  day  two, 
the  experimenter  went  over  the  eight  practice  problems  with  subjects  and  wrote  the  answers  on  the  blackboard. 

After  this  review,  students  filled  out  the  student  attitude  survey  and  teachers  completed  the  teacher  attitude  survey. 
On  the  final  day,  subjects  took  the  posttest. 

Criterion  Measures 

Pretest.  A total  of  24  problems  were  developed,  in  both  personalized  and  non-personalized  forms,  for  the 
pretest  and  posttest.  Twelve  items,  three  from  each  of  the  four  combinations  of  multiplication  and  division  math 
operations,  were  randomly  assigned  to  the  pretest  and  the  remaining  twelve  were  assigned  to  the  posttest.  Thus, 
both  the  pretest  and  the  posttest  contained  twelve  two-step  math  word  problems  involving  whole  numbers.  The 
problems  on  the  pretest  were  in  random  order  within  each  problem  type  and  those  on  the  posttest  were  in  the  same 
order  by  problem  type.  Each  test  consisted  of  three  problems  involving  a multiplication  operation  followed  by  a 
second  multiplication  operation  (multiply-multiply),  three  problems  involving  multiplication  followed  by  division 
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(multiply -divide),  three  problems  involving  division  followed  by  multiplication  (divide-multiply),  and  three 
problems  involving  division  followed  by  a second  division  operation  (divide-divide).  . 

The  pretest  was  constructed  and  named  as  two  different  forms,  Form  A and  Form  B.  The  total  of  24  items 
were  randomly  assigned  as  12  pretest  and  12  posttest  items  as  described  above.  In  Form  A,  problems  one  through 
six  were  written  as  non-personalized  problems  and  problems  seven  through  twelve  were  written  as  personalized 
problems.  In  Form  B,  the  non-personalized  problems  fro  m Form  A (problems  one  to  six  on  Form  A)  were 
converted  to  the  personalized  problems  seven  through  twelve,  and  the  personalized  problems  from  Form  A 
(problems  seven  through  twelve  on  Form  A)  were  converted  to  non-personalized  problems  one  through  six.  Thus, 
the  same  12  items  appeared  in  both  personalized  and  non-personalized  forms  across  the  two  test  versions,  with  the 
six  items  in  non-personalized  form  as  items  1-6  on  each  form  and  the  six  personalized  items  as  items  7-12  on  each 
form.  The  answer  to  each  problem  was  scored  as  correct  only  when  the  correct  final  answer  to  the  problem  was 
given.  The  KR-20  reliability  coefficient  was  .83  for  the  pretest. 

Posttest.  As  with  the  pretest,  posttest  problems  were  developed  and  administered  in  two  forms.  The  first 
six  problems  on  Form  A of  the  posttest  were  in  non-personalized  form  and  the  second  six  problems  were  in 
personalized  form.  The  first  six  problems  on  Form  B of  the  posttest  were  items  seven  to  twelve  from  Form  A in 
non-personalized  form  and  the  final  six  problems  on  Form  B (Items  7-12)  were  items  one  to  six  from  Form  A in 
personalized  form.  Thus,  the  problems  on  Form  A and  Form  B of  both  the  pretest  and  the  posttest  consisted  of  six 
non-personalized  items  followed  by  six  personalized  items.  Like  the  answers  on  the  pretest,  each  answer  on  the 
posttest  was  scored  as  correct  or  incorrect  only  based  on  the  final  answer.  The  KR-20  reliability  coefficient  was  .87 
for  the  posttest. 

Subjects  who  received  Form  A on  the  pretest  received  Form  B on  the  posttest,  and  those  who  received 
Form  B on  the  pretest  received  Form  A on  the  posttest.  The  overall  mean  proportions  correct  for  all  subjects  across 
both  tests  were  .68  for  Form  A and  .69  for  Form  B. 

Student  Attitude  Survey.  A ten-item  attitude  survey  served  as  the  criterion  measure  for  assessing  the 
students’  attitudes  and  motivation.  Eight  of  the  ten  items  were  four-choice  Likert -type  questions  that  assessed 
student  attitudes  and  continuing  motivation  toward  the  instruction.  These  items  dealt  with  such  matters  as  how 
interesting  and  how  easy  the  instructional  programs  was,  how  much  students  learned  from  it,  whether  students  could 
do  two-step  problems  well,  and  whether  they  would  like  to  do  more  problems  like  those  in  the  program.  Responses 
to  these  eight  items  were  assigned  a score  of  4 for  the  most  positive  response  and  a score  of  1 for  the  least  positive 
response.  The  two  remaining  items  were  open-ended  questions  dealing  with  student  likes  and  dislikes  about  the 
instructional  program.  The  KR-20  reliability  coefficient  for  the  eight  Likert -type  items  was  .70. 

Teacher  Attitude  Survey.  A seven -item  teacher  survey,  consisting  of  six  four-choice  Likert -type  items  and 
one  open-ended  question,  was  used  to  assess  teacher  attitudes  toward  the  instructional  program  and  the 
personalization  strategy.  Items  dealt  with  topics  such  as  the  appropriateness  of  the  program,  its  quality,  whether  it 
helped  students  learn,  the  effectiveness  of  personalization,  and  whether  the  teachers  liked  personalization.  Teacher 
responses  to  the  Likert -type  items,  like  those  of  the  students,  were  scored  from  4 (most  positive)  to  1 (least  positive). 

Data  Analysis 

The  data  analysis  for  student  achievement  was  a 2 (Treatment:  Personalization  and  Non-personalization)  x 
2 (Ability  Level:  Higher  Ability  and  Lower  Ability)  x 2 (Test  Occasion:  Pretest  and  Posttest)  x 2 (Problem  Type: 
Personalized  and  Non-personalized  problems)  repeated-measures  univariate  analysis  of  variance  (ANOVA). 
Treatment  and  ability  level  were  between-subjects  variables  in  the  analysis  and  test  occasion  and  problem  type  were 
within -subjects  variables.  Attitude  data  were  analyzed  using  a 2 (Treatment)  x 2 (Ability  Level)  x 8 (Survey  Items) 
multivariate  analysis  of  variance  (MANO  VA)  for  the  overall  survey  means. 

Results 


The  results  are  reported  in  this  section  for  achievement,  student  attitudes,  and  teacher  attitudes. 
Achievement 

The  pretest  and  posttest  data  for  the  two  levels  of  personalization,  the  two  ability  levels,  the  two  test 
occasions,  and  the  two  problem  types  are  shown  in  Table  1 . The  achievement  data  for  each  variable  in  the  four- 
factor  design  are  discussed  below. 
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Treatment.  For  personalization  level,  the  mean  overall  scores  across  the  two  test  occasions  were  8.45 
(70%)  for  the  personalized  subjects  and  7.93  (66%)  for  the  non-personalized  subjects.  ANOVA  revealed  that  the 
difference  in  mean  scores  between  the  two  personalization  levels  was  not  statistically  significant,  F(l,  132)  = 2.37, 
MSE  = 3.80,  £ = . 126.  However,  the  difference  for  treatment  in  this  analysis  was  reduced  due  to  the  fact  that  the 
pretest  and  posttest  scores  were  combined  for  each  of  the  two  levels  of  personalization  because  of  the  repeated- 
measures  design.  The  mean  scores  by  personalization  level  on  the  pretest  (that  is,  prior  to  the  treatment)  were 
identical  for  the  two  levels:  6.54  items  correct  (55%)  for  the  personalized  treatment  and  6.54  (55%)  for  the  non- 
personalized  treatment.  In  contrast,  the  mean  posttest  scores  by  treatment  were  10.35  (86%)  for  the  personalized 
treatment  and  9.32  (78%)  for  the  non-personalized  treatment. 

The  differential  pattern  of  identical  scores  for  personalization  level  on  the  pretest,  but  a higher  score  for 
personalization  than  for  non-personalization  on  the  posttest,  was  reflected  in  a significant  treatment  by  test  occasion 
interaction,  F(l,  132)  = 8.27,  MSE  = 1.09,  £ < .01,  = .06.  A post  hoc  paired-samples  t test  revealed  that  the 

posttest  score  of  10.35  for  the  personalized  treatment  was  significantly  higher  than  the  score  of  9.32  for  the  non- 
personalized  treatment,  t(67)  = 3.01,  £ < .01 . 


Table  I 

Mean  Scores  by  Treatment,  Ability  Level,  Test  Occasion,  and  Problem  Type 

Problem  Type 

Personalized  Non-personalized 


Test  Occasion 
and  Treatment 

Higher  ability 

Lower  ability 

Higher  ability 

Lower  ability 

Pretest 


Personalized 


M 

4.79 

1.94 

4.71 

1.65 

SD 

(0.98) 

(1.28) 

(1.09) 

(1.28) 

Non-personalized 

M 

4.85 

2.18 

4.35 

1.71 

SD 

(0.96) 

(1.24) 

(0.98) 

(1.22) 

Sub -totals 

M 

4.82 

2.06 

4.53 

1.68 

SD 

(0.96) 

(1.26) 

(1.04) 

(1.24) 

Posttest 

Personalized 

M 

5.82 

4.59 

5.85 

4.44 

SD 

(0.46) 

(1.84) 

(0.44) 

(1.74) 

Non-personalized 

M 

5.68 

3.94 

5.35 

3.68 

SD 

(0.59) 

(1.69) 

(0.95) 

(1.80) 

Sub -totals 

M 

5.75 

4.27 

5.60 

4.06 

SD 

(0.53) 

(1.78) 

(0.78) 

(MO) 

Overall  mean  scores  by  test  occasion  and  variables: 


Pretest  Personalized 
Non-personalized 

Treatment 
Ability 
= 6.54 
= 6.54 

Higher  = 
Lower  = 

9.35 

3.74 

Problem  Type 

Personalized 

Non-personalized 

= 3.44 
= 3.10 

Posttest  Personalized 
Non-personalized 

= 10.35 
= 9.32 

Higher  = 
Lower  = 

11.35 

8.32 

Personalized 

Non-personalized 

= 5.01 
= 4.83 
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Overall  Personalized 
Non-personalized 


8.45 

7.93 


Higher  = 10.35 
Lower  = 6.03 


Personalized 

Non-personalized 


= 4.23 
= 3.97 


Note.  Total  possible  score  equals  12  items  correct  on  each  test  for  treatment,  ability  level,  and  test 
occasion,  and  six  items  correct  for  each  problem  type. 

Ability  Level.  Higher-ability  students  outscored  lower-ability  students  across  the  two  tests,  10.35  (86%)  to 
6.03  (50%).  This  difference  for  ability  level  was  statistically  significant,  F(l,  132)  =167.49,  MSE  = 3.80,  p < .001, 
ri^  = .56. 

The  2 X 2 X 2 X 2 ANOVA  also  yielded  a significant  ability  level  by  test  occasion  interaction,  F(l,  132)  = 
52.30,  MSE  = 1.09,  p < .001,  r|^  = .28.  This  interaction  reflected  the  fact  that  higher-ability  students  improved  less 
from  pretest  to  posttest  than  lower-ability  students.  Higher-ability  subjects  had  mean  scores  of  9.35  (78%)  on  the 
pretest  and  1 1.35  (95%)  on  the  posttest,  an  improvement  of  2.00  items  correct,  whereas  lower-ability  subjects  had 
mean  scores  of  3.74  (31%)  on  the  pretest  and  8.32  (69%)  on  the  posttest,  an  improvement  of  4.58  items  correct. 

Test  Occasion.  The  mean  scores  for  test  occasion  were  6.54  (55%)  for  the  pretest,  and  9.84  (82%)  for  the 
posttest,  a mean  pretest-to-posttest  increase  of  3.30  items  correct.  This  difference  was  statistically  significant,  F(l, 
132)  = 338.85,  MSE  = 1.09,  p < .001,  = .72.  The  two  interactions  that  involved  test  occasion,  treatment  by  test 

occasion  and  ability  by  test  occasion,  were  reported  above. 

Problem  Type.  The  overall  mean  scores  for  problem  type  were  4.23  (71%)  for  the  six  personalized 
problems  and  3.97  (66%)  for  the  six  non-personalized  problems , a statistically  significant  difference,  F(l,  132)  = 
17.63,  MSE  = .51,  p < .001,  ri^  = .12.  The  ANOVA  also  yielded  a significant  two-way  interaction  for  treatment  by 
problem  type,  F(l,  132)  = 4.66,  MSE  = .51,  p <.05,  = .03.  This  interaction  reflected  the  fact  that  there  was  a 

rather  small  difference  in  the  scores  of  subjects  in  the  personalized  treatment  on  the  personalized  and  non- 
personalized  problems  (4.29,  or  72%,  on  personalized  items  and  4.16,  or  69%,  on  non-personalized  items),  and  a 
larger  difference  in  the  scores  of  non-personalized  subjects  on  these  items  (4.16,  or  69%,  on  personalized  items  and 
3.77,  or  63%,  on  non-personalized  items).  Post  hoc  paired-samples  t tests  of  the  scores  contributing  to  this 
interaction  revealed  that  the  mean  score  of  3.77  for  non-personalized  treatment  on  non-personalized  items  was 
significantly  lower  at  the  p < .01  level  than  the  means  of  each  of  the  other  three  groups. 

Both  personalized  and  non-personalized  problems  were  included  in  the  pretest  in  order  to  determine 
whether  students  would  perform  better  on  personalized  word  problems  than  on  non-personalized  problems  prior  to 
the  instruction.  The  significant  main  -effect  for  problem  type  across  test  occasions  indicates  that  subjects  did 
perform  better  on  personalized  items  than  on  non-personalized  items  both  before  and  after  they  received  instruction. 
Paired -sample  t tests  of  the  difference  between  scores  on  personalized  and  non-personalized  items  on  the  pretest  and 
again  on  the  posttest  confirmed  that  significant  achievement  differences  favoring  personalized  items  occurred  on 
both  occasions. 

Student  Attitudes 

The  mean  attitude  scores  by  treatment  and  ability  level  for  subjects’  responses  to  the  eight  statements  on 
the  four-point  Likert -type  attitude  survey  administered  after  completion  of  the  instructional  program  are  shown  in 
Table  2.  Responses  were  scored  as  4 for  the  most  positive  response  to  1 for  the  most  negative  response. 

The  overall  mean  score  across  the  eight  Student  Attitude  Survey  items  was  3.42,  a favorable  rating 
indicating  agreement  with  positive  statements  about  the  instructional  program.  The  three  highest-rated  statements 
on  the  survey  were  “I  learned  a lot  from  this  program”  (M  = 3.7 1),  “It  is  important  to  know  how  to  solve  two-step 
math  problems”  (M  = 3.68),  and  “I  would  like  to  do  more  math  word  problems  like  the  ones  in  the  program”  (M  = 
3.55).  The  lowest-rated  statement  was  “I  am  able  to  do  two-step  math  problems  well”  (M  = 3.12). 

The  data  in  Table  2 were  analyzed  using  a 2 (treatment)  x 2 (ability)  x 8 (survey  items)  MANOVA  to  test 
for  significant  differences.  A significant  overall  effect  across  the  eight  items  was  obtained  for  treatment,  (M  = 3.52 
for  personalization  and  3.31  for  non-personalization),  F(8,  127)  = 7.10,  MSE  = .13,  £ < .001,  = .08,  and  for 

ability,  (M  = 3.54  for  higher-ability  students  and  3.29  for  lower-ability  students),  F(8,  127)  = 5.72,  MSE  = .13,^< 
.001,  = .1 1.  The  treatment  by  ability  level  interaction  across  the  eight  items  was  not  statistically  significant. 

Univariate  analyses  on  the  eight  survey  items  by  personalization  level  revealed  significantly  more  positive 
attitudes  on  four  of  the  items  for  subjects  in  the  personalized  treatment  than  for  those  in  the  non-personalized 
treatment.  Students  in  the  personalized  treatment  had  significantly  more  favorable  scores  at  the  £ < .001  level  on  the 
items:  “This  program  was  interesting”  (M  = 3,62  for  personalization  and  3.24  for  non-personalization,^r|^  = .09), 
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“This  program  was  easy”  (M  = 3.50  for  personalization  and  2.85  for  non-personalization,  r|^  = .19),  and  “This 
program  had  many  familiar  persons,  places,  and  things”  (M  = 3.62  for  personalization  and  3.13  for  non- 
personalization, t|  = .11),  and  at  the  .05  level  on  the  item  “I  would  like  to  do  more  math  word  problems  like  the 
ones  in  the  program”  (M  = 3.66  for  personalization  and  3.44  for  non-personalization,  r|^  = .04). 

Univariate  analyses  for  ability  level  revealed  significant  attitude  differences  favoring  higher-ability  students 
on  five  of  the  items.  Higher-ability  students  responded  significantly  more  positively  than  lower-ability  subjects  at 
the  .001  level  to  the  statements:  “I  learned  a lot  from  this  program”  (M  = 3.91  for  higher  ability  and  3.51  for  lower 
ability,  = .14),  and  “I  am  able  to  do  two-step  math  problems  well”  (M  = 3.35  for  higher  ability  and  2.88  for  lower 
ability,  T|^  = .1 1),  and  at  the  .05  level  to  the  statements:  “This  program  was  easy”  (M  = 3.32  for  higher  ability  and 
3.03  for  lower  ability,  = .04),  “I  liked  this  program”  (M  = 3.44  for  higher  ability  and  3.12  for  lower  ability, ti^  = 
.05),  and  “I  would  like  to  do  more  math  word  problems  like  the  ones  in  the  program”  (M  = 3.68  for  higher  ability 
and  3.43  for  lower  ability,  r\^  = .05). 

The  frequency  of  constructed  responses  on  the  attitude  survey  to  the  two  open-ended  questions  about  what 
students  liked  most  and  what  they  liked  least  was  also  tabulated.  Student  responses  indicated  that  what  they  liked 
most  was  that  the  program  was  interesting,  a response  given  by  42  of  the  136  students  (31%).  The  second  most 
common  response  to  what  students  liked  most  was  the  use  of  the  names  of  cartoon  characters,  persons,  and  things 
with  which  they  are  familiar,  a response  indicated  by  38  students  (28%).  When  asked  what  they  liked  least  about 
the  program,  48  students  (35%)  indicated  nothing  and  33  students  (24%)  responded  that  the  lesson  was  too  difficult. 


Table! 

Student  Attitude  Scores  by  Treatment  and  Ability  Level 


Treatment  Ability 


Item 

Pers 

Non- 

Pers 

F 

Higher 

Lower 

F 

1 . It  is  important  to  know 
how  to  solve  two-step 
math  problems. 

3.65 

3.72 

.57 

3.72 

3.65 

.57 

2.  This  program  was 
interesting. 

3.62 

3.24 

13.24** 

3.40 

3.46 

.29 

3.  This  program  was  easy. 

3.50 

2.85 

31.00** 

3.32 

3.03 

5.41* 

4.  This  program  had  many 
familiar  persons,  places, 
and  things. 

3.62 

3.13 

15.81** 

3.49 

3.26 

2.99 

5.  1 learned  a lot  from  this 

program. 

3.69 

3.74 

.22 

3.91 

3.51 

20.85** 

I am  able  to  do  two-step  math 
problems  well. 

3.16 

3.07 

.52 

3.35 

2.88 

16.65** 

I liked  this  program. 

3.28 

3.28 

.00 

3.44 

3.12 

6.29* 

I would  like  to  do  more  math 
word  problems  like  the  ones  in 
the  program. 

3.66 

3.44 

4.81* 

3.68 

3.43 

6.26* 

Overall  means 

3.52 

3.31 

10.83** 

3.54 

3.29 

16.00** 

*£<  .05.  **£<  .01. 
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Teacher  Attitudes 


The  overall  mean  teacher  rating  on  the  six  items  on  the  Teacher  Survey  was  3.71,  a favorable  rating 
indicating  strong  agreement  with  positive  statements  about  the  instructional  program.  All  four  respondents  agreed 
very  strongly  (M  = 4.00)  with  three  statements:  “Personalization  was  a good  teaching  strategy”,  “I  liked  the 
personalized  version  of  the  program”,  and  “1  would  enjoy  teaching  a personalized  lesson  to  my  students 
occasionally.” 

On  the  one  open-ended  question  on  the  survey,  “Please  make  any  comments  or  suggestions  that  you  would 
like  to  make  about  this  instructional  program,”  two  teachers  indicated  that  making  math  word  problems  more 
personalized  increases  students’  motivation  and  interest.  One  teacher  reported  that  the  students  would  understand 
the  personalized  problems  better  because  they  could  relate  the  information  in  the  problems  to  their  real-life 
situations  and  “shorten  the  distance”  of  their  thinking  patterns  on  the  problems. 

Discussion 

The  primary  research  question  in  this  study  addressed  the  issue  of  whether  personalization  of  instruction 
increases  the  achievement  of  Taiwanese  students  on  mathematics  word  problems.  The  treatment  by  test  occasion 
interaction  revealed  that  the  personalized  treatment  did,  in  fact,  result  in  significantly  higher  pretest-to-posttest  gains 
than  the  non-personalized  treatment.  This  positive  finding  for  personalization  of  instruction  is  consistent  with  the 
results  obtained  in  several  studies  of  personalized  mathematics  instruction  in  the  United  States  (Anand  & Ross, 

1 987;  Lopez  & Sullivan,  1991,  1 992).  It  differs  from  the  overall  result  for  personalization  in  the  earlier  study  by  Ku 
and  Sullivan  (2000)  in  Taiwan,  in  which  a ceiling  effect  limited  the  potential  gain  of  higher-ability  fifth-graders. 

The  strategy  of  conducting  this  study  with  fourth-grade  students,  instead  of  fifth-graders  as  in  the  earlier 
study,  had  the  desired  result  of  reducing  the  strong  ceiling  effect  observed  previously  (Ku  & Sullivan,  2000). 

Despite  the  successful  effort  to  reduce  the  ceiling  effect  with  higher-ability  students  in  this  study,  the  significant 
ability  level  by  test  occasion  interaction  revealed  that  this  effect  was  not  completely  eliminated.  Higher-ability 
subjects  averaged  95%  (1 1 .35  of  1 2 correct)  on  the  posttest.  This  represented  only  1 7%  gain  over  their  pretest  score 
of  78%,  but  nevertheless  approached  the  maximum  possible  score  on  the  test.  In  contrast,  lower-ability  subjects 
were  able  to  make  a much  higher  gain,  due  in  part  to  their  much  lower  pretest  score  of  only  31%. 

The  significant  overall  improvement  from  55%  on  the  pretest  to  82%  on  the  posttest  indicated  that  the 
instructional  program  itself  was  generally  effective  across  all  treatment  groups.  This  increase  in  scores  occurred 
over  an  average  time  period  of  only  about  53  minutes  of  instruction  and  review.  In  addition,  the  fact  that  students 
averaged  83%  on  the  practice  items  and  82%  on  the  posttest  indicates  that  they  retained  their  learning  quite  well 
from  the  instructional  phase  of  the  study  to  the  final  testing  phase. 

The  finding  that  students  scored  higher  on  personalized  test  items  than  on  non-personalized  items,  even 
prior  to  instruction  (i.e.,  on  the  pretest),  has  potential  implications  for  mathematics  assessment.  Certainly  this 
finding  merits  further  investigation  in  other  settings  and  with  larger  samples  to  determine  its  consistency  and 
generality.  However,  it  suggests  that  students  generally  may  score  higher  on  math  problems  that  include  more 
familiar  or  personalized  settings  than  on  problems  with  unfamiliar  or  non-personalized  settings.  If  that  is  the  case, 
test  developers  who  write  standardized  and  criterion-referenced  mathematics  tests  may  want  to  consider  the 
appropriateness  of  using  settings  for  their  problems  that  are  generally  familiar  or  popular  with  the  projected  test 
population. 

The  attitude  data  clearly  indicated  students’  preference  for  personalized  instruction,  a result  consistent  with 
the  findings  in  previous  studies  (Ku  & Sullivan,  2000,  Lopez  & Sullivan,  1992,  Ross  & Anand,  1987).  The 
strongest  differences,  all  at  the  .001  level,  were  on  items  stating  that  the  program  was  interesting;  the  program  had 
many  familiar  persons,  places,  and  things;  and  the  program  was  easy.  The  first  two  of  these  statements  reflect  the 
intended  nature  of  a personalized  program  and  the  third  is  consistent  with  the  explanation  that  personalization  may 
make  learning  easier  by  reducing  subjects’  cognitive  load.  Personalized  subjects  also  agreed  significantly  more 
strongly  (^  < .05)  that  they  would  like  to  do  more  math  word  problems  like  the  ones  in  the  program,  a statement  that 
suggests  greater  continuing  motivation  or  willingness  to  return -to-task  on  their  part.  In  general,  the  significant 
differences  on  these  particular  items  support  the  claimed  motivational  and  learning  advantages  of  personalization. 

Several  significant  differences  between  ability  levels  on  the  attitude  survey  also  reflected  differences  that 
one  might  expect  between  higher-ability  and  lower-ability  students.  Higher-ability  students  agreed  more  strongly 
that  they  learned  a lot  from  the  program  (p  < .001 ),  that  they  were  able  to  do  two-step  math  problems  well  (p  < 

.001),  and  that  the  program  was  easy  (p  < .05).  These  items  appear  to  be  the  ones  in  the  survey  that  were  most 
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closely  associated  with  ability  level.  In  contrast,  higher-ability  and  lower-ability  subjects  did  not  differ  significantly 
in  their  responses  to  items  that  logically  seem  to  be  less  associated  with  ability. 

This  study  was  not  designed  as  a test  of  theoretical  explanations  for  the  effects  of  personalization. 

However,  certain  data  from  the  teacher  and  student  surveys  tend  to  support  the  “increased  interest”  and  “reduced 
cognitive  load”  explanations  described  in  the  introduction  section  of  this  paper. 

The  present  results  were  obtained  using  group  personalization  in  a low-technology  environment  that  is 
typical  of  the  schools  in  Taiwan  and  in  many  other  countries.  A basic  use  of  personalization,  or  interest-based 
instruction,  in  such  settings  is  for  teachers  to  make  a conscious  effort  to  learn  their  students’  interests  and  to 
incorporate  them  regularly  into  their  instruction.  Teachers  can  supplement  their  existing  knowledge  of  student 
interests  by  using  an  interest  survey,  as  was  done  in  the  present  study,  and/or  by  holding  occasional  discussions  with 
their  students  about  current  topics  and  events  that  may  be  popular  with  them. 
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in  Planning  For  The  Migration 

Juanda  Beck-Jones 
Shujen  Chang 
Florida  State  University 


Abstract 

With  the  increasing  use  of  web-based  instruction  in  universities,  faculty  are  being  asked  to  transition 
courses  from  the  classroom  to  the  web.  Migrating  a course  requires  planning  that  includes  changes  in  the 
instructional  strategy  and  communications  strategy  that  many  faculty  might  not  consider.  This  process  is 
particularly  challenging  for  instructors  teaching  their  first  online  course.  The  contents  of  this  paper  grew 
out  of  an  experience  in  transitioning  a course  from  face-to-face  delivery  to  web-based  delivery.  It 
represents  challenges  that  emerged  and  offers  recommendations  for  making  the  transition  easier  for  faculty 
and  students. 

Introduction 

With  the  explosion  of  web-based  technology  and  the  resultant  increasing  demand  for  web-based  courses, 
many  faculty  members  are  faced  with  teaching  their  first  online  course.  For  these  faculty,  and  others,  the  use  of 
technology  to  deliver  instruction  can  lead  to  course  management  issues  that  are  unplanned  for  and  result  in 
challenges  that  are  unexpected.  Some  of  the  challenges  that  have  been  reported  in  the  literature  are  related  to  the 
need  to  change  the  approaches  to  instruction  and  interaction  in  order  to  make  them  more  effective  for  the  online 
environment  (Pajo  & Wallace,  2001;  Boettcher  & Conrad,  1999;  Leh  & Som,  1999).  An  additional  challenge  to 
transitioning  is  the  time  associated  with  using  and  monitoring  the  web-based  environment  (Pajo  & Wallace,  2001; 
Daugherty  & Funke,  1998).  It  is  suggested  that  preplanning  to  augment  a course  to  meet  the  requirements  of  the  new 
delivery  system  might  ease  the  transition  for  faculty.  This  paper  presents  factors  that  emerged  from  reflections  from 
a transitioning  experience.  It  recommends  methods  for  making  the  transition  easier  for  faculty. 

The  Experience 

The  insights  and  recommendations  presented  in  this  paper  stem  from  an  experience  in  teaching  a course 
transitioned  from  face-to-face  to  the  web.  The  course.  Introduction  to  Instructional  Systems  Design 
(ci3.acns.fsu.edu/courses/EME  5603-01)  was  transitioned  in  the  Fall  Semester  of  2000.  This  course  is  one  of  the 
comer  stones  of  the  Masters  Degree  program  in  Instmctional  Systems  at  Florida  State  University.  The  face-to-face 
course  is  well  designed  for  an  optimal  student  experience.  It  has  been  formatively  evaluated  and  revised  over  many 
years.  It  is  a product-based  course  that  is  structured  to  provide  an  intensive,  highly  individualized  opportunity  for 
students  to  develop,  formatively  evaluate,  and  revise  their  own  instructional  module.  The  web-based  version  of  this 
course  was  designed  to  mirror  the  face-to-face  class.  All  assignments  were  the  same,  the  approach  to  the  students 
was  the  same,  and  the  products  required  were  the  same.  The  students  were  adult  learners  located  in  Virginia, 
Oklahoma,  Georgia  and  Florida.  Most  of  them  work  in  corporate  environments  as  trainers.  Their  final  projects  were 
of  equal  quality  to  the  face-to-face  class  and  the  course  was  viewed  as  a valuable  experience  for  both  students  and 
the  instmctor  in  terms  of  providing  the  flexibility  of  time  to  teach  and  learn  and  opportunities  for  high  quality 
interaction. 

While  the  course  worked,  the  process  created  some  potential  threats  to  instructional  effectiveness  such  as 
the  additional  instructional  load.  Factors  that  were  major  concerns  in  relation  to  the  transition  were:  1)  translation  of 
the  from  face-to-face  directly  to  the  web,  2)  the  highly  individualized  nature  of  the  assignments  and  the  resultant 
frequent  formalized  student- to-  instructor  interaction,  without  significant  student-to-student  interaction,  and  3)  the 
time  management  issues  regarding  managing  and  monitoring  the  course  delivered  via  the  web. 

Factors  to  Consider 

This  experience  suggested  factors  that  faculty  may  consider  in  transitioning  a course  from  face-to-face  to 
the  web.  Such  factors  might  be  planning  for  the  migration  by  1)  adapting  the  instructional  strategy  to  change  the 
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course  from  teaching  focused  to  learning  focused,  2)  developing  a time  management  strategy  for  communicating 
effectively  with  students  regarding  issues  of  course  content,  course  assignments,  technical  matters  related  to  the 
course  site,  and  sending  and  receiving  assignments,  and  3)  obtaining  support  from  the  university  for  online 
facilitation.  Recommendations  are  presented  related  to  each  of  the  three  factors. 

Recommendations 

In  terms  of  adapting  the  approach  from  teaching  to  learning,  the  instructor  might  reconfigure  activities  that 
work  in  the  face-to-face  classroom  to  achieve  the  desired  learning  outcome  in  a web-based  environment.  In 
adaptation  for  interaction  in  the  web-based  environment,  the  key  in  this  adaptation  is  balanced  interaction.  Boettcher 
and  Conrad  suggest  that  student -to-student  interaction  should  represent  40%  of  the  total  interaction  in  the  course. 

The  remaining  60%  is  divided  between  instructor-student  and  student-content  interaction,  with  more  focus  on 
instructor-to-student  interaction.  Adapting  the  interaction  to  promote  learning  on  the  web,  should  increase  student 
interest  in  the  course  and  student  satisfaction  with  the  course  (Leh  and  Som,  1999).  In  adapting  the  course,  the 
instructor  should  consider  using  methods  of  synchronous  and  asynchronous  interactions  between  student  groups  of 
various  sizes.  Synchronous  interactions  may  include  online  chats  or  small  group  meetings  and  telephone  calls. 
Asynchronous  interactions  may  include  online  seminars,  multimedia  presentations,  e-mail  and  discussion  boards. 

The  methods  used  will  depend  on  the  goals  of  the  course,  faculty  choice  of  interaction  method  and  student  choice  of 
interaction  method  (Boettcher  & Conrad,  1999).  Adapting  the  course  to  allow  for  more  balance  in  the  types  of 
interaction  should  increase  the  effectiveness  of  a transitioned  course  in  achieving  learning  outcomes. 

Time  management  is  a major  issue  for  faculty  transitioning  a course  from  face-to-face  to  the  web-based 
instruction.  It  is  essential  that  faculty  plan  to  structure  their  time  to  accommo  date  the  needs  of  students  at  a distance. 
This  involves  planning  to  check  daily  for  student  emails,  craft  emails  to  respond  to  student  emails,  check  discussion 
boards  for  student  questions  follow-  up  with  students  who  fail  to  participate,  address  students  needs  for  additional 
information  regarding  course  content,  address  group  issues  dealing  with  technical  questions  about  using  the  course 
site,  and  send  and  receive  assignments.  These  aspects  of  teaching  an  online  course  may  be  familiar  to  online 
veterans  and  many  of  these  issues  may  translate  readily  from  the  classroom  to  the  web.  However,  for  the  novice,  the 
distance  sometimes  complicates  completion  of  simple  tasks.  For  example,  communication  that  might  take  place  at 
the  end  of  class  or  in  the  office  may  take  much  longer  to  complete  effectively  online.  Technical  issues  related  to 
using  the  course  delivery  system  may  compete  for  faculty  attention  in  teaching  the  course.  Additionally,  sending  and 
receiving  assignments  may  be  complicated  by  the  technology.  In  sum,  a course  taught  at  a distance  may  require 
more  of  the  instructor’s  time  than  a face-to-face  course.  Faculty  and  administrators  should  be  aware  of  this  when 
considering  assigning  duties  and  responsibilities  that  involve  a transition  of  this  kind. 

Planning  to  address  course  management  issues  can  be  alleviated  through  additional  support  from  the 
university.  The  addition  of  a mentor/facilitator  would  assist  in  changing  the  focus  of  the  instruction  from  teaching  to 
learning  (Moore  & Kearsley,  1996).  This  facilitator  could  assist  in  addressing  many  of  the  course  management 
issues  that  affect  instructor  time.  He/she  could  also  help  in  creating  the  learning  environment,  facilitating 
communication  to  ensure  a smooth  student  experience  in  the  course  by  responding  to  emails,  checking  discussion 
boards  and  solving  technical  problems.  This  would  free  the  instructor’s  time  to  focus  on  more  substantive  issues 
related  to  achieving  learning  outcomes.  This  kind  of  support  would  appear  to  ease  the  transition  from  face-to-face  to 
the  web. 

Conclusion 

In  conclusion,  the  growth  in  the  use  of  the  web  for  delivery  of  instruction  has  led  universities  to  increase 
expectations  that  faculty  will  transition  courses  from  face-to-face  delivery  to  web-based  delivery.  To  facilitate 
effective  transition,  faculty  should  plan  to  address  course  management  issues  by:  1)  adapting  their  courses  from  a 
teaching  to  a learning  focus,  2)  structuring  their  time  to  accommodate  the  additional  demands  of  teaching  an  online 
course,  and  3)  obtaining  additional  support  from  the  university  to  facilitate  the  learning  environment.  These 
recommendations  address  issues  that  may  assist  faculty  in  planning  to  transfer  a course  from  the  classroom  to  the 
web.  The  discussion  of  the  impact  of  transitioning  on  faculty  will  surely  continue  as  demands  for  web-based  course 
offerings  increase  and  more  faculty  respond  by  transitioning  courses  to  web-based  delivery. 
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Objectives 

The  purpose  of  this  paper  is  to  present  the  results  of  three  years  of  development  and  evaluation  of  a cd- 
rom/web  hybrid  simulation  known  as  Brownfield  Action  for  an  introductory  environmental  science  course  at  an 
independent  college  for  women  located  in  a large  city  in  the  northeastern  USA. 

Perspectives 

Science  courses  often  struggle  to  link  the  material  in  lecture  and  laboratory  for  many  reasons.  Laboratories 
are  often  taught  by  different  personnel,  and  must  fit  into  constraints  of  equipment  and  time,  as  well  as  safety  and 
level  of  difficulty.  A large  survey  course  may  cover  dozens  of  topics,  whereas  a student  typically  experiences  no 
more  than  fifteen  laboratory  periods  in  a given  semester.  Larger  than  these  logistical  issues  is  the  student’s  mindset 
in  each  environment.  Life  science  lectures  require  rapid  note  taking  and  passive  concentration  as  verbal  information 
is  drilled  at  students  for  anywhere  from  one  to  two  hours.  Laboratory  sessions  typically  require  two  to  three  hours 
of  recipe-following  and  occasional  problem  solving.  These  two  functions  operate  in  isolation  because  there  is  little 
time  in  lecture  to  discuss  experimental  technique  (if  the  lecturer  even  knows  that  information)  and  a laboratory  is  not 
typically  set  up  for  someone  to  give  verbal  instruction  as  occurs  in  the  lecture  hall.  But  more  than  these  aspects, 
typical  science  students  spend  most  of  their  time  memorizing  verbal  information  for  lecture  exams,  and  memorizing 
techniques,  anatomy,  and  taxonomies  for  laboratory  practical  examinations.  Rarely  is  information  from  one  used  in 
the  other  simply  because  the  lecture  focuses  on  concepts  while  the  laboratory  teaches  technique.  Students  often 
walk  away  from  a course  feeling  as  if  it  were  two  courses  and  fail  to  get  a sense  of  the  content  as  a whole. 

Simulations  provide  the  ability  to  integrate  content  into  a complex  problem  that  students  can  explore.  The 
closest  work  to  the  experience  of  the  Brownfield  Action  design  and  development  process  can  be  seen  in  Goodrum, 
Dorsey,  and  Schwen’s  work  on  defining  and  designing  Enriched  Learning  and  Information  Environments  (ELIEs) 
{Educational  Technology  November  1993).  In  this  paper,  the  researchers  describe  how  their  experience  with 
designing  educational  environments  led  them  away  from  entirely  technology-based  and  theory -based  definitions  and 
more  towards  what  they  define  as  a socio -technical  definition  that  focuses  on  the  people  involved  and  the  specific 
work  they  are  asked  to  perform.  They  perceive  that  the  latest  learning  theory  and  the  latest  technology  does  not 
lead  to  innovation,  but  instead  that  all  innovations  are  situated  within  a context  of  people  trying  to  accomplish  work 
in  a particular  environment,  and  a well -designed  teaching  and  learning  tool  should  support  that  work.  They  go  on  to 
describe  their  design  work  as  a series  of  relationship  building  with  users,  rapid  prototyping  with  mock-ups  and  a 
focus  on  the  tasks  that  users  needed  to  perform.  The  Brownfield  Action  project  follows  a similar  model. 

Methods 


Columbia  University’s  Center  for  New  Media  Teaching  & Learning  (CCNMTL)  developed  and 
implemented  a cd  rom/web  hybrid  simulation  known  as  Brownfield  Action  in  the  Fall  of  1999  and  again  in  the  Fall 
of  2000  at  an  independent  college  for  women  located  in  a large  city  in  the  northeastern  USA,  which  models  how  an 
undergraduate  Environmental  Science  course  can  directly  integrate  its  lecture  and  laboratory  components.  Through 
presenting  students  with  a complex  problem  that  requires  the  application  of  knowledge  and  skills  learned  in  lecture 
and  in  the  laboratory  setting,  performance  and  engagement  is  improved,  and  students  experience  environmental 
science  as  a highly  integrated  field  while  developing  real-world  problem  solving  skills.  Rather  than  reserving  higher 
level  thinking  skills  such  as  analysis  and  synthesis  for  smaller,  more  advanced  courses.  Brownfield  Action  allows 
introductory  students  to  see  environmental  science  as  an  integrated  and  dynamic  part  of  society,  rather  than  a series 
of  abstract  concepts  and  recipe-driven  techniques. 

Brownfield  Action  is  a simulation  that  provides  a learning  environment  for  developing  the  skills  of  an 
environmental  site  investigator  by  placing  students  in  a virtual  town  and  asking  them  to  serve  as  consultants  to  a real 
estate  developer  who  wants  to  avoid  purchasing  potentially  contaminated  land.  Students  must  become  actively 
involved  in  the  lecture  and  laboratory  in  order  to  succeed.  They  must  learn  to  explore  and  discover  a path  to  a 
solution  to  the  problems  they  encounter  and  work  collaboratively  with  a partner  in  order  to  reach  a valid  conclusion. 
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Pairs  of  students  form  environmental  consulting  companies  to  investigate  a hypothetical  abandoned  factory  site  in  a 
small  town.  A mall  developer  who  wishes  to  purchase  the  factory  site  contracts  with  each  two -student  company  to 
conduct  an  investigation,  write  a report  recommending  a course  of  action,  and  construct  maps  of  the  site’s  basic 
geology,  topography,  and  any  contamination  they  discover. 


Figure  1.  A screen  capture  of  the  site  map  in  the  historical  section  of  the  application. 

The  user  is  preparing  to  interview  the  owner  of  the  local  nursery  in  the  town. 

To  complete  the  maps  and  report,  students  must  first  gather  a site  history  from  the  town’s  resources. 
Through  the  simulation,  students  visit  government  offices,  businesses,  and  residences  to  conduct  interviews  with 
town’s  officials  and  citizens  and  to  obtain  public  documents  (Figures  1 and  2). 
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Figure  2.  A screen  capture  of  the  municipal  directory  where  students  visit  government  offices. 

The  user  is  interviewing  the  township  mayor. 

Using  the  information  obtained  from  the  town’s  history,  students  then  conduct  a series  of  environmental 
tests  to  determine  the  presence,  extent,  and  probable  cause  of  any  contamination.  Some  tests,  such  as  soil 
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permeability,  are  conducted  as  traditional  laboratory  exercises  placed  within  the  context  of  the  simulation.  Other 
tests,  such  as  well  monitoring,  are  conducted  virtually  using  the  computer.  Over  two  million  data  points  are 
available  for  collection  over  the  64,000  square  foot  virtual  site  map,  including  bedrock  and  water  table  data  as  well 
as  contamination  concentrations  at  depths  of  over  1 50ft.  This  vast  quantity  of  data,  both  through  the  site  history  and 
the  environmental  testing,  allows  for  an  infinite  number  of  strategies  for  testing  and  a unique  data  set  for  every 
student  company.  Successful  students  work  within  the  given  operating  budget  and  clearly  identify  the  cause  and 
extent  of  the  contaminated  areas  on  the  site. 
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Figure  3.  A screen  capture  of  the  testing  interface. 

The  user  is  preparing  to  use  the  Ground  Penetrating  Radar  test  on  the  Site  Map. 

Brownfield  Action  is  not  intended  as  a cost-cutting  measure,  a time-saver,  or  a replacement  for  other 
material,  but  as  a method  of  integrating  what  was  formerly  disparate  labs  and  lectures  into  a seamless  learning 
experience  that  improves  student  learning  and  motivates  students  to  critically  consider  the  importance  of  all  the 
issues  involved  in  the  system  of  human  impact  on  the  environment. 

The  primary  faculty  member,  serving  also  as  the  subject  matter  expert,  identified  the  following  objectives: 

After  experiencing  Brownfield  Action,  students  will  be  able  to  explain  how  to  approach  and  solve  a 
scientific  problem  by: 

• describing  the  strategy  used  to  discover  contamination  sites  in  Brownfield  Action; 

• identifying  and  explaining  the  outcomes  of  environmental  tests  they  conduct  and  related 
information,  making  recommendations  and  being  aware  of  the  consequences  of  their  decisions; 

• drawing  inferences  from  data  about  structures  that  contribute  to  environmental  contamination; 
and  students  will: 

• read  articles  on  ecology  with  different  understanding,  interest,  and  personal  commitment; 

• appreciate  that  real  world  decision-making  about  ecology  involves  ambiguity  rather  than  certainty. 

To  assess  the  effectiveness  of  the  second  version  of  Brownfield  Action  (BfA2.0)  that  ran  in  the  Fall  of  2000 
in  meeting  its  objectives,  students  were  surveyed  at  the  beginning  of  the  simulation  to  obtain  an  indication  of  their 
perceptions  of  the  levels  of  knowledge  and  skill  targeted  by  the  course  with  which  they  started.  The  laboratory 
directors  prepared  a daily  implementation  log  providing  detailed  documentation  of  the  BfA2.0  experience  in  the 
labs.  The  evaluators  observed  five  lectures  and  six  of  the  three-hour  labs  to  further  their  understanding  of  the 
instructional  setting  within  which  the  Brownfield  Action  simulation  was  conducted.  At  the  end  of  the  course, 
students  rated  Brownfield  Action  on  how  well  it  had  contributed  to  the  objectives  of  the  course  and  other  matters. 
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and  all  students  participated  in  hour-long  focus  groups  to  provide  in-depth  responses  to  questions  not  amenable  to 
discovery  through  written  survey  questionnaire.  The  evaluators  also  conducted  hour-long,  post-course  interviews 
with  the  lead  designer  of  the  simulation;  the  primary  faculty  member  (subject  matter  expert),  and  the  laboratory 
directors.  Finally,  a cross  section  of  the  papers  prepared  by  BfA2.0  students  was  reviewed  and  compared  with 
student  papers  from  previous  years. 

Data  Sources 

The  entire  course  of  one  hundred  and  twelve  female  students  were  surveyed  using  two  questionnaires  (one 
at  the  beginning  of  the  semester,  one  at  the  end)  and  a series  of  focus  groups  at  the  end  of  the  term  (one  for  each  of 
the  eight  lab  sections). 

In  the  survey  questionnaire  and  focus  groups  we  investigated  students* 

• perceptions  of  pre-  and  post-test  levels  of  their  knowledge; 

• perceptions  of  the  success  of  BfA2.0  in  meeting  its  objectives; 

• perceptions  of  the  contributions  of  various  components  of  BfA  in  assisting  them  to  solve  the 
overall  problem  addressed  by  BfA2.0,  and  how  well  BfA2.0,  the  lectures  and  the  lab  succeeded  in 
directing  them  to  focus  on  the  major  problem  addressed  by  BfA2.0. 

• overall  evaluation  of  BfA2.0,  positive  and  less  positive; 

• recommendations  for  technical  improvements  in  BfA2.0; 

• perceptions  of  the  most  valuable  parts  of  BfA2.0; 

• perceptions  of  what  could  be  done  to  improve  BfA2.0. 

The  evaluation  team  also  examined  students*  final  reports  in  accordance  with  the  learning  objectives 
outlined  by  the  primary  faculty  member  as  well  as  the  daily  laboratory  implementation  log  prepared  by  the 
laboratory  directors. 

Summary  of  Results 

• Students  learned  more  and  in  greater  depth  using  Brownfield  Action  than  in  previous  years 
without  it. 

• Student  work  looked  more  authentic  and  professional,  closer  to  what  would  be  expected  of  a 
professional  performing  these  tasks. 

• Students  appreciated  how  the  simulation  contributed  to  their  understanding  of  the  material. 

• Brownfield  Action  was  successful  in  (1)  enhancing  the  scientific  literacy  of  students,  (2) 
facilitating  their  construction  of  new  meaning  based  upon  what  they  saw  and  experienced  and  (3) 
enhancing  practices  leading  to  increased  student  learning. 

• Students*  abilities  to  construct  new  meaning  based  upon  what  they  saw  and  experienced  were 
expanded. 

• Students  found  the  content  useful  and  the  simulation  a good  way  to  learn  it. 

• More  staff  development  is  necessary  to  foster  the  teaching  strategies  needed  for  a true  discovery 
process  and  to  build  comfort  with  technology. 

• Minor  technical  glitches  created  levels  of  frustration  that  interfered  with  its  ability  to  meet  its 
educational  potential.  These  glitches  notwithstanding,  the  students  gave  high  ratings  to  other 
features  of  the  technology.  In  particular,  eight  out  of  ten  students  noted  the  ease  with  which  it  was 
possible  to  move  through  information  in  each  of  the  sections  of  the  simulation. 


Discussion 

The  simulation  in  the  context  of  Brownfield  Action  is  not  just  in  the  technology,  but  in  the  classroom 
environment  set  up  around  and  supported  by  the  technology.  The  simulation  is  in  the  course  curriculum,  the  faculty 
member,  the  lab  directors,  and  the  students,  and  it  only  becomes  a simulation  once  all  of  these  factors  work  in 
parallel.  For  as  long  as  there  has  been  computing,  many  have  tried  io  simulate  aspects  of  reality  with  computing 
power,  but  the  reality  is  that  all  that  has  been  built  to  date  might  be  better  q,2\\q(\  simulators.  Simulators  model  real- 
world  processes  with  mathematical  equations  and  variables  that  can  be  manipulated  so  that  changes  can  be  observed 
and  analyzed.  The  software  for  Brownfield  is  no  different  upon  close  inspection.  It  is  a model  of  a town  with  a 
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mathematically  driven  contamination  event  that  occurs  within  a narrative,  but  the  narrative  does  not  exist  without 
the  students  fulfilling  their  role  as  investigators.  In  other  words,  the  story  does  not  get  carried  out  unless  the 
students  and  faculty  pursue  their  roles  and  fulfill  their  tasks,  or  in  effect,  write  their  parts  of  the  story  as  they  interact 
with  the  simulator.  Brownfield  Action  could  be  run  in  a classroom  without  any  technology  at  all,  and  in  fact  it  was 
done  on  a smaller  scale  for  many  years  using  an  elaborate  note  card  scheme.  The  computer  software  supports  the 
expansion  of  the  simulation  created  in  the  classroom,  giving  it  a visual  space,  a more  efficient  process  for  collecting 
data,  a means  for  communication,  and  more  definition  and  depth  (literally  and  figuratively)  to  the  environment  that 
is  to  be  examined.  The  software  itself  does  not  teach.  Instead  it  invites  the  user  to  engage  in  the  problem.  To  foster 
engagement,  one  needs  motivation  and  a safe  space  to  take  risks,  and  that  is  where  the  surround,  the  classroom,  the 
curriculum,  and  the  instructors  come  in  to  play. 

Simulations  can  be  powerful  teaching  and  learning  environments  because  operating  in  the  real  world  is  not 
really  all  that  different  from  participating  in  a simulation.  Everyone  has  roles  to  play  in  the  real  world  (probably 
more  than  one),  and  there  are  problems,  conflicts,  and  obstacles  to  be  encountered  and  overcome  in  each  of  those 
roles.  The  method  for  overcoming  these  problems  is  in  communication  and  experimentation,  which  includes 
hypothesizing,  collecting  data,  analyzing,  synthesizing,  and  applying  previous  discoveries  with  other  role  players 
using  the  tools  that  are  available.  One  of  the  tools  available  in  Brownfield  Action  is  the  software,  but  there  are  also 
physical  maps,  soil  samples,  reference  material,  and  other  similar  narratives  such  slsA  Civil  Action,  a novel  all 
students  in  the  course  read.  Most  any  profession  is  going  to  place  people  in  a similarsituation  and  demand  the  same 
action  from  its  participants.  Brownfield  Action  provides  a safe  environment,  away  from  the  consequences  of  the 
real  world  - the  money  involved  isn’t  real,  one’s  job  isn’t  at  stake,  etc...  - so  students  can  essentially  practice  living 
in  the  real  world  by  immersing  themselves  in  the  problem  to  be  solved  in  the  simulation. 

As  outlined  in  studies  such  as  Sandholtz’s  Teaching  with  Technology:  Creating  Student-Centered 
Classrooms,  instructors  need  to  phase  out  traditional  directed  or  didactic  teaching  techniques  and  work  towards 
more  guiding  and  exploring  strategies  for  teaching  with  technology  to  be  most  effective.  This  has  proved  true  with 
Brownfield  Action  as  well,  so  in  preparation  for  this  year’s  course,  we  instituted  more  staff  development  for  the 
laboratory  directors  to  assist  them  in  improving  their  instructional  techniques  as  well  as  a study  guide  to  aid  students 
in  orienting  themselves  to  the  project. 

All  known  technical  glitches  from  last  year’s  experience  have  been  remedied  for  this  year’s  course,  but 
minor  problems  may  continue  to  appear  as  they  would  with  any  project  of  this  magnitude.  Over  time  the  instructors 
and  support  staff  will  only  get  more  accustomed  to  the  common  problems  students  experience,  so  the  effects  of  these 
problems  should  be  minimal. 

All  in  all,  the  project  has  proved  successful  in  providing  students  with  a more  integrated  science  experience 
than  in  traditional  classrooms  and  CCNMTL  (http://ccnmtl.columbia.edu)  looks  forward  to  developing  more 
simulation  models  for  teaching  and  learning  based  on  this  project. 
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Pre -Kindergarten  through  Grade  Twelve  Web-based  Science  Course  of  Study 


Donna  Huber 

Mid-Ohio  Educational  Service  Center 
Ashland  University 


This  rapid  prototype  web-based  Pre-Kindergarten  through  Grade  Twelve  Science  Course  of  Study  is  being 
developed  by  teachers  for  student  and  is  based  on  state  standards.  It  is  organized  by  strand  and  grade  level 
using  benchmark  state  proficiency  outcomes  and  competencies  as  framework.  Best  practice  activities, 
assessments,  and  resources  are  correlated  to  student  outcomes.  Since  the  Ohio  Department  of  Education  is 
in  transition  from  the  current  instructional  model  to  academic  content  standards,  this  course  of  study  is  an 
ongoing  endeavor  that  will  be  revised  and  updated  to  reflect  current  best  practice  based  on  student  results 
as  new  assessments  and  diagnostic  tools  are  developed. 


Design 

Needs  Assessment 

During  the  spring  of  1999,  two  science  consultants  from  the  Mid-Ohio  Educational  Service  Center 
(MOESC)  surveyed  science  teachers  and  administrators  in  the  eighteen  school  districts  within  the  tri-county  service 
area  to  determine  their  needs  for  the  upcoming  science  course  of  study.  Respondents  to  the  survey  indicated  the 
following  design  criteria:  web-based,  interactive,  a dynamic  document,  functional,  user  friendly,  and  resource  rich. 
(MOESC,  1999) 

Rapid  Prototype  Instructional  Design  Model 

Initial  design  was  determined  by  a committee  consisting  of  teachers  from  all  participating  districts  and  a 
team  of  science  consultants  from  the  educational  service  center.  A three-year  design,  development,  and 
implementation  process  had  been  planned  prior  to  the  recruitment  of  the  teacher  team.  The  print  version  was  to  be 
completed  by  August  2000  for  delivery  to  teachers  that  fall.  Since  the  timeline  was  short,  a rapid  prototype  needed 
to  be  developed.  Jones  and  Richey  (2000)  state  that  rapid  prototyping  methodology  "encourages  communication 
between  everyone  concerned  with  the  effort.  Subject  matter  experts  assist  in  content  identification  throughout  the 
project.  End-users  react  to  the  prototype  to  provide  feedback  regarding  the  design,  instructional  activities,  and  user 
interface."  This  should  satisfy  customers,  in  this  case,  science  teachers,  "because  they  are  involved  in  an  extensive 
formative  evaluation  of  the  actual  product  throughout  its  design  and  development." 

Feedback  during  this  phase  of  development  was  provided  through  a listserve  as  the  components  were 
loaded  to  the  web.  “Prototyping  is  defined  by  Tessmer  (1993)  as  a hybrid  of  formative  evaluation  and  design 
activities.”  (Seels  & Glasgow,  1999,  p.  150)  Seels  and  Glasgow  provide  two  reasons  for  developing  a prototype. 
First,  the  designer  may  have  questions  about  the  student’s  ability  to  learn  from  and  use  the  new  system. 
Secondly,  when  a new  technology  is  involved  there  may  be  questions  about  the  design  team’s  experience 
with  new  ways  of  doing  things.  Prototype  development  allows  the  client  to  assess  the  cost  effectiveness  of 
the  new  system  and  allows  the  design  team  the  opportunity  to  learn  new  skills  in  an  environment  where  the 
consequences  are  less  expensive,  (p.  151) 

A web  site  was  devoted  strictly  to  the  design  phase  of  the  course  of  study.  E-mail  addresses  of  team 
members  were  compiled  and  notice  of  the  addition  of  new  components  was  immediately  sent  to  the  committee  for 
their  review  and  subsequent  formative  evaluation  of  the  component  being  appraised.  Changes  or  improvements  were 
made  by  posting  any  suggestions,  waiting  for  responses,  and  subsequently  making  the  adjustments. 

Organization  of  the  Course  of  Study 

The  Table  of  Contents  was  the  initial  component  to  be  created,  since  the  Ohio  Model  Competency-Based 
Science  Program  provided  a diagram  “to  communicate  the  structure  and  relationship  of  components  of  a local 
science  curriculum....”  (Ohio  Department  of  Education  [ODE],  1994,  p.  8)  Each  component  added  was  linked  to  the 
text  descriptors  in  the  Table  of  Contents.  Links  to  the  National  Science  Education  Standards  dind  the  Benchmarks 
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for  Science  Literacy  were  created  early  in  the  process  to  provide  the  committee  with  research  based  reference 
material  on  which  to  formulate  decisions  regarding  the  components  to  be  created  thereafter.  A few  teachers  who 
devoted  extra  time  to  the  creation  of  the  Ohio  Model  science  curriculum  components  earned  university  credit.  These 
were  e-mailed  to  the  designer,  converted  to  web  pages,  and  loaded  to  the  web  site  for  review  by  the  remainder  of  the 
committee. 

Grade  Level  Templates 

The  most  crucial  and  problematic  design  issue  proved  to  be  the  grade  level  templates.  Student  performance 
data  for  the  science  portion  of  the  Ohio  Proficiency  Tests  are  reported  according  to  seven  categories  as  defined  by 
the  Ohio  Model  Competency-Based  Science  Program,  Proficiency  test  learning  outcomes  “are  grouped  into  four 
strands  - Nature  of  Science,  Physical  Science,  Earth  and  Space  Science  and  Life  Science.”  (ODE,  1994,  p.  117)  and 
three  Performance  Objective  Levels  of  Understanding:  “Acquiring  Scientific  Knowledge  - observing,  collecting, 
and  recording  data  and  information  from  various  sources;  Processing  Scientific  Knowledge  - organizing, 
interpreting,  manipulating,  and  reformulating  observations  and  data;”  and  “Extending  Scientific  Knowledge  - 
applying,  formulating,  transforming,  and  communicating  ideas  in  a variety  of  contexts.”  (ODE,  1994,  p.l  1 1)  The 
difficulty  with  this,  however,  is  that  the  Ohio  Model  is  organized  into  the  following  four  instructional  strands: 
Scientific  Inquiry,  Scientific  Knowledge,  Conditions  for  Learning  Science,  and  Applications  for  Science  Learning. 
(ODE,  1994,  pp.  19-22)  and  performance  objectives  that  “have  been  constructed  by  considering  the  instructional 
objectives  from  all  four  instructional  strands....”  (ODE,  1994,  p.23)  “School  science  committees  should  use  their 
grade-level  instructional  objectives  and  the  examples  of  performance  objectives  in  the  model  to  create  district 
performance  objectives  for  the  local  science  curriculum.”  (ODE,  1994,  p.  17) 

Arriving  at  the  Framework 

Since  proficiency  results  are  made  public,  it  is  by  those  criteria  that  effectiveness  of  science  programs  is 
demonstrated.  In  order  to  improve  student  performance,  the  templates  were  designed  by  proficiency  outcome  by 
strand  and  grade  level.  Icons  were  chosen  to  represent  the  strands  of  the  learning  outcomes. 

Work  had  been  done  at  an  educational  service  center  in  the  southeastern  part  of  Ohio,  Muskingum  Valley 
Educational  Service  Center,  to  further  clarify  the  meaning  of  the  proficiency  outcomes  for  teachers.  (Muskingum 
Valley  Educational  Service  Center,  1999)  Their  teacher  inventories  were  adapted  to  frame  the  assessments  and 
suggested  activities  and  resources  for  the  grade  level  templates.  There  was  a meeting  of  the  full  committee  and  the 
template  was  accepted  as  the  framework. 

Development 

Summer  Session 

The  development  team  attended  a three-day  summer  session  to  research  and  add  resources  to  the  templates. 
Participants  were  provided  with  various  forms  of  instructional  materials,  such  as  textbooks,  literature,  kits, 
periodicals,  and  electronic  resources  and  received  instruction  on  proper  citation  of  materials  and  editing  of  web 
pages.  The  remainder  of  the  time  was  spent  incorporating  the  materials  into  the  templates. 

Board  Adoption 

The  Governing  Board  of  the  Mid -Ohio  Educational  Service  Center  approved  the  Science  Course  of  Study 
on  July  24,  2000.  In  early  August  at  an  administrative  retreat,  the  course  of  study  was  introduced  and  the  print 
version  was  distributed  to  central  office  personnel.  In  early  fall,  a rollout  of  the  course  of  study  was  held  in  each 
county  in  the  service  area  and  a print  version  customized  to  the  grade  level  of  the  teachers  was  distributed.  Teachers 
received  their  grade  level  and  the  one  immediately  preceding  and  following,  along  with  the  Table  of  Contents  and 
other  key  components. 

Additional  Formative  Evaluation 

Districts  began  curriculum  alignment  at  their  own  pace  throughout  the  first  semester  of  the  2000-2001 
school  year.  At  a high  school  principal’s  meeting  in  early  winter  where  the  course  of  study  was  being  discussed, 
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principals  expressed  dissatisfaction  with  the  initial  design  of  the  grades  nine  through  twelve  portion  of  the  course  of 
study.  The  template  approach  was  not  feasible  with  the  discrete  course  offerings.  Smith  and  Ragan  (1999)  have  cited 
Tripp  and  Bichelmeyer,  representing  the  idea  that  “the  analysis  of  needs  and  content  depends  in  part  on  the 
knowledge  that  is  gained  by  actually  building  and  using  a prototype  instructional  system.’*  (p.  376)  This  negative 
feedback  was  entirely  unexpected  by  the  designer,  who  sought  to  find  a solution  that  would  satisfy  the 
administrators  and  still  serve  the  needs  of  the  teachers  who  would  ultimately  be  the  end  users. 

Curriculum  guides  were  submitted  from  all  participating  districts  and  templates  for  subject  area  courses 
were  created  based  on  the  course  descriptions.  Pre -kindergarten  through  grade  six  teachers  attended  a seminar 
series:  "Learn  How  to  Use  the  New  Science  Course  of  Study,"  "Science  Resource  Vendor  Fair  and  Children's 
Literature,"  and  "Planning  Your  Science  Instruction  for  2001-2002."  Grades  seven  through  twelve  teachers  received 
orientation  to  the  templates  for  the  subject  area  courses.  The  nine  through  twelve  course  templates  were  further 
developed,  including  the  addition  of  subcomponents  to  the  topic  areas  of  each  discrete  course.  The  existing  web 
version  was  edited  and  teachers  added  the  course  requirements  of  each  district  and  subsequently  added  resources  to 
the  developing  courses.  Additional  resources  were  also  added  to  the  grades  seven  and  eight  courses. 

Implementation 

Average  Proficiency  Test  Performance 

Proficiency  test  results  for  the  past  three  years  on  grades  four  and  six  proficiency  tests  were  averaged  for  all 
districts.  Over  thirty  percent  of  the  students  demonstrated  average  performance  lower  than  the  minimally  proficient 
standard  for  the  following  science  strands  on  the  Fourth  Grade  Proficiency  Test:  Nature  of  Science,  Physical 
Science,  Earth  & Space  Science,  Level  of  Understanding:  Acquiring  Knowledge,  Level  of  Understanding: 

Processing  Knowledge,  and  Level  of  Understanding:  Extending  Knowledge.  An  equal  percentage  demonstrated 
lower  than  the  minimally  proficient  standard  for  the  following  mathematics  strands  on  the  Fourth  Grade  Proficiency 
Test:  Nurrbers  and  Number  Relations,  Level  of  Understanding:  Conceptual  Understanding  and  Level  of 
Understanding:  Application  & Problem  Solving.  Over  thirty  percent  of  the  students  demonstrated  average 
performance  lower  than  the  minimally  proficient  standard  for  the  following  science  strands  on  the  Sixth  Grade 
Proficiency  Test:  Nature  of  Science,  Physical  Science,  Earth  & Space  Science,  Life  Science,  Level  of 
Understanding:  Acquiring  Knowledge,  Level  of  Understanding:  Processing  Knowledge,  and  Level  of 
Understanding:  Extending  Knowledge.  An  equal  percentage  demonstrated  lower  than  the  minimally  proficient 
standard  for  the  following  mathematics  strands  on  the  Sixth  Grade  Proficiency  Test:  Problem  Solving  Strategies, 
Numbers  and  Number  Relations,  Geometry,  Algebra,  Estimation  and  Mental  Computation,  Data  Analysis  and 
Probability,  Level  of  Understanding:  Conceptual  Understanding,  Level  of  Understanding:  Knowledge  & Skills,  and 
Level  of  Understanding:  Application  & Problem  Solving.  (MOESC,  2001) 

Professional  Development  Needs  Assessment 

In  order  to  determine  the  professional  development  needs  of  the  science  and  mathematics  teachers  in  the 
participating  districts,  a Science  and  Mathematics  Needs  Assessment  Survey  was  completed  by  581  mathematics  and 
science  teachers  in  seventeen  of  the  eighteen  participating  school  districts.  Four  districts  were  chosen  by  the  Mid- 
Ohio  superintendent  to  participate  in  a Science  and  Mathematics  Needs  Assessment  Interview  to  obtain  more 
additional  qualitative  data.  (MOESC,  2001)  All  administrators  and  central  office  personnel  and  twelve  teachers  per 
district,  one  from  each  grade  level,  were  asked  what  they  saw  as  the  most  immediate  mathematics  and  science 
curriculum  need  in  their  district  and  why  and  what  types  of  professional  development  they  thought  would  address 
that  need.  They  were  also  asked  about  possible  content  area  activities  and  they  types  of  resources  that  would  help 
provide  better  mathematics  and  science  instruction  in  their  district. 

The  survey  and  interview  data  indicate  that  the  following  professional  development  opportunities  are 
needed:  searching  techniques,  integrating  technology  into  the  curriculum,  alternative  assessment,  assessing  the  needs 
of  the  learners,  analyzing  proficiency  data,  curriculum  alignment,  and  adaptations  for  special  needs.  The  majority  of 
teachers  prefer  that  the  sessions  are  two  hours  or  one  half  day  in  length,  either  during  early  release  or  during  the 
school  day  with  a substitute.  Most  teachers  would  like  to  have  work  sessions  with  teachers  at  their  own  grade  level 
or  those  just  above  or  just  below.  The  greatest  majority  expressed  preference  for  meetings  at  the  building  level, 
followed  by  the  district  level.  Stipends  were  the  preferred  type  of  compensation. 

Ongoing  Improvements 
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Ohio  is  in  transition  from  the  current  model  to  Academic  Content  Standards.  The  State  Board  of  Education 
is  scheduled  to  adopt  the  new  standards  in  December  2002.  (ODE,  2001)  The  new  state  curriculum  model  is  due 
eighteen  months  later.  The  standards  framework  consists  of  Academic  Content  Standards,  which  the  Ohio 
Department  of  Education  defines  as  “What  all  students  should  know  and  be  able  to  do;  the  overarching  goals  & 
themes.”  Benchmarks  are  defined  as  “key  checkpoints  that  monitor  progress  toward  academic  content  standards.” 
Benchmarks  are  “identified  by  grade-level  clusters/bands,”  i.e.,  K-3, 4-5,  6-8,  9-12,  that  “will  vary  across  content 
areas  and  align  with  achievement  tests  where  applicable.”  The  Grade-Level  Indicators  provide  “what  all  students 
should  know  and  be  able  to  do  at  each  grade  level”  and  can  be  used  as  checkpoints  to  “monitor  progress  toward  the 
benchmarks.”  (ODE,  2001) 

Conclusion 

This  Course  of  Study  has  served  as  the  model  for  the  Social  Studies  Course  of  Study  currently  in  the 
implementation  stage  and  the  Mathematics  Course  of  Study  beginning  this  year.  Due  to  the  experience  gained  with 
the  Science  Course  of  Study,  the  three-year  process  to  implementation  has  been  narrowed  to  two  years  for  the  Social 
Studies  and  Mathematics  Courses  of  Study  and  all  of  those  to  follow. 
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Abstract 

During  the  last  decade  US  Krl2  schools  have  approximately  tripled  their  spending  on  increasingly 
powerful  computers,  expanded  network  access,  and  novel  computer  applications.  The  number  of  questions 
being  asked  by  educators,  policymakers,  and  the  general  public  about  the  extent  to  which  students  are 
using  these  educational  technologies,  for  what  purposes,  and  to  what  effects,  has  likewise  increased. 
Exploring  the  human  implementation  process  is  thought  to  be  one  key  to  understanding  how  educational 
technologies  find,  purchase  and  evolve  in  local  classroom  environments.  This  naturalistic  inquiry  explored 
students  ’ classroom  experiences  during  the  semester-long  process  of  implementing  an  interactive  learning 
system.  Destination  Math,  in  two  eighth-grade  math  classes  in  a rural  mid-Atlantic  Junior-Senior  High 
School,  Data  collection  was  based  on  field  observations  prior  to  and  during  the  implementation  process, 
semi-structured  interviews  and  focus  group  interviews  with  students,  and  document  analysis.  Grounded 
theory  methods  were  used  to  analyze  the  data,  I found  that  students  experienced  high  levels  of  learner- 
control  when  using  the  interactive  learning  system  for  math,  given  the  variety  of  activity  choices  afforded 
by  the  instructional  medium,  the  multiple  pathways  to  representation  of  math  concepts,  and  the 
opportunities  for  math-talk  with  their  peers.  For  these  three  reasons,  students  reported  increased  interest 
in  math  in  the  Destination  Math  class,  compared  with  the  regular  math  class.  Researcher  observations 
supported  students*  self-reports:  students*  engagement  with  math,  operationally  defined  as  time-on-task, 
increased  in  the  Destination  Math  class.  These  findings  suggest  that  when  educational  technologies  are 
used  to  heighten  student-control  over  the  learning  environment,  these  instructional  tools  may  increase 
students  * interest  in  subject  matter  and  their  engagement  in  learning. 

Introduction 

Increasingly  powerful  computers,  expanded  network  access,  and  novel  computer  applications  have 
enlarged  both  investments  in,  and  expectations  for,  the  transformation  of  students’  classroom  learning  experiences. 
During  the  last  decade  US  K-12  schools  have  approximately  tripled  their  spending  on  educational  technologies 
(Quality  Education  Data  [QED],  1999).  Improving  students’  classroom  learning  experiences  by  exploiting  the 
appeal  of  challenging  interactive  learning  technologies  is  a powerful  motivation  for  such  significant  investments  in 
educational  technologies.  However,  while  investments  in  educational  technologies  have  steadily  increased,  not 
enough  money  has  been  spent  on  educational  research  (Web  Based  Education  Commission,  2000;  Shaw,  2000). 
Merely  because  curriculum  producers  are  making  software  and  their  school  customers  are  acquiring  their  products 
and  the  hardware  to  allow  them  to  be  used,  does  not  tell  us  how,  or  to  what  extent,  the  daily  lives  of  typical  school 
children  are  being  affected  or  changed  by  these  educational  technologies  (Becker,  1998,  p.  20).  The  present  research 
was  designed  to  enliven  and  enlighten  current  discussion  of  student-use  of  educational  technologies  in  classrooms, 
by  exploring  the  implementation  of  an  educational  technology  in  two  eighth  grade  math  classes. 

This  research  represents  a departure  from  the  early  research  on  educational  technologies  (1970s  and  1 980s) 
which  emphasized  the  learning  outcomes  value-added  model:  the  decontextualized,  cognitive-psych  pedigree  of 
researching  individual  students’  interactions  with  computers.  These  early  studies  looked  so  specifically  at  particular 
technologies  and  their  impact  on  single  students,  that  they  contributed  little  to  the  larger  more  challenging  project  of 
understanding  the  roles  that  technologies  can  play  in  addressing  the  key  challenges  of  learning  in  classroom 
contexts.  Recent  research  on  educational  technologies  is  characterized  by  an  awareness  that  the  impact  of 
technology  on  specific  aspects  of  teaching  and  learning  can  be  “usefully  understood  only  in  context”  (McMillan 
Culp,  Hawkins,  and  Honey,  1999,  p.  8).  Educational  technologies  alone  do  not  translate  into  improved  instructional 
outcomes:  they  matter  only  when  harnessed  for  particular  ends  in  the  social  context  of  classrooms:  they  must 
therefore  be  studied  in  these  contexts.  The  regular  math  class,  and  the  DM  class,  represented  the  contexts  for  this 
naturalistic  inquiry.  Throughout  this  fourteen-week  study,  the  focus  of  the  research  was  on  the  students’  experiences 
with  the  interactive  learning  system  in  their  math  class. 
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Research  on  classroom  innovation  has  infrequently  focused  on  students’  experiences  of  changes  in  their 
classroom  learning  environments  (Hammersley,  1999).  While  previous  research  has  identified  student  attitude 
toward  educational  technologies  as  an  important  variable  influencing  their  learning  experiences  (Francis  & Evans, 
1995;  Liu,  1999;  Christensen,  1997),  these  studies  have  not  described  students’  experiences  with  these  novel  tools 
over  time  and  in  students’  own  words.  Instead,  students’  reports  of  their  classroom  experiences  with  educational 
technologies  have  focused  on  the  frequency  of  computer  use  in  specific  courses,  the  type  of  software  utilized,  and 
students’  responses  to  an  admittedly  rough  categorization  of  activities  (Becker,  1994).  The  purpose  of  this  study  was 
to  describe  students’  experiences  with  an  interactive  learning  system  during  one  school  semester.  The  main  research 
question  asked:  How  do  students  experience  a classroom  innovation  in  the  form  of  an  interactive  learning  system  in 
their  math  class,  during  the  semester-long  implementation  process?  This  question  was  operationalized  by  asking  the 
more  informal  question:  What  do  students  do,  say  and  make  in  the  research  context  over  time? 

Research  Context 

Participants 

Participants  in  this  study  included  32  students  (14  and  18)  in  two  lower-level  eighth  grade  math  classes. 
This  research  began  in  the  regular  math  class  four  weeks  prior  to  the  implementation  of  the  interactive  learning 
system,  and  continued  during  the  ten -week  implementation  process.  Qualitative  methods  were  used  to  explore  the 
process  of  implementing  the  interactive  learning  system,  DM,  twice  weekly,  during  students’  regularly  scheduled 
(40  minute)  math  period  in  the  computer  lab,  on  Wednesdays  and  on  Thursdays.  On  alternate  days,  students  and 
their  teacher  used  the  “highly  structured”  and  “highly  prescriptive”  (Saxon,  2001)  text  -based  math  lessons,  provided 
in  the  district-adopted  math  curriculum.  In  this  study,  the  teacher  used  the  Riverdeep  (RVDP)  Interactive  Learning 
System  to  revise  math  concepts  previously  taught  to  students  using  the  Saxon  math  lessons. 

Materials 

RVDP  was  founded  in  Ireland  in  December  1995  as  a developer  and  provider  of  technology-based 
educational  solutions  for  the  United  States  kindergarten  through  high  school,  or  K-12  market.  RVDP  offers  Internet 
and  CD-ROM  based  comprehensive  courseware  and  supplemental  curricula  in  math,  science,  and  language  arts. 
Figure  1.  DM  Main  Menu  Screen  (Course  IV) 
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Bookmark  Progress  Assignment  Test 

button  button  button  button 


In  November  2000,  the  Westridge  school  district  purchased  licenses  for  use  of  the  Riverdeep  interactive 
learning  products  by  all  eighth  grade  students  and  their  teachers,  intentionally  targeting  improvement  of  students’ 
math  skills.  RVDP  curriculum  products  for  middle  school  math  include  Destination  Math  (DM),  a comprehensive 
math  program  designed  to  supplement  or  replace  traditional  math  curricula,  and  Tangible  Math  a simulation- 
based  math  program,  which  focuses  on  the  development  of  students’  problem  solving  and  analytical  skills.  In  the 
present  study,  the  teacher  chose  to  implement  the  DM  component  of  the  RVDP  suite  of  learning  resources  with  both 
her  eighth  grade  math  classes.  DM  comprises  five  math  courses  organized  in  two  series  or  curricula  for  students  in 
grades  four  to  twelve.  Participants  in  this  study  used  DM  Course  IV  Mastering  Skills  and  Concepts. 


Figure  1 presents  the  structure  of  DM  Course  IV,  used  by  students  in  this  study.  The  course  is  composed  of 
four  modules’.  Fractions,  Decimals,  Percents,  and  Integers  and  Order  of  Operations,  each  representing  a major  topic 
in  the  eighth  grade  math  curriculum.  Each  DM  module  is  further  broken  down  into  units y which  address  specific 
learning  objectives  and  are  correlated  with  state  and  national  standards  for  mathematics  learning.  The  teacher 
management  system  for  DM  contains  a bank  of  test  items  that  are  correlated  with  specific  learning  objectives,  and 
organized  by  unit.  Unit  buttons  for  the  fractions  module  are  visible  to  the  right  of  the  user’s  screen  in  Figure  1.  DM 
units  are  further  divided  into  three  sessions.  The  user  enters  DM  at  the  session  level. 

Each  DM  session  comprises  a tutorial  and  a workout.  The  user  may  also  access  a tutorial  or  workout  as  a 
system-generated  prescribed  assignmenty  based  on  the  level  of  mastery  of  learning  objectives  demonstrated  by  the 
user  on  assigned  tests.  From  the  main  menu,  DM  users  may  choose  to  work  on  a teacher-assigned  test,  a tutorial,  or 
a workout.  During  their  ten-week  use  of  DM  in  their  math  classes  at  Westridge,  students  were  instructed  to  complete 
teacher-assigned  DM  tests,  prescribed  assignments,  and  any  additional  activities  for  each  unit,  before  proceeding  to 
the  next  unit. 


Data  Collection 

Participant  observation  strategies  (observation  and  conversation)  enabled  me  to  take  an  active  role  in 
experiencing  and  enquiring  about  (Wolcott,  1999)  students’  uses  of  DM  in  their  math  classes.  In  the  initial  weeks  of 
this  study,  I focused  my  classroom  observations  by  detailing  elements  of  the  classroom  environment  (classroom  map 
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protocol),  and  describing  the  activities  of  students  therein  (activity  framework  protocol).  I recorded  students’  actions 
and  interactions  within  the  DM  math  class  using  a small  pocket  recorder.  Written  descriptions  of  participants’ 
coverbal  behaviors  (expressions,  gestures,  actions)  in  their  math  classes  informed  and  extended  students’  voice- 
recorded  data.  Combined  with  classroom  observations,  unscheduled  verbal  exchanges  with  students  provided 
valuable  opportunities  for  me  to  become  familiar  with  “the  native  communicative  repertoire’’  of  participants  in  the 
eighth  grade  math  class  (Briggs,  1984,  p.  24).  These  classroom  conversations  provided  crucial  information  on  the 
interaction  between  the  eighth  grade  students’  use  of  language,  and  their  social  behavior,  and  also  between  students’ 
social  and  cultural  norms  and  patterns  of  interaction  in  the  math  class.  Ongoing  conversations  with  students 
informed  the  development  of  semi-structured  interview  guides  throughout  the  data  collection  process. 

I planned  exploratory  and  culminating  focus  group  interviews  with  students  to  further  explore  their 
experiences  with  educational  technologies  and  math  in  general,  and  with  DM  in  particular.  Flick  (1998)  suggested 
four  processes  or  elements  of  group  discussions,  which  I used  to  structure  the  focus  group  interview  guide:  brief 
explanation  of  the  research  procedure,  preparation  for  the  discussion,  discussion  stimulus,  and  process  discussion  (p 
119-120).  ! held  exploratory  focus  group  interviews  with  students  to  probe  their  prior  experiences  with  educational 
technologies  at  home  and  m school,  their  attitudes  toward  math,  and  their  expectations  for  the  DM  learning  system. 
Culminating  focus  group  sessions  in  the  final  week  of  this  research  offered  opportunities  for  respondent  validation 
or  member  checking  of  my  own  interpretations  of  students’  experiences  with  DM  in  their  math  class. 

I used  Patton’s  (1990)  “general  interview  guide  approach’’  (p.  280)  to  plan  periodic  semi-structured 
interviews  with  individual  students.  These  interview  guides  were  developed  and  sequenced  using  Spradley’s  (1979) 
ethnographic  questions,  the  third  element  in  his  ethnographic  interview  (p.  58-68).  I used  naturalistic  sampling  (Ball, 
1990)  to  select  four  students  for  semi-structured  interviews  “to  generate  the  information  upon  which  the  emergent 
design  and  grounded  theory  can  be  based”  (Lincoln  and  Cuba,  1985,  p.  201).  Combined  with  classroom 
conversations  and  observations,  these  individual  interviews  enabled  me  to  construct  portraits  (Lawrence-Lightfoot 
and  Hoffman,  1997)  or  mini-embedded  case  studies  of  four  students  in  the  teacher’s  eighth  grade  math  classes  (two 
students  from  each  class).  Each  individual  student  portrait  served  as  a rhetorical  device,  providing  vivid  renditions 
of  broadly  observable  patterns  of  class  behavior,  thereby  directing  and  redirecting  the  most  fruitful  path  of  research 
m answering  the  research  questions.  These  four  richly  detailed  student  portraits,  grounded  in  conversation  and 
interview  data,  facilitated  “the  on-going  joint  collection  and  analysis  of  data  associated  with  the  generation  of 
theory”  (Glaser  and  Strauss,  1 967,  p.  48). 

To  further  extend  my  understanding  of  the  research  context,  I analyzed  text  documents  gathered  from  the 
research  setting  (Glesne  and  Peshkin,  1992)  which  included  a printed  copy  of  the  school  mission,  the  Saxon  math 
texts,  students  homework  assignments,  DM  test  reports,  DM  progress  reports,  and  so  forth. 

Data  Analysis 


Given  the  documented  need  for  new,  theoretically  expressed  understandings  of  students’  and  teachers’ 
experiences  of  the  classroom  implementation  of  educational  technologies,  I chose  to  analyze  my  transcribed 
research  data  (classroom  observations,  classroom  conversations,  interviews)  and  related  documents,  using  grounded 
theory  methods.  “The  value  of  the  methodology. . . lies  in  its  ability  not  only  to  generate  theory  but  also  to  ground 
that  theory  in  data”  (Strauss  and  Corbin,  1998,  p.  8).  I proceeded  from  description  to  conceptual  ordering  and 
theorizing  (three  activides  foundational  to  Strauss  and  Corbin’s  theory  building  process),  by  iteratively  coding 
transcribed  data  regarding  students’  activities  in  their  math  class.  Strauss  and  Coibin  (1990)  explained  that  coding  is 
the  process  of  analyzing  data”  (1990,  p.  61),  in  order  to  create  theory  from  data. 

I developed  a systematic  three-step  process  for  coding  all  transcribed  data.  To  generate  initial  categories 
and  to  discover  relationships  between  these  categories,  I began  by  coding  transcribed  exploratory  interviews  and 
classroom  conversations  by  hand,  using  participants’  own  words  (in  vivo  codes)  when  possible.  I then  record  coded 
data  electronically  using  the  qualitative  data  analysis  software,  N.5.  In  parallel  with  my  use  of  N.5  to  code 
transcribed  data  (line-by-line  and  open  coding),  I employed  the  concept -mapping  software  program.  Inspiration  to 
explore  relationships  within  and  between  concepts  evident  in  the  data  (axial  coding).  The  multiple  representational 
methods  for  linking  categories,  which  Inspiration  afforded,  supported  development  and  refinement  of  my  emerging 
grounded  theory  (selective  and  focused  coding)  of  students’  experiences  with  DM  in  their  math  class. 

Flick  (1998)  noted  that  this  combination  of  multiple  methodological  practices  in  a single  study  is  best 
understood  as  “a  strategy  that  adds  rigor,  breadth,  complexity,  richness,  and  depth  to  any  inquiry”  (p.  231).  I 
employed  these  methods  of  data  collection  and  analysis  to  secure  an  in-depth  and  richly-triangulated  understanding 
ot  students  experiences  with  an  interactive  learning  system  in  their  math  class.  Maxwell’s  (1992)  five  kinds  of 
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understanding  and  validity  in  qualitative  research  (descriptive,  interpretive,  theoretical,  generalizability  and 
evaluative)  were  used  to  guide  all  phases  of  data  collection  and  data  analysis. 

Results  and  Discussion 

During  the  ten-week  implementation  of  DM  in  their  math  class,  students  experienced  a level  of  control  over 
their  own  learning  that  they  had  not  experienced  in  their  regular  math  class  using  Saxon.  Control  is  relative. 

Students  spoke  positively  about  the  level  of  control  afforded  to  them  vis  -a-vis  math  activities,  representation  of  math 
content  and  the  social  context  for  these  activities:  the  opportunity  to  interact  with  their  peers,  in  the  DM  math  class. 

Activity  Choices 

Students  who  completed  their  DM  tests  and  prescribed  assignments  could  choose  math  activities  from  a 
suite  of  DM  tutorials  (providing  repetition  of  the  problem  explanation,  partial  or  complete  explanation  of  the 
problem,  and  additional  practice  problems),  workouts  (sets  of  complex  problems),  or  progress  feedback  and  reports. 
Students  frequently  contrasted  the  variety  and  choice  of  math  activities  in  DM  with  the  limited  set  of  questions  in 
their  regular  math  textbook:  ‘‘With  DM  the  choices  are  more  fun  and  there’s  more  of  them  than  over  here  [regular 
math  class].”  Students  enjoyed  having  opportunities  to  become  informed  and  make  choices  about  their  math 
learning  with  DM:  “Well  instead  of  Just,  like,  being  told  what  to  do,  you  get  to  think  about  what  you’re  gonna  do. 
Like  you  can  choose  what  you  wanna  do.”  One  student  explained  that  the  activity  choices  in  DM  enabled  him  to 
control  what  and  how  he  learned  math:  “You  can  go  with  what  you  want  to  do,  and  what  you  think  you’re  capable  of 
learning.  1 like  that  you  can  go  at  your  own  pace. . . You’re  not  held  back  by  other  people.”  Researcher  observations 
suggest  that  when  using  DM,  students  made  instructional  choices  that  supported  their  own  math  learning. 

For  example,  as  students  gained  in  their  understanding  of  the  DM  activity  system,  they  used  the  activity 
options  within  DM  tests,  tutorials  and  workouts  to  increase  their  individual  access  to  detailed  explanations  of  math 
problems.  Although  students  were  not  instructed  by  their  teacher  to  use  the  DM  test  feedback^  test  report,  and 
progress  report  activity  options,  they  discovered  and  employed  these  tools  to  review,  reflect  upon,  and  regulate  their 
own  math  learning  with  DM.  Students  who  scored  their  DM  math  test  used  instantaneous  test-feedback  to  compare 
their  own  answers  to  test  questions  with  the  correct  DM  responses  provided.  The  correct-answer  feedback  provided 
in  DM  enabled  students  to  identify  the  source  of  their  errors  by  reviewing  and  reinterpreting  both  the  test  question 
and  their  own  problem  solving  strategies.  As  one  student  explained: 

...you Just  say,  “Score  test!”  and  then  it  tells  you  what  you  got,  and  you  can  go 
back,  and  see  what  the  answer  should  have  been  and  stuff  That  kind  of  makes  ya 
try  and  figure  it  out,  like,  why  you  didn’t  get  the  right  answer.  It  shows  you  the 
right  answer.  So  you  know  how  to  do  it. 

The  following  example  represents  one -student’s  verbal  response  to  test-feedback  provided  on  the  first  part  of  her 
test  on  integers: 

Oh,  why  did  I do  that?  Oh!  I don’t  know  why  I did  that!  This  one  was 
easy,  from  least  to  greatest.  What  did  I put?  [Locating  her  own  response]  What  the 
heck!  I put  them  from  greatest  to  least!  For  [number]  38  [Pause]  Oh,  I didn’t 
divide!  I never  found  the  average.  I was  supposed  to  divide  by  10!  And  I bet  that’s 
what  happened  in  the  last  one  as  well!  Now  this  one  is  six,  negative  six.  This  is 
positive,  this  is  negative.  So  how  is  it  equal?  [pause]  Oh  I didn’t  times  it  when  I 
got  the  number!  [Pause]  I really  need  to  read  the  questions! 

Students  used  DM  test  feedback  information  in  this  way,  to  interpret,  analyze  and  improve  their  own  math 
performance.  Clariana  (1990)  found  that  students  who  received  correct  answers  after  one  missed  attempt 
outperformed  students  who  were  required  to  repeatedly  answer  questions  until  reaching  the  correct  answer.  Clariana 
noted  that  students  who  received  correct  answers  after  one  missed  attempt  used  the  immediate  feedback  they 
received  to  clarify  misunderstandings  and  use  this  information  to  solve  successive  problems.  In  the  present  study, 
students  used  immediate  test  feedback  to  begin  to  understand  and  effect  positive  changes  in  their  own  learning. 

Students  who  completed  and  scored  a teacher-assigned  DM  test  also  received  a DM  test  report  which 
provided  information  regarding  the  number  of  correct,  incorrect  and  attempted  problems,  the  students’  percentage 
score,  and  the  student’s  performance  on  individual  learning  objectives  assessed  within  the  test.  Students  learned  to 
use  the  progress  report  data  provided  (e.g.,  percentage  of  problems  correct  for  each  activity,  amount  of  time  spent  on 
each  activity,  and  date  the  activity  was  completed)  to  identify  gaps  in  their  progress,  plan  make-up  work  and 
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navigate  the  activity  options  within  DM.  They  frequently  used  these  accessible  and  transparent  reports  of  their  DM 
work  to  critically  appraise  their  own  progress  with  the  learning  system. 

Students’  sophisticated  use  of  feedback  and  report  tools  to  effectively  control  their  own  learning  in  the  DM 
class  suggests  that  learners  can  develop  the  skill  and  will  (McCombs  & Marzano,  1990)  to  manage  their  own 
learning,  given  educational  technologies  which  provide  detailed,  accessible,  and  instantaneous  feedback  to  students 
regarding  their  learning. 

Multiple  Representations  of  Math  Concepts 

Students  also  contrasted  the  multiple  and  dynamic  methods  for  representation  of  concepts  (audio,  visual- 
graphics,  visual-text),  in  the  DM  scenario-based  math  problems,  with  the  static  problem  sets  assigned  in  their  regular 
math  textbook.  “The  workbook  pages  are  just  black  and  white.  It’s  hard  to  concentrate  in  the  math  class.  DM  is 
colorful,  and  it’s  more  fun.  It  has  cool  graphics  and  pictures  and  things.  You  can  just  listen  to  the  math  on  there.” 
Students  frequently  noted  the  benefits  of  the  multiple  pathways  to  learning  (speech,  graphics,  text)  provided  in  each 
DM  tutorial  and  workout  explanation.  They  suggested  that  these  three  modes  for  representation  of  problems 
supported  their  interest  and  engagement  in  math  during  their  completion  of  DM  problem  tasks. 

Referring  to  the  audio  presentation  of  math  concepts  in  DM,  one  student  insisted,  “It  makes  it  [math]  more 
interesting.  And  the  voices,  they  say  it  in  a fun  way,  not  like  in  a serious  way,  or  like  a baby  way.”  A peer  added, 
“You  understand  things  way  better  when  you’re  hearing  somebody  say  it  than  when  you’re  just  reading!”  Students 
noted  that  the  visual  representation  of  math  concepts  in  DM  also  increased  their  interest  in  the  DM  math  activity:  “If 
you  have  something  that  you  see  there  that  catches  your  eye,  you  actually  want  to  do  it  [math  activity].”  Students 
liked  the  use  of  animated  graphics  to  support  scenario -problems  in  DM,  and  claimed  that  these  graphic 
representations  of  math  content  supported  their  understanding  of  underlying  math  concepts: 

They  give  you  a lot  of  pictures  and  stuff  that  help.  The  one  sheep  thing,  like  when  he 
was  cutting  the  wool  off  the  sheep  to  make  the  things,  they  had  the  little  sheep  in  the 
little  circle.  Then  they  colored  the  circle  that  had  the  cut  sheep,  and  they  showed  you 
the  fractions  for  how  many  cut  sheep  there  were  to  the  whole  bunch.  So,  that’s  a 
picture  that  I found  helpful  to  use.  Besides  trying  to  figure  it  out  in  your  head,  you 
have  it  right  there  on  the  computer  screen.  You  can  just  count  it,  one  by  one. 

Students  noted  that  in  addition  to  the  audio  and  visual  representation  of  math  concepts  provided  in  tutorials 
and  workouts,  the  step-by  step  text  based  explanations  of  math  concepts  in  DM  also  supported  their  engagement  in 
the  math  activity.  One  student  explained,  “They  write  out  the  problem  for  ya  in  steps.  So  its  easy  to  follow.” 

Students  suggested  that  these  three  formats  (speech,  graphics,  text)  for  representation  of  math  concepts  in 
DM  motivated  them  to  attempt  math  problems,  and  sustained  their  interest  in  completing  difficult  word  problems: 

1 find  it  hard  to  do  the  story  problems.  I think  I got  a lot  better  at  ‘em  in  the  lab... 

Because  before  when  I’d  do  ‘em  out  of  the  book,  I just  [pause]  I can  do  ‘em,  I just 
keep,  well,  I have  to  read  that  little  paragraph  thing  there  and  I always  get  lost. 

In  their  research  comparing  the  performance  of  sixth  grade  students  using  single -representation  (SR)  and 
multiple -representation  (MR)  versions  of  a computer-based  multimedia  program  for  addition  and  subtraction  of 
signed  numbers,  Moreno  and  Mayer  (1999)  also  found  that  that  the  benefits  of  using  MRs  with  students  were 
strongest  on  difficult  problems.  Students  in  the  present  study  claimed  that  the  multiple  formats  (audio,  visual- 
graphics,  visual-text)  for  representation  of  math  concepts  in  DM  supported  their  engagement  with  difficult  math 
problems,  by  allowing  students  to  choose  how  they  wanted  to  engage  with  the  math  problem. 

Given  these  three  formats  for  representation  of  math  concepts  in  DM,  control  over  the  instructional  event 
(the  math  activity)  remained  a critical  factor  motivating  students’  interest  in  math  when  using  the  interactive 
learning  system  to  complete  math  problems.  One  student  explained,  “You  can  decide  if  you  want  to  listen  to  the 
problem,  or  look  at  it...  so  you  can’t  get  lost.  It  [choice  of  representational  formats]  makes  it  [math]  more 
interesting.”  This  research  suggests  that  multiple  pathways  to  learning  which  support  diverse  learning  preferences 
among  students  may  positively  affect  their  interest  in  subject  matter.  Hannafin  and  Sullivan  (1995)  found  that 
students  using  either  text-plus-static  graphics  or  text-plus-animated  graphics  methods  for  presentation  of  math 
topics  expressed  a more  positive  attitude  toward  math  than  those  who  viewed  the  text  -only  version.  In  this  study,  the 
flexibility  of  the  educational  technology  in  facilitating  multiple  representations  of  information  enabled  students  to 
learn  in  ways  that  supported  their  own  pedagogical  preferences.  Consequently,  students  found  math  more  interesting 
when  learning  with  the  DM  interactive  learning  system  which  presented  math  problems  through  more  than  one 
modality,  than  when  using  the  math  text  in  their  regular  eighth-grade  math  class. 
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Opportunities  for  Peer  Discussion 


In  the  regular  math  class  using  the  Saxon  scheme,  students  would  quietly  complete  their  practice  set 
following  the  teacher’s  correction  of  homework  and  presentation  of  the  new  increment.  The  teacher  neither  provided 
guidelines  regarding  whether  or  not  students  could  interact  with  one  another  during  the  DM  math  class  in  the 
computer  lab,  nor  organized  formal  groups  or  pairs  of  collaborators.  Students  spontaneously  devised  strategies  to 
extend  their  experience  with  DM  beyond  their  individual  interactions  with  the  interactive  learning  system  to  their 
peers  in  the  math  class.  For  example,  many  students  made  exhaustive  efforts  to  synchronize  their  use  of  DM 
tutorials,  workouts  and  tests  with  those  students  seated  on  either  side  of  them.  Students  who  coordinated  their  pacing 
of  DM  activities  in  this  way  consistently  demonstrated  high  levels  of  interest  and  enjoyment  in  the  math  activity 
which  often  proceeded  in  game  -like  manner  with  frequent  choruses:  “Ready,  Set,  Go!”  “One,  two  three!”  “Marks, 
set,  go!”  Students  in  Mrs.  Hall’s  classes  enjoyed  learning  math  with  their  peers.  Many  students  indicated  that  the 
quality  of  the  math  experience  improved  significantly  when  they  had  opportunities  to  work  with  one-another,  and 
make  instructional  decisions.  One  student  explained: 

It’s  a different  environment  thing.  You  know  you’re  just  sitting  around  doing  things, 
and  you  find  out  something  new  and  neat  that  you  just  got  to  show  some  one,  you 
know,  like  the  people  around  you.  Or  if  you’re  getting  frustrated  with  something,  the 
other  people  are  there  to  like,  talk  to  you  about  it. 

Students  rarely  sought  assistance  with  DM  math  problems  from  their  teacher,  relying  instead  on  their  peers 
for  math  conversation  and  argumentation.  Students’  discussions  of  math  while  using  DM  progressed  through  a 
sequence  of  stages  that  included  advice-seeking,  advice-giving,  evaluation,  comparison,  clarification,  acceptance  or 
rejection  of  alternative  rationales  and  defense  of  math  claims  or  assertions.  Table  1 presents  examples  of  student 
math  talk  at  each  of  these  stages: 

Table  1.  Components  of  Students ' Math-Talk  when  using  DM^ 


Activity 

Examples 

Request  help 

“Which  one  do  I click  on,  here?”  (seek  advice) 

“Oh  man!  What’s  this  about,  anyway?  (seek  help  interpreting) 
“What’s  an  absolute  value,  again?”  (seek  definition) 

Provide  assistance  or 
advice 

“Click  that  one,  man!”  (provide  answer  without  justification) 
“Because  that’s  relationship.”  (provide  partial  explanation) 
“Because  it’s  the  difference  from  Jack’s  home  to  the  school.  Do 
you  see  that  there?”  (provide  explanation) 

“Because  it’s  five  over  two.”  (provide  example  of  math  concept) 

Evaluate  or  compare 
options 

“That  looks  wrong.”  (assess) 

“This  one  is  better.”  (compare) 

“Yeah,  I had  that  one  figured  out,  too.”  (compare,  contrast) 

Seek  clarification 

“How  do  you  know  it’s  that  one?”  (seek  justification) 
“You  did  what,  again?”  (seek  reiteration  of  suggestion) 
“So  why  did  you  do  that,  there?”  (seek  justification) 

Accept  option 

“That’s  true.”  (agree) 

“Cool!”  (accept  without  indicating  motive) 

“Yeah!  That’s  what  I was  thinkin’  - divide  and  then  simplify.  I 
got  that  too!”  (accept  as  confirmation) 

Reject  option 

“No  it’s  not!  That’s  a proper  fraction  there,  JM!”  (contradict) 
“No.  Eight’s  the  denominator  here.”  (explain) 

“See,  smartie!  I told  you  it  was  this  one,  not  a whole  number!” 
(self-applaud) 

Defend  choice 

“Oh,  no  it  aint!  That’s  the  opposite  of  an  improper  fraction, 
there,  GS!”  (reject,  justify) 

“It  has  to  be  this  one  - I know  those  are  all  wrong!”  (justify  by 
elimination) 

“If  you  add  them  up  you’ll  get  four  sixths,  and  that  has  to  be  two 
thirds  in  lowest  terms.”  (recall  mathematical  proof) 

Note.  ^Excerpted  text  from  students’  classroom  conversations 
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The  informal  nature  of  students’  math-talk  (frequently  punctuated  with  vociferous  argumentation,  lavish 
expression  and  theatrical  gesture)  during  their  use  of  DM  belied  the  depth  of  students’  discussion  of  math  concepts. 
Through  the  production  of  argument  and  counter-argument,  students  frequently  attempted  to  persuade  their  peers 
that  certain  choices  or  decisions  were  preferable  to  concurrent  choices  or  decisions  in  resolving  DM  math  problems. 
Students  demanded  support  and  justification  for  mathematical  assertions  and  claims  presented  by  their  peers:  these 
demands  were  met  using  mathematical  data,  facts  and  evidence. 

While  this  research  supports  the  claim  that  novel  educational  technologies  can  trigger  a restructuring  of 
classroom  experience,  one  which  extends  and  elaborates  the  possibilities  for  student  interaction  (Kerr,  1996),  it  also 
shows  that  the  social  situation  for  students’  use  of  educational  technologies,  can  be  a powerful  determinant  of  their 
verbal  behavior  in  these  learning  environments. 

Increased  Interest  and  Engagement  in  Math 

For  the  three  reasons  discussed  (activity  choices,  multiple  representations  of  math  content,  and 
opportunities  for  peer  collaboration)  students  claimed  that  using  DM  increased  their  interest  in,  and  engagement  and 
productivity  with,  math:  “The  choices  are  just  more  fun,  and  there’s  more  of  them,  and  it’s  more  interesting  to  see 
an’  hear  the  math.  You’re  just  busy  all  the  time.”  One  student  explained,  “With  DM  you  actually  want  to  do  the 
math,  then.  And  it  means  you  actually  want  to  work  harder  and,  like,  get  it  done,  and  see  how  much  you  can  cover 
and  stuff”  Another  student  contrasted  his  poor  work  ethic  in  the  regular  math  class,  with  his  sense  of  efficacy  when 
learning  with  DM:  “Well  here  I could  actually  get  my  work  done.  I could  actually  do  my  work  in  here.  Over  there  I 
would  do  nothin’  cause  I’m  lazy  and  it’s  boring.  Here  it’s  a lot  better...  DM  helps.  It  gets  you  to  like  it  a little  more, 
so  you  actually  get  to  do  stuff.  You  want  to  do  work:  you  don’t  just  sit  there.” 

My  observations  of  students’  math  discussions  and  activities  with  DM  supported  their  self-reports  of 
increased  engagement  with  math  in  the  DM  class.  During  their  first  visits  to  the  computer  lab  to  use  DM,  I noted 
that  students  accessed  the  system  with  little  delay,  engaged  in  math-talk  with  one  another,  completed  prescribed 
assignments,  and  frequently  remained  in  the  computer  lab  beyond  the  five-minute  bell  which  signaled  the  end  of 
math  work  in  the  regular  math  class.  To  document  students’  level  of  engagement  with  math  when  using  DM,  I 
routinely  described  the  activities  of  a random  sample  of  approximately  one  third  of  the  students  in  each  of  the  eighth 
grade  math  classes  approximately  ten  minutes  into  each  class  session  with  DM,  occasionally  repeating  the  activity 
with  a new  sample  of  students  ten  minutes  before  the  end  of  each  session.  Table  2 summarizes  the  activities 
demonstrated  by  students  while  using  DM  in  the  computer  lab,  classified  as  time -on-task  and  time  -off -task,  based  on 
21  recorded  observations  (one  for  each  student  sample)  taken  in  15  DM  math  classes 

Table  2.  Description  ofTime-on-Task  Categories 


Time-on-Task Time-off-Task 

Student  engaged  in  math  talk:  Student  talk  is  not  about  math: 

Student  discusses  math  with  peer  or.  Student  asks  peer  for  gum  or  candy 

teacher 

Student  reads  or  thinks -aloud  math 
Student  engaged  in  math  activity: 

Student  performs  calculations  using 
calculator  or  paper 
Student  proceeds  from  one  screen  to 
another  without  visible  distraction 

Student  is  disengaged  from  math  activity: 
Student  randomly  clicks  buttons  on  screen 
without  obvious  purpose  or  progression 
Student  closes  DM  program  and  logs  off 

computer  before  the  5 minute  bell 

Student  faults  hardware  for  time-off-task: 
Student  restarts  computer 
Student  changes  monitor  settings 

Student  handles  power  chords 


Student  activity  is  not  about  math: 
Student  reads  book  of  English  poetry 
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To  further  explore  students’  claims  of  high  levels  of  engagement  with  math  in  the  DM  class,  I used  the  categories 
represented  in  Table  2 to  classify  the  behavior  of  each  student  within  each  observation  sample  as  either  on-task  or 
off-task.  Figure  2 presents  a graphical  representation  of  this  time -on-task  data,  which  was  taken  for  students  in  both 
their  regular  math  class,  during  the  four-weeks  prior  to  the  implementation  of  DM  (Figure  2 A),  and  in  the  DM  math 
class  during  the  ten -week  implementation  process  (Figure  2 B): 


Figure  2.  Students ' Time-on-Task  in  the  Math  Class 
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In  contrast  with  Figure  2 A which  represents  descriptions  of  students’  time -on-task  in  the  regular  math  class.  Figure 
2 B shows  that  an  average  of  95%  of  students  were  found  to  be  on-task  with  math,  when  using  DM.  From  the  first 
until  the  final  log,  there  is  very  little  variation  in  students’  time  spent  on-task,  while  using  DM.  Students’ 
engagement  with  math  in  the  DM  class,  defined  operationally  here  as  time  -on-task,  began  and  remained  high 
throughout  this  ten-week  implementation  of  the  interactive  learning  system  for  math.  This  analysis  supports 
students’  claims  that  they  were  more  interested  in  math,  and  more  engaged  in  math  activities  when  using  DM, 
compared  with  their  regular  math  class,  using  the  Saxon  math  scheme. 

Conclusions  and  Recommendations 

Existing  research  advocates  the  use  of  educational  technologies  in  classrooms  to  increase  students’  interest 
in  specific  subject  matter  (Webster,  1990;  Yusuf,  1995).  The  present  study  supports  this  claim,  and  proposes  that  the 
level  of  learner-control  afforded  to  students  by  an  educational  technology  is  positively  related  to  students’  level  of 
interest  in,  and  engagement  with,  the  subject  matter.  Researcher  observations  of  students’  use  of  the  interactive 
learning  system  in  their  math  class,  combined  with  individual  and  focus  group  interviews  with  students,  suggest  that 
the  activity  choices,  multiple  representation  formats  for  learning,  and  opportunities  for  peer  discussion  of  math 
concepts  which  students  experienced  when  they  used  DM  to  learn  math,  increased  students’  interest  in  math,  and 
resulted  in  high  levels  of  engagement  with  math.  Students  experienced  high  levels  of  control  over  the  instructional 
enterprise,  when  using  the  interactive  learning  system. 

While  learner  control  has  generally  been  used  to  refer  to  the  delegation  of  instructional  decisions  to  learners 
regarding  the  sequence  and  pacing  of  instructional  activities  and  the  identification  of  learning  needs  (Johnson  and 
Johnson,  1996),  this  research  suggests  that  learners’  decisions  regarding  the  social  context  for  their  learning  are  also 
critical  to  our  understanding  of  how  learner-control  evolves  during  the  classroom  implementation  of  educational 
technologies.  Further  exploration  of  students’  math  talk  while  using  educational  technologies  for  classroom  learning 
would  inform  our  understanding  of  how  students  support  one  another  in  developing  learner-control  strategies  during 
the  initial  implementation  of  educational  technologies  in  their  classes.  This  research  would  enable  us  to  identify 
strategies  that  students  adopt  or  co-create  to  control  their  own  learning  using  linear  and  open-ended  learning  systems 
in  classrooms. 

This  research  suggests  that  in  classroom  environments  which  afford  opportunities  for  students  to  display 
what  they  know  and  what  they  can  do  when  using  motivational  educational  technologies,  students’  classroom 
conversations,  or  math-talk  may  provide  compelling  evidence  of  their  understanding  of  math  concepts.  Implications 
for  classroom  practice  include  providing  opportunities  for  students  to  engage  in  sense-making  practices  with  their 
peers  as  they  work  to  understand  mathematical  concepts.  As  educators,  we  are  called  to  redress  our  unfamiliarity 
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with  children’s  ways  with  words  (Heath  1983)  - their  ways  of  organizing  their  experience  and  expressing  meaning, 
and  examine  how  technology  enhanced  learning  environments  may  provide  a catalyst  for  exploring  evidence  of 
understanding  in  students’  classroom  conversations. 

This  study  shows  that  by  increasing  students’  opportunities  exert  their  own  instructional  preferences  in 
learning  required  course  content,  educational  technologies  may  positively  influence  students’  interest  in,  and 
engagement  with,  subject  matter.  In  light  of  the  tangible  benefits  for  students,  this  research  challenges  us  to  further 
explore  the  evolution  of  learner-control  in  technology-enhanced  learning  environments,  and  thus  re-examine  the  role 
of  the  learner  in  instructional  contexts. 
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School  Library  Media  Specialists  and  Fair  Use:  The  Ultimate  Gray  Area? 

Rebecca  P.  Butler 
Northern  Illinois  University 


Introduction 

As  library  media  specialists,  we  often  find  that  we  are  considered,  along  with  our  administrators,  the  school 
experts  in  copyright  law.  Questions  we  may  ask  ourselves,  given  such  issues,  include:  (1)  Am  I liable  if  people  in 
my  school  violate  copyright  law  with  library  media  center  (LMC)  materials  and/or  equipment?  (2)  How  can  I stop 
illegal  copying  of  computer  software  in  my  school?  (3)  Are  schools  exempt  from  fair  use  guidelines  when  using 
print  and  non-print  media  for  instructional  purposes?  (4)  How  can  I encourage  teachers  to  comply  with  copyright 
law?  And,  (5)  is  copyright  law  really  a “gray  area?”  These  questions  and  more  will  be  addressed  in  this  article. 

We  will  look  first  at  copyright  law  and  fair  use,  as  well  as  the  educational  multimedia  guidelines  - two  areas  which 
support  some  copying  without  obtaining  author  or  owner  permission. 

What  is  Fair  Use? 

Section  107  of  the  1976  Copyright  Act  covers  “Fair  Use,”  a legal  principle  supporting  some  limitations  to 
copyright  holders’  exclusive  rights.  Therefore,  Fair  Use  is  law.  What  this  means  for  us  in  the  school  library  media 
center,  is  that  we,  our  faculty,  staff,  and  students  may  use  portions  of  copyrighted  materials  depending  on: 

( 1 ) purpose  and  character  of  use ; 

(2)  nature  of  the  work; 

(3)  portion  being  copied;  and 

(4)  marketability  of  the  work.* 

Fair  use  factor  #1 , the  purpose  and  character  of  use,  covers  how  the  user  wishes  to  use  the  copied  materials. 
Copying  of  a nonprofit,  educational,  or  personal  nature,  tilts  in  favor  of  fair  use.  Copying  of  a comme  rcial  nature 
tilts  in  favor  of  the  copyright  owner.  This  means,  in  some  instances,  that  an  educator  may  make  multiple  copies  of 
certain  works  for  teaching  purposes  but  may  not  sell  these  copies.  This  fair  use  factor  also  supports  quotations 
integrated  into  a paper  or  project,  whether  in  print  or  electronic  format. 

. Fair  use  factor  #2,  nature  of  the  work,  examines  characteristics  of  the  work  under  consideration  for 
copying.  It  considers  whether  the  work  is  fact  or  fiction,  published  or  unpublished.  Nonfiction,  published,  print 
works  are  most  likely  to  be  considered  fair  use,  given  this  factor.  This  means  that  a factual  article  on  the  history  of 
instructional  technology  (IT),  published  in  a professional  IT  journal,  is  more  likely  to  be  copy-able  than  is  an 
unpublished  story  written  by  a leader  in  the  field. 

Fair  use  factor  #3,  portion  being  copied,  considers  how  much  of  the  work  the  user  wishes  to  copy.  Given 
this  factor,  he  smaller  the  amount  used,  the  better.  However,  the  law  does  not  provide  exact  amo  unts.  Instead,  fair 
use  factor  #3  is  evaluated  two  ways:  quantitatively  and  qualitatively.  Quantity,  the  more  easily  measurable  piece, 
considers  the  amount  copied  relative  to  the  whole  as  well  as  the  amount  needed  to  achieve  the  copying  objective. 
Qualitative  measurement  is  much  more  creative.  This  measurement  is  concerned  with  the  concept  of  substantiality, 
whereby  the  copying  of  the  heart  of  the  work — no  matter  how  small — is  too  much.  An  example  of  this  might  be  a 
two -hour  long  video,  from  a particular  school  library  media  center  (SLMC),  whose  whole  story -line  culminates  with 
a minute-long  clip  on  the  most  challenged  item  to  remain  on  the  shelf  in  that  SLMC.  Although  the  culmination 
point  is  very  short,  if  that  is  the  heart  of  the  video,  then  that  particular  clip  is  too  much  to  copy. 

Fair  use  factor  #4,  marketability  of  the  work,  involves  the  effect  that  copying  part  or  all  of  a particular  work 
will  have  on  the  commercial  marketplace.  This  factor  focuses  on  economics.  For  example,  if  the  copying  of  the 
work  will  affect  how  much  money  the  copyright  owner  can  earn  off  sales  of  that  item,  then  any  copying  may  be  too 
much.  In  terms  of  this  fair  use  factor,  the  person  copying  needs  to  ask  the  question:  Is  my  copying  this  item  going 

to  hurt  its*  marketability?^’^’  ^ 

While  users  and  owners/creators  of  works  may  differ  widely  in  their  interpretations  of  the  fair  use  factors 
and  the  applicability  of  these  factors  to  various  circumstances,  the  more  common  interpretations  of  the  four  factors 
are  those  discussed  above.  It  is  important  to  remember  that  all  four  of  these  factors  should  in  place  for  any  copying 
to  be  considered  legal  under  copyright  law.  Next,  we  address  the  educational  multimedia  guidelines. 

Educational  Multimedia  Guidelines 
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The  Educational  Multimedia  Guidelines  are  not  law  and  are  much  more  restrictive  than  the  fair  use  factors 
covered  above.  However,  these  guidelines  are  supported  by  many  user  groups  and  organizations  representing 
owners,  because  the  guidelines,  if  properly  followed,  do  not  normally  violate  copyright  law.  Accordingly,  the  user 
may  borrow  (the  smaller  amount  the  better): 

Motion  Media:  10%  or  3 minutes 
Text:  10%  or  1000  words 
Music/Lyrics/Music  Video:  10%  or  30  seconds 

Illustrations/Photos:  5 by  an  individual  artist/photographer;  from  a published  collected  work:  10%  or  15 
images 

Numerical  Data  Sets:  10%  or  2500  fields/cells  5 

These  guidelines  are  for  multimedia  projects  only  and  are  more  restrictive  than  copyright  law.  However, 
many  users  apply  these  guidelines  to  all  types  of  media  under  the  assumption,  since  they  are  so  restrictive,  that  they 
will  then  not  be  liable  for  any  copyright  violations.  This  is  often  true,  but  does  not  have  to  be  so.  What  individual 
owners  or  creators  may  want  for  the  use  or  copies  of  their  works  varies  greatly.  6 

Am  I liable  if  faculty  in  my  school  violate  copyright  law  with  library  media  center  (LMC)  materials  and/or 
equipment? 

This  question  relates  to  school  library  media  specialist  (SLMS)  liability.  Answer:  copyright  law  follows 
the  “chain  of  command.”  Thus,  the  SLMS  might  find  him/herself  included  in  the  list  of  those  in  violation,  even  if 
s/he  had  no  knowledge  of  the  violation,  IF  s/he  had  provided  the  equipment  or  materials  used  in  the  illegal  copying. 
The  school  administrator  could  also  be  held  liable,  as  another  step  in  the  chain.  ^ 

How  can  I stop  illegal  copying  of  computer  software  in  my  school? 

Physically,  you  may  not  be  able  to.  However,  you  can  make  such  copying  harder  to  achieve,  by  keeping  all 
archival  software  copies  under  lock  and  key,  keeping  a record  of  all  software  licenses  in  an  obvious  place,  teaching 
and  informing  your  users  of  fair  use  and  other  copyright  law  applications,  and  placing  notices  that  illegal  use  of  the 
equipment  for  copying  may  result  in  fines  or  imp  risonment  on  all  equipment  that  can  be  used  to  copy.  ^ 

Are  schools  exempt  from  fair  use  guidelines  when  using  print  and  non-print  media  for  instructional 
purposes? 

A quick  and  dirty  answer:  no!  However,  many  owners,  publishers,  and  others  who  represent  those  who 
own  or  create  copyright  works  have  criteria  which  may  make  some  copying  for  educational  purposes  OK.  Each 
work  must  be  viewed  individually  to  see  if  it  would  be  in  violation  or  not.  9 

How  can  I encourage  teachers  to  comply  with  copyright  laW? 

While  there  is  no  one  answer  to  this  question,  there  are  many  things  which  can  make  copyright  compliance 
more  obvious  to  faculty.  Such  items  include:  by  example  (The  SLMS  demonstrates  proper  copyright  compliance.); 
providing  current  copyright  information;  keeping  all  archival  software  copies  under  lock  and  key;  keeping  a record 
of  all  software  licenses  in  an  obvious  place;  faculty  in -services,  i.e.,  teaching  and  informing  your  faculty  of  fair  use 
and  other  copyright  law  applications;  and  placing  notices  that  illegal  use  of  the  equipment  or  materials  for  copying 
may  result  in  fines  or  imprisonment.  In  addition,  the  SLMS  may  work  alone  or  in  tandem  with  other  teachers  to 
teach  copyright  compliance  to  students.  Thus  eventually,  students  may  observe  and  also  encourage  teachers  to  be 
copyright  compliant. 

Conclusion:  The  Ultimate  Gray  Area 

Copyright  issues  are  often  considered  the  “ultimate  gray  area,”  because  there  may  be  more  than  one  answer 
to  any  one  question,  depending  on  who  is  asking,  what  the  medium  under  question  is,  who  owns  that  medium,  and 
why  copying  or  use  of  it  is  important.  However,  the  bottom  line  is:  it’s  illegal  to  violate  copyright  law  - anytime. 
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anywhere,  for  any  reason.  Thus,  if  a user  is  sure  that  s/he  wants  to  use  or  copy  a work,  and  such  use/copying  falls 
outside  the  fair  use  guidelines,  it  is  important  to  make  sure  permission  has  been  obtained  from  the  owner/creator  of 
the  work  or. . . .don’t  copy. 
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Abstract 

The  growing  interest  in  the  field  of  educational  technology  in  the  design  of  constructivist  learning 
environments  has  led  to  a renewed  examination  of  problem-based  learning  (PBL),  an  approach  to 
instruction  in  which  all  learning  results  from  students ' efforts  to  solve  a complex  problem.  Concurrent 
advances  in  technology  make  it  possible  to  use  this  approach  with  a wider  range  of  audiences  than  have 
traditionally  used  PBL,  yet  few  guidelines  exist  for  designers  interested  in  creating  computer-based  PBL 
environments.  This  paper  presents  some  of  the  lessons  learned  from  the  process  of  developing  Alien 
Rescue,  winner  of  the  2001  Learning  Software  Design  Competition  sponsored  by  the  University  of 
Minnesota. 

Alien  Rescue 

Alien  Rescue  is  a computer-based  PBL  program  for  use  in  sixth  grade  science  classes.  The  primary 
learning  objectives  of  Alien  Rescue  focus  on  the  solar  system  and  the  scientific  instruments  used  to  investigate  it, 
though  the  program  offers  ties  to  other  areas  of  the  curriculum,  including  writing  and  mathematics. 

The  science  fiction  premise  of  Alien  Rescue  takes  students  to  a newly  operational  international  space 
station  where  they  become  a part  of  a worldwide  effort  to  rescue  alien  life  forms.  The  program  begins  with  the  7- 
minute  long  Opening  Scenario,  which  presents  the  central  problem  of  the  program,  and  which  students,  acting  as 
scientists,  are  asked  to  participate  in  solving.  A spaceship  carrying  six  species  of  aliens  fleeing  their  own  planetary 
system  have  arrived  in  Earth  orbit.  Their  ship  was  damaged  during  their  voyage,  and  except  for  their  engines  and 
computer  databases,  little  of  their  technology  continues  to  function.  In  order  to  survive,  they  must  find  new  homes 
on  worlds  that  can  support  their  life  forms.  Having  picked  up  Earth  broadcasts,  the  aliens  learned  our  languages 
with  the  intention  of  asking  for  our  help  to  relocate  to  worlds  in  our  solar  system.  However,  when  their  life  support 
failed  completely,  the  aliens  could  only  complete  a distress  message  to  be  sent  once  they  reached  Earth  orbit,  then 
entered  a state  of  suspended  animation,  where  they  must  remain  until  they  are  safely  relocated  to  suitable  worlds. 

Students  are  informed  that  they  are  just  one  of  many  teams  of  scientists  participating  in  this  rescue 
operation,  and  that  their  task  is  to  determine  the  most  suitable  relocation  site  for  each  alien  species.  In  order  to  solve 
this  problem,  students  must  engage  in  a variety  of  activities.  They  must  learn  about  the  aliens  and  identify  the  basic 
needs  of  each  species.  To  do  so,  students  search  the  Alien  Computer,  which  they  are  informed  was  moved  from  the 
alien  ship  to  the  international  space  station  so  that  they  could  conduct  their  research.  They  must  then  investigate  the 
planets  and  moons  of  our  solar  system,  searching  them  for  possible  matches  with  the  needs  of  the  aliens.  Students 
gather  this  information  in  two  ways.  They  can  search  the  Solar  System  Database,  a resource  within  the  program 
containing  information  about  the  sun,  nine  planets,  and  ten  of  the  large  moons  of  our  solar  system.  Then,  to  gather 
needed  information  that  is  missing  from  the  Solar  System  Database,  students  can  use  a simulation  within  the 
program  that  allows  them  to  design  and  launch  probes  to  other  worlds.  Students  interpret  the  data  returned  from 
these  probes,  applying  the  information  they  glean  back  to  the  problem.  Finally,  students  must  use  the  information 
they  gather  to  select  a new  homeworld  for  each  species  and  justify  their  decisions.  They  identify  their  choices  and 
write  a rationale  for  each  choice  in  the  Recommendation  Form.  To  discourage  students  from  developing  a solution 
without  time  for  adequate  investigation,  the  Recommendation  Form  does  not  appear  on  screen  until  students  have 
used  the  program  for  300  minutes. 

To  support  students  as  they  work  toward  developing  a solution.  Alien  Rescue  offers  the  Expert  Tool.  This 
tool  provides  video  of  a character  who  is  scientist  working  on  the  same  problem  as  the  students  as  he  interacts  with 
the  resources  within  the  program.  The  expert  models  his  process  as  he  woiks  on  finding  a home  for  one  of  the 
species.  He  does  not  help  students  with  the  other  species,  nor  does  he  direct  them  to  work  in  a particular  way. 
However,  by  making  explicit  the  strategies  an  expert  brings  to  bear  as  he  works  on  the  problem,  the  Expert  Tool 
supports  students  in  developing  effective  problem-solving  strategies. 

Alien  Rescue  is  an  example  of  a student-centered  learning  environment.  Students  are  engaged  in  a complex 
task,  but  they  determine  and  control  their  process.  Yet  they  are  not  expected  to  determine  this  process  or  develop  a 
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solution  on  their  own.  Alien  Rescue  was  designed  to  be  used  in  classrooms  rather  than  as  a product  for  home  use, 
and  we  consider  the  social  negotiation  that  occurs  as  students  interact  with  their  teacher  and  peers  to  be  an  essential 
source  of  support  for  learning  through  this  program.  The  teacher,  working  as  a facilitator,  pushes  students  to 
articulate  their  planning  and  reasoning,  challenges  their  misconceptions,  promotes  peer  collaboration,  and  builds 
connections  to  a wide  range  of  scientific  topics.  Peers  provide  support  for  the  complexities  of  the  environment  and 
help  each  other  to  refine  their  understandings  as  they  discuss/argue  the  merits  of  alternative  solution  plans. 

ProblenvBased  Learning 

PBL  is  an  instructional  approach  in  which  learning  occurs  as  a result  of  students’  efforts  to  develop  a 
solution  to  a complex  problem.  Instruction  begins  with  the  presentation  of  a problem  situation.  Students  work  in 
teams  to  identify  problem  constraints,  form  hypotheses,  collect  and  analyze  data,  and  develop  a solution  plan. 

Along  the  way,  students  discover  that  they  need  a great  deal  of  factual  information  in  order  to  understand  the 
problem,  determine  all  their  options,  and  develop  a viable  solution.  In  order  to  collect  it,  they  use  the  same  tools 
experts  would  use:  existing  informational  resources  such  as  books  and  computer  databases,  and  domain  specific 
tools  such  as  microscopes,  calculators,  or  maps. 

The  literature  on  PBL  has  suggested  a number  of  benefits  for  this  approach,  including  high  levels  of 
intrinsic  motivation  (Albanese  & Mitchell,  1993;  Stepien,  Gallagher,  & Workman,  1993),  enhanced  problem-solving 
skills  (Gallagher,  Stepien,  & Rosenthal,  1992;  Williams,  1993),  and  more  effective  self-directed  learning  (Aspy, 
Aspy,  & Quinby,  1993;  Blumberg  & Michael,  1992).  Despite  the  potential  for  learning  that  PBL  offers,  the 
realization  of  these  benefits  is  far  from  assured.  The  success  of  PBL  depends  on  students’  “willing  cognition,”  their 
intrinsically  motivated  efforts  to  gather  the  information  and  develop  the  skills  they  need  in  order  to  create  well- 
reasoned  solutions.  Without  this  investment  of  cognitive  effort,  students  may  fail  to  recognize  the  relevant  nuances 
of  the  problem,  neglect  to  identify  pertinent  learning  needs,  or  rush  to  develop  solutions  that  turn  out  to  be  non- 
viable.  Establishing  and  maintaining  this  willing  cognition  in  students  must  therefore  be  an  overarching  goal  for 
designers  of  PBL  programs. 

Guidelines  for  the  Design  of  PBL  Programs 

While  the  design  of  Alien  Rescue  was  informed  by  both  theory  and  research,  a number  of  additional 
insights  arose  through  the  process  of  development  and  testing.  In  the  remainder  of  this  paper  we  share  some  of  these 
insights  in  the  belief  that  they  can  help  to  guide  the  design  of  future  computer-based  PBL  environments.  The 
following  suggestions  address  issues  related  to  the  design  of  the  central  problem  of  a PBL  program,  the 
informational  resources  provided,  the  program  interface,  and  the  support  materials  for  classroom  teachers. 

Develop  an  interesting  and  rich  problem  that  creates  a need  for  information  and  multiple  applications  of  key 
strategies.  The  most  important  design  task  in  the  development  of  a PBL  program  is  the  creation  of  the  central 
problem.  This  problem  will  affect  students’  willingness  to  take  ownership  over  their  process  and  learning.  It  will 
also  determine  what  learning  needs  students  identify  and,  as  Stepien,  Gallagher,  and  Workman  (1993)  point  out, 
what  information  they  must  “run  into”  in  order  to  solve  the  problem.  To  promote  students’  “willing  cognition”  and 
maximize  the  potential  for  learning,  the  central  problem  of  a PBL  program  should  have  five  characteristics.  First,  it 
should  be  interesting  to  the  target  audience  in  order  to  promote  students’  ownership  over  their  process  and  solution. 
For  example,  in  our  early  planning  phase  for  Alien  Rescue,  we  initially  considered  having  students  identify  a world 
for  human  colonization.  We  eventually  decided  that  students  would  find  aliens  more  interesting  and  would  therefore 
be  more  intrinsically  motivated  to  invest  cognitively  in  solving  the  problem. 

Second,  the  problem  needs  to  be  challenging,  so  that  a solution  does  not  seem  obvious.  Students  are 
accustomed  to  well-structured  problems  where  all  necessary  information  is  presented  in  the  problem  itself,  the 
problem  has  a single  correct  solution,  and  the  time  to  solution  is  relatively  brief.  If  students  perceive  that  the 
problem  is  not  challenging,  they  may  fail  to  recognize  the  need  for  additional  knowledge  and  “solve”  the  problem 
without  fully  understanding  it.  Once  students  believe  that  they  have  completed  their  work,  they  may  be  reluctant  to 
invest  further  cognitive  effort  in  developing  a solution. 

Third,  the  problem  needs  to  be  manageable  so  that  students  persist  in  their  efforts  to  develop  a solution.  As 
students  recognize  the  complexity  of  the  problem,  they  may  become  frustrated  if  they  believe  that  it  is  too  difficult 
or  that  they  cannot  learn  all  they  need  to  know  in  order  to  develop  a solution.  Balancing  challenge  with 
manageability  and  effectively  communicating  both  to  students  can  be  quite  difficult.  In  our  testing  Alien 
Rescue,  we  have  found  that  students  generally  recognize  the  challenge  relatively  quickly,  but  take  some  time  to 
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recognize  its  manageability.  With  six  species  of  aliens,  each  with  a different  set  of  needs,  and  nineteen  worlds  to 
consider  as  possible  homes  for  them,  the  problem  requires  students  to  conduct  a great  deal  of  research,  a fact  which 
they  recognize  as  they  come  to  understand  the  problem.  Early  in  the  program  we  have  seen  some  students  express 
some  apprehension  and  frustration  as  they  experience  uncertainty  about  what  they  should  do  and  are  unable  to  solve 
the  problem  quickly.  This  typically  evaporates  as  students  come  to  understand  the  resources  within  the  program  and 
begin  to  develop  a process  to  work  toward  solution.  What  gets  students  over  this  hurdle  and  keeps  them  cognitively 
engaged  is  their  interest  in  the  problem  and  in  the  virtual  environment  in  which  the  problem  is  set.  In  this  way,  the 
rich  media  and  interesting  tools  provided  within  the  program  may  serve  to  motivate  students  to  persist  long  enough 
to  recognize  that  they  can  manage  the  challenge  the  problem  presents. 

Fourth,  the  central  problem  of  the  PBL  program  should  cause  students  to  recognize  the  need  for 
information  and  skills.  When  students  recognize  that  they  do  not  have  all  of  the  information  they  need,  they  seek  it, 
and  it  is  this  effort  to  meet  these  learning  needs  that  drives  students’  investigation  and  which  results  in  their  natural 
acquisition  of  key  terminology,  factual  information,  and  concepts  within  the  problem  domain.  Because  students 
have  determined  the  need  for  certain  knowledge  and  have  identified  a method  for  acquiring  that  knowledge,  all  their 
learning  is  meaningful  to  them.  It  is  therefore  essential  that  the  problem  lead  learners  to  recognize  that  they  do  not 
have  all  of  the  information  they  need  to  develop  a solution. 

Finally,  the  problem  should  create  multiple  opportunities  for  students  to  apply  the  same  problem-solving 
strategies  in  order  to  encourage  students  to  reflect  on  and  refine  them.  In  Alien  Rescue,  students  must  select  homes 
for  five  species  of  aliens  (the  expert  recommends  a solution  for  the  sixth).  While  the  needs  of  each  species  are 
different  and  the  conditions  on  each  world  vary,  the  process  for  developing  a solution  for  each  species  is  basically 
the  same.  Students  must  consider  different  factors  and  constraints,  but  they  can  apply  the  same  problem-solving 
strategies.  This  gives  students  an  opportunity  to  refine  their  process  and  reflect  on  what  is  effective. 

Design  miniature  problems  into  the  environment.  PBL  provides  not  only  a problem  but  also  a context  in  which  that 
problem  takes  place.  While  it  is  impossible  for  designers  to  control  all  aspects  of  the  PBL  experience,  it  is  possible 
to  design  small  complexities  into  that  context  that  learners  must  encounter  in  order  to  solve  the  problem.  The  need 
to  cope  with  constraints,  determine  alternative  resources  for  data  collection,  and  interpret  data  provides  challenges 
that  encourage  deep  thinking  and  an  ongoing  need  for  the  development  of  problem-solving  strategies.  This  helps  a 
PBL  environment  to  reflect  real  world  problem  solving,  where  overcoming  numerous  hurdles  is  a common 
necessity.  These  miniature  problems  also  provide  opportunities  for  rich  discussions  on  science. 

One  example  of  a miniature  problem  built  into  Alien  Rescue  occurs  in  the  alien  computer.  Rather  than 
using  the  names  of  elements  to  communicate  the  composition  of  their  atmospheres,  the  aliens  show  spectrograms. 
Spectrograms  show  the  spectral  signature  of  an  element;  the  spectrogram  of  each  element  is  unique  and  universal. 
The  aliens  communicate  this  information  using  spectrograms  ostensibly  because  they  were  unable  to  learn  the 
English  translation  for  elements.  However,  students  this  age  have  never  encountered  spectrograms  and  are  confused 
by  them  when  they  see  them  in  the  alien  computer,  yet  they  recognize  a need  for  the  information  these  spectrograms 
communicate.  A number  of  things  happen  in  class  as  a result  of  this  miniature  problem.  First,  the  teacher  can 
decline  to  tell  students  how  to  get  the  information  they  need  to  interpret  the  spectrogram  and  instead  use  this 
problem  as  a vehicle  for  encouraging  collaboration  and  peer  interdependence.  Second,  it  challenges  students  to 
investigate  more  deeply,  actively  exploring  the  environment  to  figure  out  how  to  solve  this  problem  themselves.  As 
some  students  do  solve  it,  they  become  experts  on  this  particular  problem,  and  when  their  peers  seek  help  they  can 
provide  it.  Finally,  these  miniature  problems  provide  a jumping  off  point  for  rich  class  discussions.  For  example, 
the  spectrogram  problem  can  lead  to  discussions  on  spectroscopy  or  starlight.  The  miniature  problems  Alien 

Rescue  can  lead  to  class  discussions  on  a wide  variety  of  science  topics,  including  radio  waves,  magnetic  fields, 
Galileo  and  the  moons  of  Jupiter,  geological  activity,  supernova,  ice,  gravity,  meteors,  and  atmospheres. 

Provide  access  to  all  necessary  information  within  the  environment  without  suggesting  the  usefulness  of  that 
information  in  the  development  of  a solution.  One  reason  that  teacher  development  of  PBL  programs  is  so  difficult 
is  that  students  must  have  access  to  an  adequate  number  of  resources  to  meet  their  learning  needs,  but  not  so  many 
that  they  are  overwhelmed  and  unable  to  find  the  information  they  need.  By  providing  all  necessary  information 
within  the  program,  designers  of  computer-based  PBL  programs  can  assure  its  accessibility.  However,  providing  a 
single,  well-structured  informational  resource  can  limit  the  range  of  considerations  students  make,  suggesting  a 
solution  without  requiring  sufficient  cognitive  effort  on  the  part  of  the  learner. 

Because  one  of  our  goals  in  Alien  Rescue  was  to  promote  students’  mindful  search  for  useful  information, 
we  wanted  to  avoid  providing  resources  that  encouraged  a passive  page-turner  mode  of  use,  or  which  were 
structured  to  suggest  certain  solutions.  To  accomplish  this,  we  used  two  strategies  to  maintain  the  complexity  of 


123 


612 


real-world  problem  solving  and  create  a need  for  students  to  work  purposefully.  First,  more  information  than  is 
needed  is  included  in  the  program  so  that  students  must  discriminate  between  what  is  useful  and  what  is  not.  For 
example,  the  alien  computer  contains  information  about  the  aliens’  needs,  but  it  also  contains  information  about  the 
uninhabited  worlds  in  their  solar  system,  their  journey,  and  their  languages.  Second,  the  needed  information  is 
divided  among  multiple  tools  so  that  students  must  consider  a variety  of  data  sources.  Some  of  these  resources  are 
purely  informational,  while  others  require  students  to  conduct  investigations  to  collect  data.  For  example,  the  Solar 
System  Database  contains  text  and  graphics,  while  the  probe  simulation  requires  students  to  design  probes,  launch 
them,  and  interpret  the  data  returned  from  them.  Taken  together,  these  strategies  encourage  students  to  think  about 
what  information  they  need  before  conducting  research  so  that  they  are  not  overwhelmed  by  irrelevant  facts.  This 
balance  of  accessibility  and  complexity  makes  it  possible  for  learners  to  be  successful  within  a challenging  program. 

Use  spatial  relationships  to  help  users  become  familiar  with  the  tools  available  within  the  environment.  The  large 
number  of  informational  resources  and  organizational  tools  users  need  within  a PBL  environment  presents  a 
challenge  for  the  design  of  the  interface.  Information  buried  several  layers  deep  within  the  interface  may  never  be 
found.  By  their  very  nature,  the  problems  used  in  PBL  occur  within  some  context;  a computer-based  program  can 
exploit  this  quality  by  creating  a virtual  setting  for  students’  work.  By  structuring  this  virtual  setting  so  that  spatial 
relationships  are  established  among  the  resources,  the  interface  can  help  learners  understand  and  remember  how  to 
access  useful  tools. 

Alien  Rescue  contains  thirteen  tools,  most  with  different  purposes  and  interface  features  particular  to 
themselves.  To  help  learners  understand  the  variety  of  these  tools  and  their  relationships,  we  created  a two-level 
interface.  The  first  level  is  the  international  space  station,  a futuristic  virtual  environment  which  consists  of  five 
rooms  that  learners  can  navigate  among,  using  the  arrow  keys  on  their  keyboards.  These  rooms  contain  resources 
students  will  only  need  for  part  of  the  program.  The  second  level  is  the  goggles  interface.  In  the  Opening  Scenario, 
students  are  told  to  imagine  that  they  are  wearing  goggles  that  provide  access  to  a variety  of  tools  throughout  their 
work.  Tabs  along  the  sides  and  bottom  of  the  screen  provide  access  to  these  tools,  and  students  can  open  and  close 
them  using  the  mouse.  This  two  level  interface  creates  relationships  between  the  tools  that  supports  students  in 
discovering  and  remembering  how  to  access  the  wide  variety  of  resources  available  within  the  program.  Our  testing 
with  Alien  Rescue  has  shown  that  students  are  typically  able  to  navigate  easily  among  the  various  tools  by  the  third 
day  of  class,  quickly  accessing  the  tools  they  need  to  conduct  their  work. 

Create  the  need,  opportunity,  and  support  for  collaboration.  The  need  for  information  that  is  not  instantly  apparent, 
a rich  content,  and  the  occurrence  of  numerous  small  problems  within  a PBL  program  all  add  to  the  complexity  of 
the  task.  Unable  to  handle  this  complexity  individually,  students  naturally  seek  support.  \n  Alien  Rescue,  we 
typically  witness  students  first  turning  to  their  teachers  for  help.  But  when  the  classroom  teacher  refuses  to  solve 
their  problems  for  them  and  instead  directs  them  toward  their  peers,  students  begin  to  understand  the  value  of 
collaboration.  Allowing  and  encouraging  a collaborative  environment  within  the  classroom  leads  students  to  see 
their  peers  as  resources  for  dealing  with  difficulties,  which  in  turn  can  enhance  both  learning  and  motivation. 

Designers  can  only  control  the  content  of  the  computer-based  programs  they  create;  they  exercise  little 
control  over  the  real-world  contexts  in  which  the  program  is  used.  Therefore  designers  can  only  create  a need  for 
collaboration.  Providing  the  opportunity  and  support  for  collaboration  is  the  responsibility  of  the  classroom  teacher. 
Support  materials  for  teachers  should  therefore  include  strategies  on  how  to  encourage  and  support  collaboration. 

The  teacher’s  manual  for  Alien  Rescue  offers  three  strategies  that  teachers  can  use  to  support  collaboration.  First,  as 
described  above,  teachers  can  redirect  students’  questions  to  their  peers.  This  giving  and  receiving  of  help  lays  a 
groundwork  for  more  complex  forms  of  collaboration,  such  as  shared  planning  and  division  of  labor.  As  some 
students  begin  to  develop  ongoing  collaborative  relationships  with  a few  peers,  teachers  can  use  a second  strategy, 
peer  modeling,  to  encourage  greater  collaboration  among  other  students.  In  peer  modeling,  the  teacher  asks  students 
who  are  successfully  collaborating  to  describe  their  process  to  their  classmates.  Teachers  may  guide  this  description 
through  the  use  of  specific  questions,  but  they  make  it  clear  that  students  developed  and  control  their  own  process, 
and  that  there  is  more  than  one  way  to  collaborate  successfully.  Through  the  use  of  peer  modeling,  teachers  provide 
students  with  examples  of  the  logistics  of  collaboration  and  legitimize  it  as  a successful  strategy  for  accomplishing 
complex  tasks.  Finally,  teachers  are  encouraged  to  discuss  the  role  of  collaboration  in  the  scientific  community. 

One  of  the  goals  of  Alien  Rescue  is  to  help  students  understand  the  real  work  of  scientists,  and  that  work  is  usually 
conducted  within  collaborative  communities.  Teachers  can  describe  how  scientists  typically  work  in  research  teams, 
pooling  their  various  areas  of  expertise  and  sharing  the  responsibility  for  the  investigations  they  conduct.  They  can 
also  explain  the  importance  of  scientists  publishing  their  findings  and  building  on  the  work  of  other  scientists. 

Again,  this  strategy  legitimizes  collaboration  by  framing  it  as  an  aspect  of  the  work  of  “real”  scientists. 
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Designing  effective  PBL  programs  is  a complicated  affair,  and  this  paper  addresses  only  a few  of  the 
lessons  learned  through  the  process  of  designing  Rescue.  As  problem-based  learning  becomes  a more  widely 
accepted  approach,  more  discussion  is  needed  about  effective  design  principles. 

For  more  information  on  Alien  Rescue,  visit  our  website  at  www.alienrescue.com 
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Abstract 


Trigger  visuals  (Cyrs,  1997)  created  by  graduate  students  using  MacroMedia  Director  in  a Computer 
Graphics  for  Learning  course  in  the  United  States  and  in  a Visualization  for  Learning  course  in  Poland 
were  shown.  These  were  shared  among  the  U.S.  and  Polish  students  and  were  discussed  in  Web-based 
computer  conferences.  The  content  and  style  of  the  U.S.  students'  visuals  were  analyzed  by  both  U.S.  and 
Polish  students  as  well  as  the  researchers  to  reveal  culture  as  it  was  manifested  through  the  students*  art 
shared  over  geographic  distances 

Overview 

Graduate  students  in  a Computer  Graphics  for  Education  course  in  the  United  States  and  in  a Visualization 
course  in  Poland  created  multimedia  “trigger  visuals”  using  multimedia  software.  The  “trigger  visuals”  presented  a 
scenario,  problem,  or  opportunity  in  order  to  trigger  an  emotional  rather  than  a rational  response  from  the  viewer 
and  to  stimulate  discussion  (Cyrs,  1997).  Then  students  attempted  to  share  their  visuals  with  each  other  as  Web 
documents  on  a shared  Web  site  and  discussed  their  contents  in  computer  conferences.  The  content  and  style  of  the 
U.S.  students'  visuals  were  analyzed  by  both  U.S.  and  Polish  students  as  well  as  the  researchers  to  reveal  culture  as  it 
was  manifested  through  the  students’  art  shared  over  geographic  distances. 

Theoretical  Framework 

Multicultural  understanding  is  the  appreciation  of  both  similarities  and  differences  as  well  as  beliefs, 
experiences,  values,  and  behaviors  across  distinct  and  identifiable  cultures  within  and  across  groups  and  societies 
(Timm,  1996).  A group’s  beliefs,  experiences,  values,  and  behavior  can  be  revealed  through  its  artistic  creations. 
Artistic  style  is  “visual  synthesis  of  the  elements,  techniques,  syntax,  inspiration,  expression,  and  basic  purpose” 
(Dondis,  1973).  This  style  “describes  the  means  by  which  aesthetic  ends  are  achieved,  the  values  reflected  in  those 
ends,  and  the  culture  within  which  those  values  prevail”  (Mullet  & Sano,  1995). 

Several  telecommunications  exchange  projects  have  emphasized  the  educational  goals  of  equity  and 
multicultural  understanding.  For  example,  AT&T  Learning  Circles  were  found  to  reduce  isolation  and  broaden 
students’  experiences  (Riel,  1995).  Similarly,  rural  Ohio  high  school  students  who  exchanged  email  with  adult 
mentors  realized  that,  “[t]he  components  of  a meaningful  life  had  apparently  changed  as  a result  of  their  interaction 
with  others  from  outside  their  usual  circle  of  contacts”  (Tille  & Hall,  1998,  p.  116).  Another  study  explored  cultural 
similarities  and  differences  manifested  through  children’s  artwork  shared  via  telecommunications  among  four 
teachers  and  their  students  in  Texas  and  in  Mexico  (Cifuentes  & Murphy,  2001). 

According  to  Hofstede  (1997),  cultures  differ  across  four  dichotomous  dimensions:  large  vs.  small  power 
distance,  strong  vs.  weak  uncertainty  avoidance,  individualism  vs.  collectivism,  and  masculinity  vs.  femininity. 

These  four  dimensions  are  described  in  depth  by  Hofstede  and  can  be  used  to  understand  and  explain  phenomena  in 
cultures.  For  instance,  Hofstede  used  the  four  dimensions  to  characterize  interactive  styles  across  several 
nationalities. 

Cultures  also  differ  in  their  communication  styles.  Hall  (1976)  distinguishes  between  high  context 
communication  and  low  context  communication.  In  high  context  cultures  such  as  that  of  Poland,  communication 
relies  on  indirect  verbal  messages  that  are  dependent  on  context  clues.  In  such  cultures,  “very  little  is  in  the  coded, 
explicit,  transmitted  part  of  the  message”  (p.  91).  Thus  students  in  high  context  cultures  are  likely  to  write  less  and 
rely  more  on  the  physical  context.  In  low  context  cultures  such  as  that  of  the  United  States,  on  the  other  hand, 
messages  tend  to  be  direct,  explicit,  and  highly  structured.  Understanding  these  differences  is  critical  for 
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telecommunications  partners  with  different  cultural  styles.  Clearly,  there  is  much  to  learn  from  multicultural  and 
intercultural  experience. 

Methods 


An  instructor  in  Texas  and  one  in  Poland  each  required  their  students  to  develop  trigger  visuals.  The  20 
U.S.  students  were  required  to  post  them  on  the  Web.  Four  Polish  students  participated.  Students  were  then 
encouraged  to  discuss  the  contents  of  the  visuals  in  computer  conferences  using  FirstClass'^'^. 

We  used  the  two-phase  process  of  content  analysis,  open  coding  and  focused  coding  as  described  by 
Emerson,  Fretz,  and  Shaw  (1995),  to  analyze  the  trigger  visuals  {see:  Boud,  Pearson,  1984;  Cullen,  1989;  Dylak, 
1995)  created  by  U.S.  students  and  the  computer  conferences.  During  open  coding  we  analyzed  the  visuals  to 
identify  ideas,  themes,  or  issues  {if  it  make  sense  - inspired  or  triggered).  This  process  involved  writing  initial 
memos  to  ourselves  (Miles  & Huberman,  1994).  For  focused  coding,  we  examined  the  visuals  as  well  as  our  memos 
on  a item-by-item  basis,  giving  special  attention  to  categories  identified  during  open  coding,  Hofstede*s  four 
dimensions,  and  Hall’s  theory  of  context.  Doctoral  students  received  training  and  then  examined  the  visuals  to 
categorize  them  according  to  those  dimensions  and  provided  a rationale  for  their  categorization.  They  independently 
completed  a matrix  of  the  dimensions  as  manifested  through  the  visuals  for  the  U.S.  students’  cultures.  After 
compiling  students’  categorizations  and  rationales  in  a summary  matrix,  we  paid  closest  attention  to  the  students’ 
rationales.  Next,  we  will  insert  our  own  judgment  and  understanding  to  summarize  and  draw  conclusions  about  the 
data. 

Findings  and  Educational  Value 

Artifacts  have  been  collected  and  analyzed;  however,  it  is  too  soon  to  draw  conclusions  regarding  what  data 
reveal  about  the  cultures.  Learning  networks  allow  students  to  virtually  cross  borders  to  collaborate  with  distant 
others  so  that  they  “actively  construct  knowledge  by  formulating  ideas  into  words  that  are  shared  with  and  built 
upon  through  the  reactions  and  responses  of  others”  (Harasim,  Hiltz,  Teles,  & Turoff,  1995,  p.  4). 

Learning  environments  composed  of  multicultural  learners  can  foster  understanding  between  peoples  on 
opposite  sides  of  geographic  borders  to  fulfill  the  desirable  ends  of  education  (Postman,  1995)  and  prepare  students 
for  world  citizenship  (Parker  et  al.,  1999).  Fabos  and  Young  (1999)  charge  that  one  way  to  “cultivate  more  critical 
and  political  sensibilities  among  students”  is  to  replace  existing  theories  of  the  “other”  or  the  “foreign”  with  ones 
that  “relate  the  content  of  telecommunication  projects  to  students’  individual  and  collective  lives,  that  analyze 
broader  social  and  political  issues”  (p.  241).  This  study  of  shared  multimedia  explored  the  possibility  of  learning 
how  to  analyze  artifacts  to  gain  understanding  of  distant  others.  The  study  demonstrated  that  students  can  share 
artifacts  and  then  interpret  them  to  learn  about  cultures.  They  can  gain  insight  regarding  each  other’s  similarities  and 
differences  by  participating  in  telecommunications  partnerships.  In  networked  classrooms  students  can  connect  with 
distant  others  to  learn  about  and  from  their  perspectives.  In  addition,  distance  technologies  can  foster  team  teaching 
and  multicultural  relationships  across  geographical  distance. 
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Abstract 

This  study  investigated  skills  that  lead  to  generation  of  facilitative  visualizations  during  study  time  for 
college  age  students.  An  orientation  prepared  students  in  an  experimental  group  to  1)  identify  the 
underlying  structure  of  a given  text,  2)  represent  identified  interrelationships,  3)  generate  pictorial  as  well 
as  verbal  study-notes,  4)  visually  connect  concepts  to  prior  learning,  and  5)  illustrate  distinctive  features  of 
concepts.  Students  who  showed  interrelationships  among  concepts  in  their  study -notes  performed  better  on 
a test  on  science  concepts  than  those  who  did  not.  No  other  significant  effects  were  identified. 

Background  and  Theoretical  Perspective 

In  Visual  Communicating.  Wileman  defines  visualization  as  the  process  of  graphically  or  pictorially 
representing  facts,  directions,  processes,  data,  oiganizational  structures,  places,  chronologies,  generalizations, 
theories,  and  feelings  or  attitudes  (1993).  In  our  study,  we  are  interested  in  the  power  of  student-generated 
visualization  to  facilitate  learning.  As  Dewey  remarked  early  in  the  20*^  century,  the  work  of  instruction  would  be 
“infinitely  facilitated”  if  teachers  would  see  to  it  that  their  students  were  “forming  proper  images....  The  image  is 
the  great  instrument  in  instruction.  What  a student  gets  out  of  any  subject ...  is  simply  the  images  which  he  himself 
forms  with  regard  to  it.”  Dewey  goes  on  to  say  that  teachers  would  be  wise  to  spend  time  “training  the  student’s 
power  of  imagery  and  in  seeing  to  it  that  he  is  continually  forming  definite,  vivid,  and  growing  images  of  the  various 
subjects  with  which  he  comes  in  contact  in  his  experience”  (as  cited  in  Wileman,  1993,  p.7). 

Student-generated  visuals  surpass  illustrations  in  their  effectiveness  for  instruction  because  they  are  more 
personally  meaningful  and  relevant  to  students’  understandings  and  prior  knowledge,  and  because  they  contribute  to 
construction  of  meaning  (Anderson-Inman  & Zeitz,  1993;  Finke,  1990;  Gobert  & Clement,  1999;  Papert,  1991). 
When  students  are  able  to  manipulate  images  during  knowledge  construction,  they  tend  to  engage  more  in  the 
meaning-making  process  and  understand  and  remember  concepts  better  than  through  the  traditional  transmission 
approach  of  instruction  (Jonassen,  2000).  Additionally,  students’  visualizations  manifest  the  content  and  the 
structure  of  their  knowledge  of  concepts,  which  provides  their  teachers  with  access  to  their  levels  of  understanding. 

Objectives 

This  study  investigated  the  skills  that  lead  to  generation  of  facilitative  visualizations  during  study  time. 
Specially,  researchers  examine  the  effects  of  teacher  encouragement  to  generate  visualizations,  the  effects  of  teacher 
encouragement  and  orientation  to  visualization,  and  the  effects  of  the  use  of  each  of  four  visualization  skills  and 
rehearsal  on  test  scores. 

Methods 


The  participants  in  this  study  were  75  undergraduate  students  in  an  introductory  oceanography  course  at  a 
large  university.  Twenty  participants  were  sophomores,  22  participants  were  seniors,  and  others  are  juniors.  Of  the 
75  students  who  attended  class  to  participate,  27  were  male  and  48  were  female.  The  participants  represented  22 
majors  from  throughout  the  university.  Thirty  participants  were  education  majors.  The  other  45  students  were  from 
majors  ranging  from  journalism  to  biology.  Participants  were  randomly  assigned  to  three  groups  in  a posttest-only- 
control-group  design.  The  data  sources  included:  (a)  test  scores,  (b)  students'  study  notes,  (c)  "The  Student's 
Questionnaire",  and  (d)  the  researchers' journals. 

• Unguided  group-  received  75  minutes  of  placebo  instruction  on  use  of  bacteria  to  clean  the  Valdez, 
Alaska  oil  spill  from  the  investigator.  It  then  received  an  essay  on  the  ocean’s  role  in  the  greenhouse 
effect  to  study  and  students  were  given  one  hour  for  unguided,  independent  study  prior  to  taking  a test. 
Students  handed  in  study-notes  before  being  tested  on  the  concepts  of  the  text. 

• Encouraged-to-visualize  group-  received  75  minutes  of  the  placebo  instruction  from  the 
investigator.  It  then  received  the  essay  on  the  ocean’s  role  in  the  greenhouse  effect  to  study  and 
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students  were  encouraged  to  visualize  during  study  time.  Students  were  given  one  hour  for 
independent  study  prior  to  taking  test.  They  handed  in  study-notes  before  being  tested  on  the  concepts 
of  the  text. 

• Oriented  group-  was  asked  to  study  two  paragraphs  of  text  and  handed  in  their  study -notes  after 
studying.  Based  on  the  analysis  of  students’  study-notes,  the  investigator  was  able  to  classify  the 
students  in  this  group  as  either  visualizers  or  non-visualizers  prior  to  receiving  the  orientation.  Then, 
the  group  received  a 75 -minute  orientation  to  visualization  from  the  investigator.  It  then  received  an 
essay  on  the  ocean’s  role  in  the  greenhouse  effect  to  study  and  students  were  encouraged  to  visualize 
during  study  time.  Students  were  given  one  hour  for  independent  study  prior  to  the  test.  They  handed 
in  study-notes  and  were  then  tested  on  the  concepts  of  the  text. 

Placebo  instruction  was  designed  so  that  all  three  treatment-groups  would  spend  equal  time  with  the 
investigator.  For  the  placebo,  the  investigator  presented  a research  study  on  the  use  of  bacteria  to  clean  up  oil  spills. 

The  oriented  group  received  a 75-minutes  orientation  to  visualization.  Students  examined  short  text  in  order 
to  practice  identifying  underlying  structure  of  that  text  that  can  be  expressed  as  one  of  the  following 
interrelationships — causal,  oppositional,  sequential,  chronological,  categorical,  comparative,  or  hierarchical.  The 
instructor  then  introduced  the  students  to  the  concept  of  self-generated  visualization  as  a study  strategy.  An  advance 
organizer  of  the  four  skills  and  rehearsal  was  presented.  The  four  skills  were  — ( 1 ) represent  identified 
interrelationships,  (2)  generate  pictorial  as  well  as  verbal  study-notes,  (3)  visually  connect  concepts  to  prior  learning, 
and  (4)  illustrate  distinctive  features  of  concepts.  The  instructor  modeled  generation  of  visualizations  of  the  text  on 
the  overhead  transparency.  After  each  example  was  modeled,  students  practiced  generating  visualizations  using  each 
of  the  skills  one  at  a time.  They  then  shared  their  visualizations  with  other  students  and  received  feedback  from  the 
students  regarding  the  effectiveness  of  the  visualization  at  communicating  the  text,  which  was  visualized.  Because 
the  training  and  practice  time  was  limited,  students  had  opportunities  to  visualize  short  pieces  of  text  containing  only 
one  concept. 

The  easy  for  studying  was  6 1/2  double -spaced  pages  of  text  written  at  the  12*^  grade  level  on  the  ocean’s 
role  in  the  green  house  effect.  All  participants  took  the  test  at  the  end  of  an  hour  study  to  determine  the  effects  of  the 
experimental  treatment.  The  test  contained  30  multiple  choice  questions,  which  was  criterion  referenced  according 
to  the  objectives  of  the  test  student  studied  (r=.  86)  and  was  validated  by  five  oceanographers.  The  test  scores  were 
compared  across  groups. 

In  addition,  three  raters  analyzed  each  student’s  study-notes  and  notes  in  the  textual  material  to  estimate  the 
extent  to  which  each  student  applied  the  four  visualization  skills  when  generating  study-notes.  Application  of  each 
skill  was  estimated  on  a scale  of  1 to  5 on  “The  Visualization  Skills  Inventory”  (Cifuentes,  1992).  For  example,  if 
students  made  no  attempt  to  show  interrelationships  they  received  a 1 . If  they  showed  interrelationships  with 
numbering,  highlighting,  arrows,  etc.  they  scored  a 2.  If  they  showed  interrelationships  through  the  generation  of  an 
outline  or  other  primarily  verbal  means,  they  scored  3 points.  If  they  showed  interrelationships  by  creating  a 
structural  diagram  or  a verbal/pictorial  representation  showing  one  relationship,  they  scored  4 points.  If  they  showed 
interrelationships  by  creating  a structural  diagram  or  a verbal/pictorial  representation  showing  more  than  one 
relationship,  they  scored  5 points  (see  Figure  1).  The  interrater  reliability  was  reported  as  r=  .97  for  showing 
interrelationships,  r=  .97  for  balanced  pictorial/verbal  notes,  r=  .96  for  showing  relationships  to  previously  know 
material  and  r=  .99  for  showing  distinctive  features. 

Participants  were  classified  as  either  visualizers  or  non-visualizers  by  examining  the  unguided  group  and 
encouraged  to  visualize  group’s  study -notes  that  participants  generated  while  they  studied  ’’The  Greenhouse  Effect” 
and  by  examining  the  study -notes  generated  by  the  oriented  toward  visualization  group  prior  to  the  orientation.  If 
subjects  in  each  of  the  groups  included  any  visualization  in  their  study-notes,  they  were  classified  as  visualizers. 

After  taking  the  test,  all  participants  filled  in  ’’The  Student’s  Questionnaire”  that  asked  them  to  rate  the 
extent  that  they  had  previously  been  exposed  to  the  information  in  the  greenhouse  effect.  To  determine  if  groups 
varied  in  their  prior  knowledge  of  the  textual  material,  an  ANOVA  was  conducted.  No  difference  was  found.  The 
questionnaire  also  asked  students  to  report  the  extent  to  which  they  used  their  study-notes  to  rehearse  when 
preparing  for  the  test.  The  extent  to  which  students'  used  their  study-notes  for  rehearsal  was  estimated  from  question 
1 on  the  questionnaire.  Participants  who  scored  a one  or  a two  were  categorized  as  non-skill -users.  Participants  who 
scored  a three,  four,  five  or  six  on  were  categorized  as  skill  users.  Additionally,  students  were  asked  to  describe  in 
detail  the  steps  that  they  took  to  prepare  for  the  test. 
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Figure  1.  Skill  users  vs.  non -skill  users  of  showing  interrelationships  after  the  orientation  (scale  of  1-5  on  each  skill). 


We  applied  content  analyses  approaches,  as  described  by  Emerson,  Fretz,  and  Shaw  (1995),  to  the 
researchers’  journal  entries,  students’  study-notes,  and  questionnaire  results.  During  and  upon  completion  of  data 
collection,  we  used  the  two-phase  process  of  content  analyses,  open  coding  and  focused  coding,  to  analyze  the  data 
and  identify  factors  contributing  to  the  effectiveness  or  lack  of  effectiveness  of  visualization  as  a study  strategy  for 
learning  the  text  regarding  ocean’s  role  in  the  greenhouse  effect. 

Results 


Based  upon  subjects’  self-reports  regarding  their  prior  knowledge  of  the  ocean’s  role  in  the  greenhouse 
effect,  it  was  determined  that  groups  did  not  differ  in  their  prior  knowledge  of  the  text.  However,  groups  did  differ  in 
their  use  of  study  time.  Most  importantly,  subjects  in  the  oriented  group  wanted  more  study  time  to  prepare  for  the 
test  while  students  in  the  unguided  group  and  the  encourage  to  visualize  group  wanted  less  time  to  prepare  for  the 
test. 

Regarding  subjects’  prior  knowledge  of  visualization,  30  of  the  75  subjects  were  education  majors  and  were 
likely  to  have  been  exposed  to  visualization  in  education  classes  at  the  university.  Informal  interviews  revealed  that 
at  least  5 education  majors  had  been  exposed  to  the  strategy.  Subjects  were  categorized  as  visualizers  or 
nonvisualizers  based  upon  their  note-taking  strategies  prior  to  orientation  to  visualization.  Nine  of  the  1 1 identified 
visualizers  were  education  majors.  Results  of  an  ANOVA  revealed  that  there  were  no  significant  differences 
between  visualizers  and  nonvisualizers  on  the  test  (see  Table  1). 


Unguided 

Encouraged 

Oriented 

Total 

Mean  of  Test  Score 

Visualize  r 

2 

8 

1 

11 

86.06  % 

Non-visualizer  ] 

18 

20 

26 

64 

79.40% 

There  were  no  statistically  significant  differences  identified  between  groups  on  test  scores  (see 

Table  2.  Group  Contrasts  of  Performance  on  the  Test 

Contrast 

df 

SS 

MS 

F 

P 

Effect  Size 

Unguided  vs  Encouraged 

1 

1.37 

1.37 

0.16 

0.69 

0.12 

Encouraged  vs  Oriented 

1 

2.99 

2.99 

0.34 

0.56 

-0.24 

Unguided  vs  Oriented 

1 

0.18 

0.18 

0.02 

0.89 

-0.14 

alpha  = .017 


However,  ANOVA  revealed  that  students  who  showed  interrelationships  among  concepts  in  their  study-notes 
performed  better  on  the  test  than  did  students  who  did  not  show  interrelationships  among  concepts  in  their  study- 
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notes.  No  other  statistically  significant  differences  were  found  (see  Table  3).  In  addition,  Cohen’s  d indicated  a 
positive  medium  effect  size  (d=0.57)  for  the  pairwise  comparison  of  those  participants  who  showed 
interrelationships  in  their  study  notes  and  those  participants  who  did  not  show  interrelationships.  Additionally,  the  d 
for  the  pairwise  comparison  of  those  students  who  balanced  their  notes  pictorially  and  verbally  and  those  students 
who  did  not  was  0.37,  and  the  d for  those  students  who  connected  concepts  to  their  prior  learning  and  those  students 
who  did  not  was  61.34.  These  practical  effect  sizes  indicate  that  showing  interrelationships,  balancing  pictorially  and 
verbally,  and  relating  new  concepts  to  prior  learning  made  a significant  difference  on  test  scores  (see  Table  4). 


Source 

df 

SS 

MS 

F 

P 

Show  Interrelationships 

Between  Groups 

1 

42.61 

42.61 

5.22 

0.02* 

Within  Groups 

73 

596.06 

8.17 

Total 

74 

638.66 

Balance  Pictorially/Verbally 

Between  Groups 

1 

20.94 

20.94 

2.47 

0.10 

Within  Groups 

73 

617.72 

6.46 

Total 

74 

638.66 

Relate  New  to  Old  Material 

Between  Groups 

1 

17.41 

17.41 

2.05 

0.15 

Within  Groups 

73 

621.25 

8.51 

Total 

638.66 

Show  Distinctive  Features 

Between  Groups 

1 

6.72 

6.72 

0.76 

0.38 

Within  Groups 

73 

631.94 

8.66 

Total 

74 

638.66 

Rehearse 

Between  Groups 

1 

2.22 

2.22 

0.26 

0.61 

Within  Groups 

73 

636.44 

8.72 

Total 

74 

638.66 

‘ p<.05 


Table  4.  Mean  Scores  and  Standard  Deviations  on  the  Test  for  Those  Who  Used  and  Those  Who  Did  Not  Use 
Four  Visualization  Skills  and  Rehearsal 


Skill  Use  in  Study-notes 

N 

Test 

Score 

Mean 

Standard  Skewness  Kurtosis 
Deviation 

Effect 

Size^ 

Show  Interrelationships 

52 

81.79 

9.30 

-0.27 

-0.42 

0.57 

Did  not  Show  Interrelationships 

23 

76.30 

9.89 

-0.04 

-1.23 

Balance  Pictorially/Verbally 

39 

81.82 

9.40 

-0.27 

-0.40 

0.37 

Did  not  Balance  Pictorially/Veibally 

36 

78.25 

9.93 

-0.08 

-.097 

Relate  New  to  Old  Material 

38 

81.76 

9.15 

-0.33 

-0.18 

0.34 

Did  not  Relate  New  to  Old  Material 

37 

78.50 

10.17 

-0.02 

-1.00 

Show  Distinctive  Features 

45 

80.93 

9.80 

-0.20 

-0.68 

0.21 

Did  not  Show  Distinctive  Features 

30 

78.86 

9.72 

-0.21 

-0.79 

Rehearse 

54 

80.46 

9.36 

-0.29 

-0.45 

0.13 

Did  not  Rehearse 

21 

79.19 

10.91 

0.20 

-1.06 

Effect  Size  show  interrelationship  * ^ did  not  show  interrelationship)  / SD  weighted 
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The  subjects  who  were  oriented  to  visualization  by  the  investigator  were  asked  to  describe  their  reactions  to 
visualization  as  a study  strategy.  Comments  were  generally  positive.  Twenty-one  of  the  27  subjects  used  the  words 
"helpful**  or  "useful"  when  expressing  their  opinions  of  visualization.  However,  opinions  conflicted  regarding  the 
efficiency  of  generating  visualizations.  Some  students  felt  that  the  time  visualizing  was  time  well  spent;  others  felt 
that  visualization  took  too  much  time  for  busy  students  whose  time  is  limited.  Opinions  also  conflicted  regarding 
the  amount  of  effort  required  to  visualize.  Some  felt  that  visualization  was  difficult  while  others  felt  that  it  was 
easy.  Conflicts  like  these  indicate  that  individual  differences  between  subjects  contributed  to  their  opinions  of 
visualization.  Perhaps  the  students  who  found  that  visualization  was  easy  and  time  saving  were  visual  thinkers, 
while  those  who  found  that  visualization  was  hard  and  time  consuming  were  not  visual  thinkers. 

Educational  Signiflcance 

The  study  is  important  in  that  it  provides  evidence  in  the  growing  body  of  visualization  research.  Based 
upon  the  findings,  it  is  recommended  that  students  be  trained  to  represent  interrelationships  that  are  causal, 
oppositional,  sequential,  chronological,  comparative,  categorical  and  hierarchical.  More  powerful  orientations  to  . 
visualization  need  to  be  designed  and  implemented  in  order  to  investigate  the  effects  of  using  visualization 
instruction  on  learning.  Training  sessions  might  need  to  be  longer  than  the  visualization  orientation  in  this  study  and 
should  provide  more  guidance,  practice  and  feedback  with  textual  material  that  is  representative  of  the  length  of 
material  that  students  are  required  to  remember  for  classroom  testing.  A study  similar  to  this  study  should  be 
conducted  allowing  for  flexible  study  time.  In  such  a study,  students  would  take  the  test  when  they  want  to  so  that 
time  would  not  be  a limitation.  Based  upon  student  behavior  in  this  study,  students  trained  in  visualization  skills 
might  choose  to  spend  more  time  -on-task  and,  therefore,  might  out  perform  untrained  students. 

Also,  upon  development  of  a reliable  and  valid  measure  of  visual  ability,  aptitude-treatment-interaction 
studies  should  be  conducted  to  determine  ways  to  adjust  instruction  to  students'  individual  differences.  In  this  study, 
some  students  claimed  that  visualization  was  easier  or  less  time  consuming  than  other  strategies  while  other  students 
indicated  that  visualization  was  harder  and  more  time  consuming  than  other  strategies.  Perhaps  visualization 
facilitates  learning  for  visual  thinkers  but  does  not  facilitate  learning  for  non-visual  thinkers. 

As  well  as  having  differing  effects  on  different  learners,  visualization  may  enhance  memory  more  for 
certain  kinds  of  content  than  for  others.  Both  concrete  and  abstract  prose  recall  have  been  shown  to  be  enhanced  by 
visualization.  More  studies  are  needed  that  compare  visualization's  effectiveness  across  various  learning  domains 
and  various  subject  matter. 

College  level  educators  should  not  require  students  to  visualize  in  preparation  for  a test  because 
visualization  may  interfere  with  some  students*  efficient  use  of  study  time.  However,  we  have  shown  that 
encouragement  to  visualize  will  not  interfere  with  efficient  study  and  may  provide  students  with  means  to  interact 
with  the  content  they  are  studying.  College  level  educators  should  encourage  learners  to  visualize  using  learner 
techniques  for  representing  interrelationships  that  are  causal,  oppositional,  sequential,  chronological,  comparative, 
categorical,  and  hierarchical. 

Future  research  should  investigate  the  impact  of  the  use  of  computer  graphics  software,  such  as 
AppleWorks^^  and  Microsoft'*''^  Drawing  and  Painting  tools  on  the  effectiveness  of  student-generated 
visualizations.  Cifuentes  and  Hsieh  (2001)  indicate  that  having  students  generate  their  visualization  on  computers 
offer  several  advantages  over  pen  and  paper,  such  as  ease  of  subsequent  revision  and  effortlessness  of  creating 
sophisticated  looking  graphics.  However,  learners  may  be  distracted  by  the  fun  computer  software  and  the  computer 
graphics  tools  during  learning.  More  studies  need  to  be  conducted  in  schools  with  reliable  computers  to  further 
examine  the  effect  of  the  use  of  computer  graphics  tools  on  visualization  learning. 
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Interface  Design  and  Software  Tools  for  Creating  a Multimedia  Program  Measurement 

Instrument 

Maria  Loma  A.  Kunnath 
Instructional  Systems 


Abstract 

This  paper  discussed  how  the  author  developed  the  interface  for  the  Multimedia  Program  Measurement 
Instrument  based  on  proven  design  principles  from  the  behavioral  and  cognitive  sciences  and  using  a 
variety  of  software.  This  is  a program  developed  for  her  dissertation  research  which  was  used  for  the 
initial  pilot  and  field  testing  in  the  Spring  and  Summer  2000  semesters  followed  by  a second  pilot  and  field 
testing  and  Final  Experimentation  in  the  Fall  2000  semester  at  the  University  of  Central  Florida,  C<Sd 
Instructional  Systems.  Her  study  concluded  in  the  Spring  2001  semester.  The  research  study  addressed  the 
effect  of  three  types  of  icon  symbol  formats  viz.,  abstract,  drawing-pictorial,  and  photographic-pictorial  on 
a user's  learning  and  performance.  The  measurement  instrument  consisted  of  a lesson  followed  by  a quiz 
on  the  Advanced  Features  of  a Digital  Video  Camcorder*’.  Divided  into  three  parts,  the  first  part 
discussed  the  design  of  the  interface  and  how  principles  of  design  were  applied  for  ease  of  use.  The  second 
part  of  the  paper  discussed  the  tools  for  creating  the  multimedia  program  and  to  capture  user  *s  action  e.g. 
clicks  and  scores  on  the  test.  The  paper  is  concluded  in  the  third  part  with  a recommendation  for  applying 
similar  approach  to  a variety  of  uses,  whether  standalone  CD-based  or  web-based. 

The  Design  of  the  Interface 

Factors  that  were  given  primary  consideration  in  designing  the  multimedia  (Fleming  and  Levie  (Eds.) 

1993)  measuring  instrument  interface  for  ease  of  use,  mental  effort  (Winn  1993),  attention  (Aspillaga  1996)  and 
clarity  of  presentation  were:  consistency,  predictability,  simplicity  (Haag  and  Snetsinger  1993)  and  information 
density  (Mandel  1997  and  Tullis  1997).  The  program  interface  starts  with  a ‘welcome  login’  window  where  the  user 
enters  personal  information.  The  whole  experiment  is  concisely  and  briefly  explained  using  minimum  text  and 
presented  via  pop  up  boxes  while  the  user  mouse  over  conceptual  words.  This  is  done  to  minimize  a user’s  mental 
effort.  The  user  is  then  familiarized  with  the  lesson  interface  explaining  what  to  anticipate,  what  a user  is  allowed 
and  not  allowed  to  do  during  the  experiment.  During  the  lesson  (trials  and  sections)  the  user  is  guided  by  pop-up 
text  prompts  telling  the  user  to  go  to  next  lesson,  trial  or  when  done  with  the  last  trial  3,  section3^  to  proceed  to 
Quiz. 


Using  a real-world  (Mandel.  1997)  digital  camcorder  metaphor,  the  screen  used  a jpeg  DVCAM  photo 
(Fig.  1)  which  served  three  purposes:  as  background  for  the  content  area  (DVCAM) ; as  context  that  acquainted 
users  on  the  location  of  the  parts  and  control  dials  and,  as  a bounding  box  that  held  all  icon,  text  and  movie 
information.  This  background  appeared  throughout  the  Lesson  only.  For  text,  the  location,  size,  color  and  type  of 
font  (Ross  1988)  stayed  the  same  for  different  types  of  information.  For  example,  the  font  type  (AGaramond),  color 
and  size  and  location  for  the  ‘Step  Number’  button  stayed  the  same  throughout  the  lesson.  The  same  rule  applies  to 
the  heading  and  prompts.  The  importance  of  the  text  information  determined  the  size  of  font,  i.e.  words  such  as 
‘Lesson,  Camera,  Program,  VCR  and  Quiz’  needed  to  be  emphasized  and  had  to  use  (font  size  14)  than  the  heading 
text  (font  size  12).  The  icon  images  were  of  the  same  size  and  location  with  a buttonized,  raised  3-D  look. 
Predictability  was  solved  by  using  a consistent  design  of  having  the  icon  in  the  upper  left  comer,  step  number  on  top 
of  the  icon,  text  meaning  below  the  icon  and  process  meaning  in  motion  video  on  the  right  of  the  icon.  This 
arrangement  made  it  easy  to  locate  information  minimizing  confusion  and  getting  lost  on  the  interface.  Textual 
prompts  appeared  whenever  a section,  trial  or  lesson  was  concluded,  guiding  the  on  what  to  do  and  where  to  click 
next.  The  objects  on  the  interface  such  as  the  icon,  the  text  explanation  and  the  movie  file,  which  appeared  after  the 
icon  was  invoked,  are  grouped^  within  a 3.5”  radius.  I maintained  the  same  radius  for  presenting  single-answer 


^ Experiment  began  Spring  Semester  2000  and  concluded  Spring  2001  at  the  University  of  Central  Florida,  Orlando, 
Florida  on  UCF  Graduate  Student  Volunteers. 

^ The  last  part  of  the  lesson 

^ Close  Proximity  concept  - information  is  processed  faster  and  understood  better  when  located  close  to  one  another 
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multiple -choice  question.  I used  a bigger  4.5**  radius  for  the  several  answer  multiple-choice  questions.  This 
avoided  the  information  seeking  behavior  of  the  eyes  to  move  around  the  interface  when  trying  to  locate 
information  (Tullis,  1997).  This  kind  of  perceptual  grouping’  encouraged  to  make  meaningful  associations  among 
the  presented  information  (Winn  1993).  Overall,  the  interface  had  a simple,  uncluttered  look. 


Location  of  Information  stayed  consistent  in  every  window 
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Buttons  and  active  links  appeared  only  when 
needed  and  prompts  appeared  only  at  appropriate  times 

Fig.  2 Screen  Capture  of  the  Lesson  Interface  (not  to  scale)  Fig.  I Button  ized  Images 

The  Tools 


The  Lesson  was  divided  into  three  sections.  One  section  presented  the  sequential  steps  to  set  the  Canon  ZR 
digital  video  DV  camcorder  DVCAM  to  CAMERA  mode.  Once  in  that  mode,  the  DVCAM  can  be  set  up  in  various 
ways®  which  will  have  an  effect  on  the  recorded  output.  For  illustrative  purposes  and  to  show  the  basic  difference 
between  digital  video  recording  in  contrast  to  standard  analog  video  recording,  the  option  of  setting  the  DVCAM  to 
DIGITAL  EFFECTS  for  the  Camera  Menu  was  chosen.  The  steps  lead  to  finally  setting  the  DVCAM  to  Digital 
Effects  thereafter  showing  the  different  effects  one  can  use  for  a project.  Of  the  seven^  built-in  digital  effects,  only 
four,  Fade-Trigger,  Art.  Sepia  and  Strobe  were  used.  Another  section  presented  the  sequence  of  steps  to  set  the 


’ Perceptual  Grouping  - Buttons,  objects  such  as  a group  of  menu  buttons,  in  this  case,  group  of  information  that 
relates  to  the  icon  being  grouped  together  are  meaningfully  associated  with  one  another. 

® Camera  Menu  options  are:  Shutter  Speed,  Digital  Effects,  Image  Stabilizer,  Digital  Zoom,  16:9,  White  Balance; 
Self-Timer;  Remote  Sensor;  Recording  Mode;  Audio  Mode;  Wind  Screen;  Backlight;  Brightness,  Date  and  Time 
set. 

^ Fade  Trigger-At  Start,  the  scene  gradually  fades  in  from  a black  screen.  When  stopped,  it  gradually  fades  out; 
Wipe-At  Start,  the  picture  begins  as  a thin  vertical  line  in  the  center  of  the  screen  gradually  expanding  sideways  to 
fill  up  the  whole  screen.  At  Stop,  the  image  is  wiped  from  both  sides  of  the  screen;  Scroll-  At  Start  the  picture 
appears  from  the  right  hand  side  of  screen  expanding  sideways  to  fill  up  the  whole  screen.  At  Stop,  the  image  is 
wiped  off  Art-  Adds  a paint-like  solarization  effect  to  the  image.  Black  and  White-Records  the  image  in  black  and 
White;  Sepia -re  cords  a monochrome  image  with  a sepia  tint:  Strobe-On-screen  actions  become  a series  of  still 
images  i.e.  slow  motion  effect. 
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DVCAM  to  PROGRAM  Mode.  In  this  mode,  six  options^®  are  available  to  suit  the  recording  environment.  For 
illustrative  purposes  the  LOW  LIGHT  option  was  selected.  When  set  to  Low  Light,  the  DVCAM  can  record  in 
dimly -lit  places.  The  other  section  presented  the  sequence  of  steps  for  setting  the  DVCAM  to  VCR  mode.  When  in 
the  VCR  mode,  a user  can  view  the  recorded  product. 

Each  step  is  represented  in  three  different  icon  design.  For  that  reason,  MS  Image  Composer  and  JASC 
PaintshopPro*  ^ were  used.  Image  Composer  was  useful  for  minimum  photographic  image  enhancements  such  as 
cropping,  erasing,  sharpening,  blurring  and  texturizing.  PainshopPro,  another  image  editing  software’s.  Drawing 
Tool,  not  only  allowed  freehand  drawing  of  shapes,  geometric  shapes,  vector  arrows  etc.  but  also  allowed  enhancing 
of  graphic  objects.  Through  its  Image  and  Effects  Menu,  I was  able  to  fill  shapes  e.g.  arrows  with  colors,  rotate  the  f 
vectors  and  enhanced  their  brightness  and  illumination.  The  last  feature  was  particularly  useful  in  enhancing  the 
photographic-pictorial  symbols  resulting  in  a sharper,  clearer  image.  When  all  the  graphical  digital  assets  were 
completed,  each  one  was  given  a uniform  dimension  and  applied  a Buttonize  effect  giving  it  a raised  clickable  3-D 
look  (Fig  2).  Video  files  were  created  by  first  recording  the  different  steps  in  analog  format.  Using  a software 
converter  called  DAZZLE,  these  analog  files  were  converted  into  MPEG  files.  Once  captured  and  saved  onto  the 
PC’s  hard  drive,  these  MPEG  files  were  enhanced,  edited  and  cut.  When  set  in  Snap  Capture  and  View  mode,  these 
same  MPEG  files  were  used  to  extract  JPEG  images  as  well.  To  create  the  entire  multimedia  program,  Macromedia 
Authorware  Attain  5 was  used.  Authorware  is  an  object-oriented,  multi-platform  authoring  tool  which  recognizes 
various  file  formats  for  animation,  audio,  video,  text  and  graphics.  Knowledge  Objects  are  mini  programs  that  can 
be  added  to  the  main  authorware  application.  In  Authorware  there  is  a Quiz  Knowledge  Object,  an  Application 
Knowledge  Object,  a Movie  Controller  Knowledge  Object  etc.  Its  Knowledge  Objects  Application  shell  was  used 
to  develop  the  LOGIN  portion  which  is  where  the  user  inputted  personal  information  such  as  name  (first,  last  and 
middle)  and  subject  number.  The  rest  of  the  multimedia  measuring  instrument  program  was  completed  using  the 
flowchart,  icon-based  authoring  approach. 

Once  operational  Camtasia,  a screen  recording  software  was  used  to  track  user’s  actions  . Time  and  scores 
on  the  quiz  were  captured  on  the  hard  drive.  These  MPEG  files  dynamically  recorded  all  user  actions  in  real-time 
which  can  be  replayed  and  reviewed.  User’s  behavior  on  the  program  such  as  movement  of  the  cursor  appearing  as 
an  arrow  or  a hand  can  be  observed  and  counted  during  replay. 

Conclusion 


The  behavioral  and  cognitive  design  principles  that  guided  the  creation  of  the  multimedia 
measuring  instrument  can  be  used  for  similar  learning,  performance  and  evaluation  exercise  for  different  content 
areas  in  interactive  simulations,  CD  or  web-based  training  and  other  technology -related  training.  The  tools  I used 
for  creating  the  instrument  are  but  a few  of  the  many  applications  available.  It  is  now  common  practice  to  download 
trial  versions  of  software*^  for  30-45  functional  days.  After  the  elapsed  period,  these  downloaded  applications  will 
stop  working  and  will  prompt  the  evaluators  to  purchase  the  product.  Much  of  my  instructional  technology 
application  try  outs  are  done  this  way. 
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Abstract 

Electronic  portfolios  have  great  potential  for  promoting  reflective  independent  learning  at  all  grade  levels, 
from  elementary  school  to  graduate  teacher  training.  Developing  an  electronic  portfolio,  however,  involves 
mastering  both  the  complexities  of  designing  and  developing  portfolios  and  the  complexities  of  designing 
and  developing  multimedia  and/or  World  Wide  Web  materials.  This  paper  presents  strategies  for 
efficiently,  effectively,  and  economically  carrying  out  the  multiple  processes  involved  in  electronic 
portfolios,  emphasizing  strategies  that  work  in  real-world  settings. 

Fulfilling  the  Promise  of  Electronic  Portfolios 

Assessment  is  the  “glue”  that  holds  together  planning  and  instruction.  A growing  trend  in  educational 
assessment  is  the  attempt  to  develop  and  use  more  authentic,  performance-based  assessment  procedures  and  tools. 
Authentic  assessments  provide  students  the  opportunity  to  demonstrate,  rather  than  tell  or  be  questioned  about,  what 
they  know.  The  use  of  portfolios  to  assess  student  learning  is  an  approach  to  authentic  assessment  that  is  increasing 
in  popularity  at  all  levels  of  education,  from  elementary  school  (e.g.,  Fogarty,  1996)  through  graduate  teacher 
training  (e.g.,  Krause,  1996). 

The  Portfolio  Process 

Portfolios  are  more  than  a collection  of  artifacts;  they  are  “a  systematic  and  organized  collection  of 
evidence  used  by  the  teacher  and  the  student  to  monitor  growth  of  the  student’s  knowledge,  skills,  and  attitudes.” 
(Vavrus,  1990,  p.  48).  Portfolios  enable  students  to  become  more  aware  of  themselves  as  learners  and  to  take 
ownership  in  the  processes  and  outcomes  of  their  own  learning. 

Danielson  and  Abrutyn  (1997)  identified  four  basic  steps  in  the  portfolio  development  process.  Portfolio 
development  begins  with  the  collection  process.  Students  assemble  work  samples  and  other  materials  that 
demonstrate  the  processes  and  products  of  their  learning,  based  on  the  purpose  of  the  portfolio.  The  next  step  is 
selection,  in  which  the  student  selects  examples  of  his  or  her  best  work  (for  a product  portfolio)  or  work  that 
exemplifies  specific  criteria  (for  a process  portfolio).  Students  should  select  sufficient  materials  to  demonstrate  the 
full  range  of  their  learning. 

A third  (and  key)  step  in  the  portfolio  development  process  is  reflection.  Students  analyze  each  item  they 
include  in  their  portfolio  to  identify  ways  in  which  developing  or  collecting  the  item  contributed  to  their  learning. 
The  reflective  process  is  a tool  for  the  analysis  of  learning  and  self-growth.  In  reflecting  on  each  item  in  their 
portfolio  students  go  beyond  summaries  to  analyze  and  synthesize  their  knowledge,  skills,  and  attitudes  as  they 
develop.  Students  who  are  new  to  portfolios  will  need  considerable  guidance  and  practice  in  order  to  develop  the 
ability  to  engage  in  insightful,  higher-order  reflective  thinking.  Reflections  can  be  categorized  as:  {\)  reflection-on- 
action  (looking  back  and  reviewing  what  was  done);  {2)  reflection-in-action  (reflecting  while  in  the  act  of  carrying 
out  a task);  and  (3)  reflection-for-action  (reviewing  what  was  done  and  identifying  future  actions  and 
improvements)  (Killian  and  Todnem,  1991). 

The  final  step  in  the  portfolio  process  {^projection,  in  which  students  look  ahead  to  where  their  learning 
will  take  them.  They  set  goals  for  the  future,  and  examine  their  past  work  in  order  to  identify  patterns,  strengths,  and 
weaknesses. 

The  Potential  Role  of  Technology  in  the  Portfolio  Process 

Traditional  portfolios  are  print-based.  However,  as  new  and  more  powerful  technologies  become  a regular 
part  of  learning  and  instruction,  more  and  more  teachers  and  teacher  educators  are  seeking  ways  to  use  various 
technologies  to  create  electronic  portfolios  (Barron,  (2000);  Venezky  (1998)).  Electronic  portfolios  have  several 
potential  advantages  over  traditional  portfolios,  and  they  validate  the  use  of  technology  as  an  essential  part  of 
learning  and  instruction  for  both  students  and  teachers:  “Releasing  the  power  of  technology  in  the  classroom  via 
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portfolios  enables  the  teacher  to  reach  more  students  effectively  and  efficiently,  while  engaging  them  in  a more 
realistically  meaningful  process  of  learning.”  (Cole,  Ryan,  Kick,  and  Mathies,  2000,  p.  50). 

Electronic  Portfolios:  Challenges  and  Opportunities 

The  increased  effectiveness  and  efficiency  of  electronic  portfolios  comes  at  a price.  The  electronic  portfolio 
process  includes  all  of  the  complexities  of  the  traditional  portfolio  process  (collection,  selection,  reflection,  and 
projection).  Added  to  these  are  the  additional  complexities  associated  with  designing,  developing,  and  managing 
electronic  (digital)  information.  Ivers  and  Barron  (1998)  have  identified  five  stages  in  the  development  of 
multimedia  materials:  (1)  Decide/assess  (determining  the  goals,  audience,  and  the  needs  of  the  audience);  (2) 
Design/plan  (identify  content  and  sequence  of  presentation);  (3)  Develop  (collect  and/or  generate  components;  (4) 
Implement  (present  the  multimedia  materials);  and  (5)  Evaluate  (measure  and  Judge  the  effectiveness  of  the 
multimedia  materials).  If  the  multimedia  materials  are  web-based,  an  additional  layer  of  complexity  is  added  related 
to  publishing  the  materials  on  the  World  Wide  Web  (dealing  with  network  protocols,  moving  and  managing 
information  among  several  different  computers,  etc.).  Both  the  portfolio  development  process  and  the  technology 
design/development  process  must  be  carefully  coordinated  in  order  to  achieve  the  stated  goals  of  the  portfolio 
process. 

Electronic  portfolios  range  in  complexity  from  a floppy  disk  of  word  processor  files  to  full-featured  World 
Wide  Web  sites  with  extensive  multimedia  information  and  interactive  capabilities.  There  is  a rather  steep  learning 
curve  for  both  teachers  and  students  in  becoming  adept  at  developing,  using,  and  managing  electronic  portfolios. 

Not  all  students  and  teachers  will  have  the  knowledge  and  skills  required  to  use  every  type  of  technology  they  might 
want  to  incorporate  into  their  electronic  portfolios. 

Electronic  Portfolios:  Strategies  for  Effective  and  Efficient  Use 

Should  developing  multimedia  and  World  Wide  Web  design  and  development  expertise  be  considered  part 
of  the  portfolio  development  process  or  as  a separate  task?  Is  there  time  in  the  school  day  to  learn  new  technological 
tools  while  at  the  same  time  learning  about  and  implementing  the  complex  portfolio  process?  The  strategies 
presented  in  the  following  sections  demonstrate  the  interrelationship  between  the  dual  processes  of  portfolio 
development  and  digital  media  development,  and  illustrate  potential  ways  to  make  the  development  of  electronic 
portfolios  a more  efficient,  effective,  and  (it  is  hoped)  cost-effective  process. 

Using  Technology  to  Support  and  Enhance  the  Collection  Process 

Word  processor,  spreadsheet,  and/or  database  software  can  be  used  as  appropriate  both  to  generate  and  to 
collect  artifacts  for  the  portfolio.  It  is  likely  that  many  of  the  artifacts  to  be  incorporated  into  the  portfolio  already 
exist  as  digital  files;  collecting  them  as  digital  information  is  a logical  alternative  to  producing  print  versions  of  the 
information  and  trying  to  organize  the  large  amount  of  paper  typically  produced  for  a portfolio. 

Charts,  drawings,  photos,  and  other  non-print  documents  can  be  scanned  and  converted  to  appropriate 
digital  form.  Digital  cameras  (still  cameras  and/or  camcorders)  can  be  used  to  document  events  and  processes  in  a 
form  that  is  much  more  difficult  (if  not  impossible)  using  traditional  portfolio  formats.  The  use  of  technologies, 
therefore,  enhances  not  only  the  depth  but  also  the  breadth  of  the  portfolio  contents  while  making  it  easier  to 
produce  and  collect  relevant  information  for  the  portfolio. 

Using  Technology  to  Support  and  Enhance  the  Reflective  Process 

The  reflective  process  is  very  complex.  Technology  can  support  the  analysis  of  learning  and  self-growth 
involved  in  the  reflective  process  is  several  ways.  Even  a relatively  low-level  technology  such  as  a word  processor 
can  make  recording  and  organizing  reflective  statements  more  efficient.  Date  and  update  information  can  be 
included  in  the  name  of  the  word  processor  files,  so  that  the  files  will  be  arranged  chronologically  when 
automatically  sorted  by  name. 

Initially  reflective  statements  can  be  added  to  a word  processor  file  as  the  reflections  occur  to  the  student. 
After  reviewing  all  of  the  reflective  statements,  the  student  can  use  the  outline  feature  of  the  word  processor  to 
organize  and  sequent  the  collection  of  reflective  statements.  The  various  reflective  statements  can  be  organized  by 
theme  and  sub-theme  using  different  levels  of  headings.  Students  who  have  a higher  level  of  proficiency  in 
technology  (for  example,  students  proficient  in  HTML  or  web  authoring  tools)  can  organize  their  reflective 
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statements  as  an  interrelated  set  of  web  pages.  Using  web  technology  make  possible  the  organization  of  reflective 
statements  in  a multi-linked,  hypertext  format  to  express  multiple  interrelationships  among  reflective  statements. 

Using  Technology  to  Support  and  Enhance  Collaborative  Activities 

Portfolios  have  a dual  purpose.  They  provide  a vehicle  for  self-reflection  to  guide  the  student’s  future 
learning.  In  addition,  they  provide  a vehicle  to  document  and  to  communicate  of  the  student’s  accomplishments. 
Technology,  especially  network  technologies,  can  enhance  the  collaborative  aspects  of  portfolio  development  in 
several  ways.  Word  processor  documents  and/or  web  pages  that  initially  were  used  to  collect  all  possible  reflective 
statements  can  be  edited  as  part  of  the  selection  process  of  portfolio  development.  Electronic  mail  can  be  used  to 
communicate  with  co-leamers  and  the  teacher,  particularly  when  the  communications  are  the  one-to-one  type.  Co- 
leamers  can  use  the  “track  changes”  feature  included  in  most  word  processors  to  embed  peer  review  comments  in 
the  student’s  original  file  and  return  the  annotated  file  to  the  original  author. 

Presentation  technologies  such  as  PowerPoint  can  be  used  to  develop  a summary  document  that  presents 
the  major  components,  analyses,  and  reflections  contained  in  the  full  portfolio.  Online  discussion  groups  facilitate 
distribution  of  information  from  the  teacher  and  the  sharing  of  information  and  resources  among  students.  Students 
can  publish  drafts  of  their  portfolios  on  a web  site  (either  as  part  of  a WebCT  or  Blackboard  course  site  or  on  a site 
developed  specifically  to  support  the  portfolio  process)  to  facilitate  peer  review  of  their  portfolios. 

Using  Technology  to  Support  Portfolio  Assessment  and  Presentation 

The  same  technologies  that  support  peer  review  and  communication  can  be  used  by  the  teacher  to  provide 
individualized  feedback  during  the  portfolio  development  process  and  for  assessment  of  the  final  version  of  the 
portfolio.  Teacher  comments  and  suggestions  can  be  directed  confidentially  to  an  individual  student  via  e-mail  or 
through  a discussion  group,  or  the  teacher  can  use  a discussion  group  to  broadcast  comments,  clarifications,  and 
suggestions  that  arose  in  the  context  of  interacting  with  an  individual  student  to  the  entire  group  of  students. 
Comments  can  be  sent  to  the  student  via  e-mail.  Teachers  can  use  the  “track  changes”  feature  included  in  most  word 
processors  to  return  an  annotated  version  of  documents  with  the  teacher’s  comments  embedded  at  appropriate  places 
in  the  student’s  original  documents. 

The  use  of  appropriate  technologies  also  facilitates  the  teacher’s  assessment  of  the  final  versions  of  the 
portfolios.  Students  can  submit  the  final  versions  of  their  portfolios  as  a folder  of  files  (in  Zip  form  if  the  files  are 
large)  that  can  be  sent  to  the  teacher  as  an  attachment  to  an  email  message.  If  all  students  are  proficient  in 
developing  and  publishing  web  materials,  students  can  publish  their  portfolios  on  a designated  web  site.  The 
instructor  can  then  develop  (or  commission)  an  “overview/directory”  page  with  links  to  each  student’s  portfolio. 

If  students  submit  their  portfolios  in  digital  form  (e.g.,  as  a folder  containing  all  of  the  files  that  comprise 
the  portfolio  contents  or  as  a web  page),  all  of  the  portfolios  to  be  evaluated  can  be  stored  in  one  place,  either  on  an 
individual  computer  or  an  a web  site.  This  will  facilitate  both  individual  assessment  and  comparison  of  various 
students’  portfolios. 

These  same  technologies  (PowerPoint,  web  pages,  etc.)  that  students  used  to  develop  working  drafts  of 
their  portfolios  can  be  used  to  present  and  distribute  the  final  versions  of  their  portfolios.  These  technologies  make  it 
possible  for  individuals  (a  student,  a parent,  a teacher,  a prospective  employer  of  a pre -service  teacher)  to  view  the 
complete  portfolio.  It  is  also  much  easier  for  a teacher  to  preserve  samples  of  exemplary  portfolios  to  be  used  as 
guidelines  for  students  in  future  classes. 

Students  who  have  the  requisite  technology  skills  can  transfer  their  complete  portfolio  to  a CD-ROM  for 
archiving  and  distribution.  Certainly  being  able  to  give  a prospective  employer  a CD-ROM  provides  definite 
advantages  over  giving  him  or  her  a large  bulky  binder. 

Using  Technology  to  Support  the  Portfolio  Process:  Questions  and  Cautions 

The  major  question  to  be  answered  in  deciding  whether  and/or  howto  implement  electronic  portfolios  is: 
What  is  the  appropriate  use  of  technology  in  the  portfolio  process?  In  answering  this  question,  one  must  address  (a) 
the  purpose  of  portfolios  in  a particular  learning  situation;  (b)  the  skills  and  knowledge  of  the  students,  both  in  the 
area  of  portfolio  development  and  in  the  use  of  various  technologies;  and  (c)  the  audience  for  the  portfolios 
(including  both  immediate  and  potential  long-term  audiences). 
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Several  specific  questions  must  be  addressed  in  planning  for  the  use  of  electronic  portfolios  and  in  deciding 
the  extent  to  which  electronic  portfolios  can  be  used  in  a “real-world”  situation  with  a given  group  of  students. 

These  questions  include: 

What  level  of  technology  knowledge  and  skills  do  students  begin  with? 

How  will  students  develop  additional  technology  knowledge  and  skills  if  they  are  needed? 

What  resources  (equipment,  software,  technical  support,  materials,  time)  are  available  to  support  the 
development  of  electronic  portfolios? 

In  answering  all  of  the  above  questions,  and  in  every  phase  of  the  design  and  development  of  a technology- 
based/technology -facilitated  portfolio  program,  the  major  principle  to  keep  in  mind  is  that  learning  and  the  needs  of 
learners  should  determine  which  technologies  are  used  and  how  they  are  used.  Technology  should  be  an  enabler,  not 
a driver,  of  the  process.  It  is  hoped  that  this  paper  will  provide  guidance  and  support  to  educators  already  engaged  in 
or  considering  the  use  of  electronic  portfolios. 
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Abstract 

Next  to  playing,  matching  instructional  features  with  children 's  characteristics  may  stimulate  children  *s 
development  in  early  education.  Instructional  lines  of  ordered  playing  and  learning  appliances,  inclusion 
of  diagnostic  and  achievement  indicators,  flexible  grouping  of  students,  screening  of  initial  student 
characteristics,  and  supportive  software,  are  expected  to  optimize  early  education  for  students  and 
teachers.  In  two  Dutch  kindergarten  schools,  instructional  changes  were  realized  in  a use-oriented  method 
by  co-development  with  teachers  and  management.  Also,  a software  prototype  designed  to  manage  and 
optimize  early  education  was  developed  and  tested.  Information  is  given  about  the  digital  instructional 
management  prototype,  development  and  implementation  experiences  in  early  education,  and  first  effects  of 
using  this  software  in  practice.  Further  possibilities  to  optimize  education  in  kindergarten  and  other 
educational  types  are  discussed. 

Learning  and  instructional  theories  focus  on  individual  or  small-group  learning,  although,  in  school 
practice,  learning  usually  occurs  in  the  whole  group  or  class  (Collier,  1994).  Within  a whole  group,  class  level 
learning  expectancies,  complexities,  and  norms  may  then  interfere  with  small-group  and  individual  learning 
possibilities  and  potentials.  This  is  true  in  part  icular  for  learners  who  seem  to  be  at  risk  in  some  respect,  e.g.  a 
learner  functioning  relatively  low  compared  to  most  other  learners  (Walker,  Kavanagh,  Stiller,  Golly,  Severson,  & 
Feil,  1998),  or  relatively  high  as  specified  by  e.g.  King,  O'Shea,  Joy  Patyk,  Popp,  Runions,  Shearer,  and  Hendren 
(1985). 

To  support  development  and  learning  processes  and  effects,  each  student’s  characteristics  should  be 
matched  with  actual  instructional  and  didactic  features  (Van  Merrienboer,  1997).  A learner  ‘deviating’  from  the 
other  learners  may  therefore  profit  most  from  instructional  management  designs  which  optimize  teaching  and 
learning  for  all  students  actually  present  in  class.  However,  giving  more  attention  to  more  details  of  learning 
processes  requires  much,  and  often  too  much,  of  the  teacher.  As  differences  in  development  and  learning  between 
children  can  already  be  rather  huge  at  the  start  of  kindergarten,  providing  adequate  instruction  and  prevention  of 
learning  de-motivation  and  problems  may  become  too  complicated  for  the  teacher  (Jones,  Gullo,  Burton-Maxwell, 

& Stoiber,  1998).  In  this  respect.  Information  and  Communication  Technology  (ICT)  can  lend  a hand  (Chang,  2001; 
Sinko  & Lehtinen,  1999).  ICT  consists  of  different  kinds  of  electronic  hardware  and  software  which,  in 
combination,  may  support  the  preparation,  execution,  and  evaluation  of  networked  teaching  and  learning  processes 
and  effects  (Lally,  2000). 

In  research  to  explore  the  role  of  ICT  in  optimizing  instructional  and  didactic  management,  qualitative 
research  to  model  didactic  and  learning  processes  was  carried  out  in  ten  Dutch  secondary  schools  varying  in 
background  characteristics  (Mooij  & Smeets,  2001).  Five  successive  phases  and  respective  models  of  ICT 
implementation  were  found: 

1 . Incidental  and  isolated  use  of  ICT  by  one  or  more  teachers 

2.  Increasing  awareness  of  ICT  relevance  for  the  school,  at  all  levels 

3.  Emphasis  on  ICT  coordination  and  hardware  within  school 

4.  Emphasis  on  instructional  and  didactic  innovation  and  ICT  support 

5.  Use  of  ICT -integrated  teaching  and  learning,  independent  of  time  and  place 

After  acquisition  of  hardware  and  software  in  phase  3,  in  phase  4 ICT  is  designed  to  support  learning 
processes  in  more  flexible  and  optimizing  ways.  In  phase  5,  education  is  being  restructured  from  the  perspectives  of 
the  learners.  This  process  requires  transformation  of  teaching  and  learning  processes  by  changing  instructional 
features  and  didactic  management,  in  close  relationship  to  ICT  conditions  supporting  both  individualized  and  small - 
group  learning  in  and  outside  school. 

Optimizing  education  since  the  start  of  kindergarten  may  be  the  most  promising  effort  to  prevent  de- 
motivation  and  learning  problems  (cf  Walker  et  al.,  1998).  Therefore,  the  question  for  research  is  how  model  5 
could  be  designed  and  implemented  in  early  education  to  realize  the  advantages  potentially  related  to  this  model,  in 
particular  for  students  at  risk.  To  answer  this  question,  I will  first  focus  on  theoretically  relevant  instructional  and 
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management  conditions  to  support  students  since  the  start  of  kindergarten  (cf  Van  den  Akker,  1999).  Next,  in  co- 
development with  teachers  and  management,  some  required  educational  changes  are  realized  in  kindergarten 
practice.  Then,  relevant  software  is  implemented  and  the  user  effects  of  this  first  prototype  are  revealed  and 
evaluated. 

Theory 


Bergqvist  and  Saljo  (1998)  report  about  grades  1-3  of  four  elementary  schools  in  Sweden.  The  schools  use 
an  individualized  curriculum  in  an  age-integrated  classroom  and  the  students  are  aged  seven  to  nine.  The  researchers 
concentrated  on  student  and  teacher  cooperation  in  discussing  the  student’s  weekly  planning  and  working  or,  in 
other  terms,  about  learning  to  self-regulate  the  schoolwork.  Their  observations  reveal  that  many  responsibilities  are 
conveyed  from  the  teacher  to  the  student  because  social,  pedagogical,  and  learning  roles  are  intricately  related  to  the 
instructional,  didactic,  and  school-wide  curriculum  organization  of  both  teaching  and  learning.  Moreover,  a student 
is  functioning  better  if  he  or  she  is  able  to  choose  playing  or  instructional  activities  according  to  or  slightly  above  his 
or  her  actual  level  of  competency  within  a certain  field.  Comparable  results  are  found  by  the  American  research  of 
Jewett,  Tertell,  King-Taylor,  Parker,  Tertell,  & Orr  (1998)  who  concentrate  on  pedagogical  and  curricular  aspects 
teachers  have  to  realize  to  help  children  with  special  needs  in  kindergarten.  If  this  matching  is  not  realized, 
motivation  and  achievement  problems  may  turn  up  for  students  functioning  at  relatively  lower  or  higher  levels  of 
competency  than  their  peers. 

Theoretically,  curricular  features  should  be  designed  in  such  a way  that  concrete  instructional  and  didactic 
processes  are  supporting  each  learner’s  activities  at  any  time,  in  a positive  social  context.  Learning  could  be 
designed  also  for  a small  group  of  students  helping  each  other,  or  collaborating  with  one  other  (Jones,  Rasmussen,  & 
Moffitt,  1997).  In  this  situation,  the  teacher’s  attention  can  concentrate  more  on  students  who  most  need  his  or  her 
attention.  Such  an  instructional  design  would  require  flexible  instructional  lines  with  clear  diagnostics  next  to 
situations  of  free  playing,  including  flexible  grouping  and  organization  throughout  the  educational  career.  On  the 
student’s  side,  it  should  be  clear  which  competencies,  and  which  initial  levels  of  competencies,  should  be  stimulated 
in  which  ways,  to  optimize  developmental  progress  since  the  start  in  kindergarten. 

Free  Playing  And  Instructional  Lines 

In  early  education,  free  playing  activities  are  usually  based  on  children's  own  initiatives  and  choices 
(Pellegrini  & Boyd,  1993).  The  concept  of ‘instructional  line’  can  be  used  to  refer  to  a specific  set  of  learning 
activities  ordered  according  to  instructional  difficulty  level  or  social  didactic  aspects,  e.g.  motor  behavior,  social- 
emotional  development,  projects,  language  and  literacy,  (preliminary)  arithmetic,  (preliminary)  reading,  and 
(preliminary)  writing.  The  line  concept  denotes  a hierarchical  arrangement  of  curricular  concepts  and  sub -concepts 
corresponding  with  specific  instructional  or  didactic  play  materials,  representing  specific  learning  or  play  activities. 
For  example,  sensorimotor  development  for  four-  to  six-year-olds  generally  starts  with  global  movement  with  the 
whole  body,  followed  by  movement  with  the  arms  and  hands,  and  then  by  paying  attention  to  writing  conditions, 
e.g.  direction  in  moving,  training  of  regularity  in  movement  with  hands  and  fingers,  and  motor  exercises  evolving 
into  preliminary  writing. 

Diagnostic  And  Progress  Indicators 

Within  instructional  lines,  reliable  and  valid  indicators  need  to  be  integrated  to  diagnose  and  evaluate 
learning  processes  and  their  outcomes  on  every  student,  from  the  start  in  kindergarten  onwards.  Monitoring  is 
important,  in  particular  to  realize  a timely  promotion  of  the  development  of  children  at  risk.  Also,  a standardized 
diagnostic  or  age-normed  achievement  test  should  be  included.  Each  student's  progress  can  then  be  evaluated 
continuously  by  individual,  social,  and  age-normed  diagnostic  and  achievement  tests  in  the  architecture  of  lines  (cf. 
Byrne,  1998;  Wegerif,  Mercer,  & Dawes,  1998). 

Flexible  Grouping  And  Organization 

Making  the  organizational  grouping  of  students  more  flexible  according  to  learners’  characteristics  and 
instructional  procedures  seems  another  precondition  to  promote  school  careers  of  students  at  risk  in  particular 
(Bennathan  & Coxall,  1998;  Cooper  & Ideus,  1998).  Flexible  grouping  in  small  groups  can  for  example  be  designed 
on  the  basis  of  students’  competency  levels,  learning  style,  or  specific  didactic  preferences  or  requirements  in  case 
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of  certain  handicaps.  Moreover,  flexibilizing  of  students’  grouping  can  also  stimulate  cooperation  between 
professionals  in  and  outside  school,  to  support  children  better  than  in  their  earlier  development  (Mangione  & Speth, 
1998). 

Screening  Of  Children’s  Entry  Characteristics 

Close  observation  and  analysis  of  interactions  between  instructional  features  and  students’  learning 
characteristics  reveal  that  development  and  learning  problems  may  start  early  in  kindergarten  (Skinner,  Bryant, 
Coffman,  & Campbell,  1998).  Given  the  differences  in  development  between  children  already  at  the  very  start  of 
kindergarten,  it  is  important  to  present  adequate  and  diverse  kinds  of  playing  and  learning  materials.  This  is  most 
relevant  for  children  deviating  most  from  the  other  children  in  class.  To  get  relevant  information  as  soon  as  possible, 
it  is  helpful  to  screen  a child’s  starting  characteristics  by  parents  at  intake,  and  by  the  class  teacher  after  the  child’s 
first  month  in  school.  Communication  about  differences  between  the  perceptions  of  parents  and  kindergarten,  and 
taking  adequate  curricular  action  if  indicated,  can  prevent  motivational,  social,  emotional,  or  cognitive  problems  of 
vulnerable  students  in  particular.  The  teacher  or  another  professional  can  additionally  diagnose  or  assess 
characteristics,  and  assign  specific  playing  and  instructional  activities  to  a child  or  a small  group  of  children,  if 
possible  in  cooperation  with  parents.  Guidelines  to  this  optimization  process  are  presented  and  discussed  by  Mooij, 
Terwel,  and  Huber  (2000). 

Software  Features 

Initially,  it  is  not  clear  how  software  could  be  designed  to  support  the  optimization  possibilities  sketched 
above.  For  this  reason,  it  was  decided  to  first  start  the  developments  sketched  above  in  kindergarten  practice.  In 
doing  this,  it  could  be  checked  in  how  far  design  features  of  the  software  could  be  based  upon  educational  features, 
and  how  their  specifics  had  to  be  to  help  teachers.  At  the  same  time,  implementation  and  first  effects  of  the  software 
could  be  observed. 

Method 


The  theoretically  desirable  educational  features  were  not  known  to  exist  in  practice.  Therefore,  a 
developmental  project  was  planned  in  two  regular  Dutch  kindergartens  for  children  aged  four  to  six.  The  usual 
planning  system  in  these  kindergartens  consisted  of  a planning  board  on  the  wall,  with  a differently  colored  column 
for  each  day  of  the  week.  Small  groups  of  students,  corresponding  with  certain  table  groups,  were  indicated  by 
different  logos  and  colors.  Activities  to  be  done  by  a small  group  were  assigned  by  placing  the  tags  of  these  students 
on  a logo  representing  a certain  kind  of  activity,  on  a certain  day  of  the  week,  on  the  planning  board. 

In  the  years  1997-2000,  teachers,  management,  and  the  researcher  collaborated  to  develop  instructional 
lines  based  on  the  regular  playing  and  learning  appliances  in  the  kindergarten  classes.  Recent  methodology  supports 
a strategy  in  which  users,  for  example  teachers  and  school  staff,  collaborate  with  researchers  and  other  specialists  to 
secure  validity  of  innovation  processes  (cf  Blumenfeld,  Fishman,  Krajcik,  Marx,  & Soloway,  2000;  Clark  & Estes, 
1999;  Kensing,  Simonsen,  & Bodker,  1998).  Wilson  (1999)  expects  that  ‘use-oriented’  strategies  ‘(...)  increase  the 
likelihood  of  successful  implementation  because  they  take  the  end  use  into  account  at  the  beginning  design  stages’ 

(p.  13).  This  development  co-occurred  with  flexibilization  of  grouping  of  students,  and  the  screening  of  students* 
entry  characteristics  by  parents  and  the  class  teacher. 

In  the  first  period,  attention  was  focused  on  rearranging  and  registering  the  appliances  to  clarify  the 
instructional  and  didactic  aspects  of  the  playing  and  curriculum  system.  Next,  the  use  of  the  registered  appliances 
either  by  the  teacher  or  by  self-management  of  the  students  was  made  concrete.  Parallel  to  this,  and  in  the  same 
collaborative  way  (cf.  Ely,  1999),  a first  prototype  of  a computer  program  was  designed  and  developed.  The  design 
concentrated  on  extending  optimization  features  and  possibilities  of  the  planning  board,  to  better  assist  teachers  and 
students  (Crook,  1998).  Essential  software  functions  of  this  ‘digital  instructional  management’  were  related  to 
registering  and  manipulating  information  about  instructional  lines,  students  and  teachers,  and  promoting  students’ 
instructional  self-management.  The  information  from  educational  practice  was  collected  into  digital  databases  within 
the  framework  of  a standalone  computer. 

Results 

Software  Features 
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Information  on  Instructional  Lines.  A first  main  feature  of  the  prototype  is  the  creation,  change,  or  removal 
of  an  instructional  line.  Lines  developed  refer  to,  for  example,  motor  behavior,  social-emotional  development, 
projects,  language,  (preliminary)  arithmetic,  (preliminary)  reading,  and  (preliminary)  writing.  Each  instructional  line 
in  kindergarten  class  and,  correspondingly,  in  the  software,  is  characterized  by  a specific  logo,  a specific  color,  and  a 
corresponding  name  or  text.  Activities  or  tasks  within  each  line  are  visually  represented  by  a photograph  of  the 
object  as  present  in  class  because  four-year-olds  must  be  able  to  work  with  the  program,  A screenshot  of  this  feature 
of  the  prototype  is  given  in  Figure  1 , 

A second  main  feature  concerns  the  input,  change,  or  removal  of  activities  or  tasks  within  specific  levels  of 
an  instructional  line.  Within  a line,  activities  or  tasks  are  ordered  by  difficulty  level.  To  stimulate  students 
adequately,  variants  of  lines  referring  to  different  developmental  levels  were  constructed  for  e.g.  ‘normaT  students 
developing  in  a more  or  less  regular  way,  students  requiring  special  or  remedial  activities,  and  highly  able  students 
who  are  advanced  on  the  topic  of  the  line.  Moreover,  an  activity  can  be  tagged  with  an  indicator  meaning  that  the 
student  has  to  go  to  the  teacher  in  order  to  continue.  For  example,  an  indicator  may  mean  that  a student’s  initial  level 
of  language  competency  has  to  be  screened,  as  a basis  for  further  support  and  placements.  In  the  same  vein,  a 
standardized  diagnostic  or  achievement  test  can  be  included,  in  particular  for  measurement  with  a conspicuous 
student  or  a student  at  risk.  Another  potential  use  is  the  formation  of  a small  group  of  students  by  the  teacher,  to  do 
the  tagged  activity.  A screenshot  of  ordering  an  activity  by  using  a photograph,  name,  and  description  into  an 
instructional  language  line  at  level  6 is  shown  in  Figure  2. 

Figure  L Symbols  (Logo,  Color,  Name  Of  Line)  Representing  Four  Instructional  Lines 
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Figure  2.  Ordering  Of  Activity  (Photograph,  Name,  Description)  Into  Instructional  Line  And  Level 
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A third  feature  of  the  software  is  the  possibility  to  get  an  overview  of  the  content  of  an  instructional  line,  or 
a set  of  variants  of  an  instructional  line,  at  a specific  level  of  difficulty. 

Information  about  Students.  Another  main  feature  of  the  software  has  to  do  with  registering  administrative 
information  about  a student:  adding  a new  student,  removing  a student,  or  changing  existing  information  about  a 
student.  For  example,  the  teacher  can  register  students  by  integrating  their  photographs  in  the  database.  The 
photographs  of  all  children  in  one  class  can  be  shown  in  one  screen.  Furthermore,  the  registering  of  each  student’s 
initial  characteristics  is  carried  out  in  conformance  with  a psychometrically  controlled  procedure  based  on 
quantitative  longitudinal  research  with  966  four-year-olds  (Mooij,  2000).  An  overview  of  initial  characteristics  of  a 
student,  and  a comparison  between  the  information  from  parents  and  teacher,  can  be  produced  automatically. 

A second  main  feature  here  is  placing  a student  within  an  instructional  line.  Teachers  can  insert  and  order 
pictures  of  didactic  activities  or  materials,  and  assign  different  activities  to  different  students.  A screenshot  of  these 
possibilities  is  given  in  Figure  3.  This  figure  shows  that  a teacher  can  successively  select  a child  (see  Program  Line  1 
with  the  photograph  of  the  student),  select  a kind  of  instructional  line  (2),  the  actual  difficulty  level  of  this  line  (3), 
the  variant  of  the  line  for  the  student  (remedial,  regular,  fast,  or  some  specific  material:  see  4),  and  finish  by  saving 
the  changes  made  (5).  The  next  time  this  boy  accesses  the  computer  software,  his  choices  to  play  or  work  are 
determined  by  the  teacher’s  instructional  management  decisions.  In  this  way,  the  student’s  choices  and  consequent 
activities  are  regulated  by  the  digital  instructional  management  system. dt  should  be  noted  that  the  student  is  not 
working  on  the  computer  to  complete  activities,  though  teachers  can  of  course  decide  to  include  this  possibility  as 
one  of  the  alternative  instructional  lines. 

Figure  3.  Placing  A Student  Into  An  Instructional  Line,  With  Level  And  Other  Specifics 


Third,  the  software  allows  automatic  logging  of  activities  for  each  of  the  students  working  with  the 

software. 

Information  about  Teachers.  Here  the  most  important  facilities  refer  to  adding  a new  teacher  to  a specific 
class,  removing  a teacher,  and  changing  existing  information  about  a teacher. 

Students’  Instructional  Self-Management.  A student  can  click  his  or  her  own  photograph  on  the  screen  that 
contains  all  the  photographs  of  the  students  in  class.  What  the  computer  screen  then  looks  like  is  shown  in  Figure  4. 
The  screen  shows  the  photograph  and  the  name  of  the  student  (top-right  comer).  The  top-left  comer  presents  the 
object  or  material  that  the  student  is  actually  working  on.  The  three  icons  at  the  bottom  of  Figure  4 each  illustrate 
one  possibility:  the  student  is  ready  and  wants  to  stop  with  this  task  (left  icon),  the  student  wants  to  select  a new 
activity  (middle  icon),  and  the  student  made  a wrong  choice  and  goes  back  one  decision  (right  icon).  Using  this 
interface,  the  student  has  to  know  which  virtual  assignment  corresponds  with  which  real  playing  or  instructional 
activity  in  the  classroom.  Of  course,  a student  can  be  helped  by  another  student  or  by  the  teacher. 
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Children  usually  play  or  work  with  the  real  three-dimensional  materials,  and  not  on  the  computer.  This 
feature  seems  to  suit  children  of  this  age  best,  and  it  also  overcomes  computer  access  constraints.  Moreover,  a 
teacher  can  change  or  extend  playing  or  other  structured  activities  within  a line,  to  improve  the  educational 
processes  or  to  check  their  desired  effects  on  one  or  more  students. 

Functioning  Of  The  Prototype 

Implementation  and  Effects  in  Kindergarten  Practice.  First  of  all,  by  using  the  prototype  the  teachers 
discovered  that  their  playing  and  learning  activities  and  materials  needed  acute  extension.  This  insight  was  based  on 
the  screening  of  the  students’  initial  competencies  by  the  parents  and  the  class  teacher,  and  the  subsequent  need  to 
integrate  the  different  levels  of  initial  competence  into  adequately  differentiating  playing  situations  and  instructional 
lines.  The  differences  between  their  students  were  much  bigger  than  was  accounted  for  in  the  available  activities, 
appliances,  and  materials.  A clear  consequence  was  that  a great  deal  of  the  school  budget  was  spent  on  buying  new 
playing  and  learning  materials. 

Figure  4.  Screen  Showing  The  Alternatives  Reflecting  A Student's  Self-Management 
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Second,  teachers  learned  pedagogically  and  instructionally  relevant  relationships  between  the 
characteristics  and  activities  of  the  children,  and  the  use  of  playing  and  learning  materials  and  appliances  in 
kindergarten.  According  to  the  teachers,  this  enabled  them  to  use  the  materials  to  promote  the  functioning  of  the 
students  much  better  than  before.  However,  teachers  took  much  more  time  than  before  to  instruct  the  students.  Since 
the  class  size  (25-30  students)  imposed  severe  time  restrictions,  the  time  problem  was  resolved  by  calling  in  parents 
to  instruct  specific  students.  This  was  not  really  satisfying.  Much  more  teacher  assistance  seems  needed  to 
adequately  integrate  student  differences  present  at  the  start  of  kindergarten.  The  software  thus  highlights  that  there 
may  be  more  work  than  only  one  teacher  in  a class  can  handle. 

Third,  some  of  the  students  of  four,  five,  or  six  years  old  learned  to  get  along  with  the  program  very  soon. 
They  could  assist  other  students  too,  if  necessary.  Some  of  the  students  had  more  difficulty  in  using  the  program. 
Because  students’  self-management  was  clearly  stimulated  by  the  software,  the  teacher’s  opportunity  to  devote  more 
time  to  the  students  who  needed  this  assistance  most,  was  enlarged. 

Finally,  a main  experience  of  all  persons  involved  in  the  development  was  that  the  co-development  of 
curriculum,  learning,  and  ICT  conditions  in  practice  started  a learning  process  for  all  (cf  Remillard,  2000).  The 
collaboration  between  teachers,  management,  research,  and  software  developers,  according  to  a use-oriented  design, 
proved  to  be  very  worthwhile. 

Potential  to  Optimize  Instructional  Management.  The  first  prototype  in  kindergarten  suggests  more 
possibilities  of  software  to  optimize  instructional  and  didactic  management  processes.  Instructional  lines  can 
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function  as  main  vehicles  to  explicitly  define,  integrate,  and  evaluate  curricular  features  and  processes,  diagnostics 
and  specific  assessments,  and  learning  processes,  at  different  educational  levels  simultaneously.  In  addition  to 
ordering  and  presenting  instructional  lines,  ICT  can  help  to  diagnose  and  assess  each  student’s  progress  in  both 
individual  and  group-normed  ways,  and  to  construct  specific  instructional  lines  for  marginal  students  or  students  at 
risk,  if  necessary  in  collaboration  with  external  specialists.  ICT  can  also  support  networked  instructional  and 
didactic  management  for  various  types  of  users  at  different  levels  of  the  educational  system,  e.g.  students,  teachers, 
school  administration  and  school  management,  and  external  professionalists  (see  e.g.  Tymms,  Merrell,  & 
Henderson,  2000).  An  exp  lorative  overview  of  relevant  levels,  potential  users,  and  optimization  functions  of  ICT,  is 
given  in  Table  1. 

Table  1.  Optimization  Functions  Of  ICT  At  Various  User  Levels 


Users  /High-Low 
Educational  Levels 

Optimizing  Functional  Features  of  ICT 

Policy  / Development 
/ Research 

Collect  and  evaluate  data  from  (interactions  between)  curriculum  and  learning  at  all  levels. 
Plan,  coordinate,  develop,  evaluate  quality  aspects  and  optimization  of  curriculum  / learning. 

Regional  Instances 
(e.g.  School 
Psychologists, 
Advisors,  Youth  Aid, 
Municipal  Policy) 

Aggregation  of  in  -school  data  to  data  at  regional  level  and  regional  developments. 

Plan,  coordinate,  develop,  evaluate  quality  aspects  and  optimization  of  curriculum  / learning. 
Develop  and  evaluate  qualitiesof  regional  features  relevant  to  curriculum  / learning. 

Develop,  evaluate  optimization  of  curriculum  / learning  re.  students  and  specific  students  at  risk. 
Collaborate  with  school  / location  management,  teachers,  students,  and  parents. 

School  Board, 
Management,  Staff 

Analysis  of  curriculum  / learning  data  re.  unit  / location  / school  level  and  developments. 

Plan,  coordinate,  develop,  evaluate  quality  aspects  and  optimization  of  curriculum  / learning. 
Develop,  evaluate  optimization  of  curriculum  / learning  re.  students  and  specific  students  at  risk. 

Teachers 

Analysis  of  curriculum  / learning  data  re.  student  / (small-)  group  level  and  developments. 

Plan,  coordinate,  develop,  evaluate  quality  aspects  and  optimization  of  curriculum  / learning. 
Develop,  evaluate  optimization  of  curriculum  / learning  re.  students  and  specific  students  at  risk. 
Complete  / change  / output  indicators  re.  levelrspecific  curriculum  and  student  characteristics. 
Complete  / change  / output  indicators  re.  instructional  lines  and  groupings  for  student(s). 
Complete  / change  / output  (normed)  indicators  on  individual  and  group  achievement. 

Parents 

Completion  of  initial  and  other  competencies  of  their  own  child. 
Overview  of  their  child’s  developments  and  progress  (‘virtual  portfolio’). 

Students  / Learners 

Automatic  logging  and  storing  of  activities  chosen,  and  logging  of  processes  and  results. 
Select  actual  or  next  instructional  line,  and  actual  or  next  activity. 

Plan,  coordinate,  develop,  evaluate  quality  aspects  and  optimization  of  curriculum  / learning. 

Integration  of  Diagnostic  and  Achievement  Evaluation.  The  evaluation  of  the  prototype  also  suggests  the 
specification  of  three  kinds  of  activities  within  an  instructional  line:  regular  activities,  evaluative  activities,  and 
normed  activities.  Normed  diagnostic  and  achievement  indicators  in  instructional  lines  are  defined  as  representing  a 
generally  valid  kernel  structure  of  pedagogical-didactic  features  of  the  curriculum.  This  kernel  allows  a structured 
diagnostic  evaluation  of  a child’s  characteristics  and  progress  in  individual,  social  or  group,  and  normed  respects,  in 
the  course  of  time.  Such  a multidimensional  evaluation  scheme  is  needed  to  provide  a complete  responsible 
stimulation  of  the  development  of  a child,  from  the  start  in  kindergarten  onwards.  Moreover,  the  kernel  structure  can 
be  changed  locally  or  extended  into  local  instructional  lines  which  better  fit  situational  teaching  or  learning 
conditions,  as  long  as  the  normed  referents  in  the  lines  are  kept  intact.  The  same  or  comparable  indicators  can  be 
used  as  a common  frame  of  reference  for  the  preventive  commitment  of  external  specialists  like  a school 
psychologist  (cf.  Griffin  & Beagles,  2000). 

Discussion 

In  a four-year  project,  attention  was  focused  on  the  improvement  of  early  educational  practice  in 
kindergarten.  To  this  end,  instructional  lines  with  respect  to  ordinary  playing  and  learning  appliances  were  co- 
developed with  teachers  and  management.  Ordering  of  instructional,  didactic,  and  diagnostic  materials  occurred 
with  symbols  (logo,  color,  names)  to  make  them  concrete  for  young  children.  Also,  software  was  developed  and 
used  in  two  classrooms  to  check  their  functioning  in  educational  practice. 

The  results  can  be  summarized  as  follows.  First,  starting  characteristics  of  each  student  are  measured  and 
discussed  systematically  by  parents  and  teacher.  Problems  or  risk  characteristics  can  get  more  systematic  and 


149 


preventive  pedagogical  attention,  if  necessary  by  early  inclusion  of  specialists  outside  school.  This  means  that 
preventive  cooperation  between  parents  and  kindergarten  teachers  can  increase  considerably  in  comparison  with 
current  practice  in  early  education. 

Second,  next  to  the  usual  free-playing  and  whole -group  sessions,  each  student  can  now  get  systematic  and 
immediate  curricular  support  at  his  or  her  own  levels  of  competency.  Controlled  specific  support  can  become 
available  for  the  students  who  need  this.  Also,  automatic  logging  and  monitoring  of  student  and  class  or  school 
results  becomes  available,  which  is  not  dependent  on  only  one  teacher  or  small  group  of  teachers. 

Third,  the  software  allows  a growth  in  the  independence,  self-regulation,  and  self-responsibility  of  the 
students,  which  is  also  possible  because  the  students  themselves  can  assist  each  other  in  communicating  with  the 
computer.  This  advantage  will  increase  when  the  students  get  older. 

Fourth,  specific  operationalization  of  e.g.  giving  equal  opportunities  to  students  from  ethnic  minorities, 
introducing  quality  standards  in  education,  or  increasing  safety  at  school  becomes  possible.  Here,  systematic 
innovation  support  will  be  needed  in  the  developing  kindergartens  and  schools.  In  the  long  run,  the  software  can  act 
like  a supportive  planning  and  instructional  management  system  for  teachers,  students,  the  school,  the  school  board, 
and  the  parents  alike. 

Fifth,  though  quantitative  data  are  not  yet  available,  the  hypothesis  seems  legitimate  that  the  use  of  digital 
instructional  management  in  early  education  will  stimulate  the  optimization  of  educational  processes  and  outcomes. 
Compared  with  traditional  early  education,  more  prosocial  and  constructive  learning  processes  can  be  realized  in 
practice,  in  collaboration  between  teachers  and  school  staff,  students,  parents,  and  specialists  from  outside  school 
but  working  within  the  framework  of  the  same  instructional  lines  and  progress  evaluation.  Quantitative  research  to 
verify  this  hypothesis  for  students  at  risk  in  particular  should  be  designed,  but  until  now  research  facilities  are  only 
present  for  developmental  follow-up  projects.  A first  project  in  early  and  primary  education  concentrates  on 
developing  the  pedagogical-didactic  kernel  structure,  as  defined  above.  Moreover,  implementation  in  practice  co- 
occurs with  development  and  implementation  of  a second  prototype  of  the  digital  instructional  management  system. 
The  design  of  this  Internet-based  software  is  specified  on  the  outlines  given  in  Table  1 . 

A second  follow-up  project  is  carried  out  in  secondary  education.  First,  local  instructional  lines  are 
developed  and  introduced  in  school  practice.  Teachers  develop  these  lines  within  a self-chosen  project  on  ‘Water 
and  environment',  for  small  groups  (2-4)  of  students.  To  aid  the  self-management  of  students,  teachers  create 
activity  worksheets  for  each  activity  within  the  instructional  lines.  The  worksheets  contain  aspects  of  preparatory 
instructional  specifics,  the  actual  performance,  and  the  evaluation  of  project  activities.  Curriculum  and  learning 
materials  are  present  in  or  around  the  classroom,  school  library,  or  the  Internet.  The  worksheets  provide  instructional 
information  about  learning  processes,  materials  and  sources,  appliances  that  can  be  used,  and  diagnostic  evaluation. 
Evaluation  and  judgment  become  more  individualized,  but  reliable  or  valid  achievement  indicators  or  norms  hardly 
exist  at  present.  Development  of  such  indicators  will  become  possible  by  authoring  Internet-based  software, 
comparable  to  the  second  prototype  for  early  education.  This  prototype  for  secondary  education  is  now  under 
development. 
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Short  Description 

This  session  will  analyze  peer  online  discourse  on  a collaborative  team  problem  solving  assignment  in  a 
large  college  class,  in  an  attempt  to  understand  the  group  dynamics  in  online  collaborative  environments.  Issues  and 
problems  associated  with  peer  online  collaboration  and  implications  for  the  possible  roles  of  instructors  and 
instructional  designers  will  be  discussed. 

Abstract 

Collaborative  learning  has  been  gaining  increased  attention  in  the  practice  and  research  of  education.  As 
Vygotsky  (1978)  points  out,  social  interactions  play  a very  important  role  in  learning.  Research  shows  that 
learning  to  work  with  a small  group  can  promote  group  learning  as  opposed  to  individual  learning. 
(Johnson  and  Johnson,  1975;  Hamm  and  Adams,  1992;  Bosworth  and  Hamilton,  1994;  Bruffee,  1999). 
Computer  technologies,  as  scholars  believe,  can  support  collaborative  learning  and  be  beneficial  to 
learners  in  many  different  ways  (Harisim,  1990;  Bonk  and  King,  1998;  Lin  et  al,  1999;  Schwartz  et  al, 
1999).  Various  forms  of  Conputer  Supported  Collaborative  Learning  (CSCL)  have  now  been  widely  used 
as  supplements  to  traditional  classroom  learning  as  well  as  in  distance  education  (Bonk  and  King,  1998). 

Research  shows  that  learning  in  groups  improves  students ' achievement  of  learning  objectives.  Some 
indicate  that  a facilitator  will  smooth  the  process  of  collaboration  (Johnson  et  al,  1987;  Hooper,  1992; 
Moore  and  Kearsley,  1996;  Brandon  & Hollingshead,  1999;  Bernard  et  al,  2000).  Advocates  for  peer- 
controlled  collaboration  claim  that  learners  usually  feel  more  comfortable  discussing  without  the  presence 
of  the  instructor.  This  may  be  explained  by  research  which  indicates  that  when  participants  have  more 
active  control  of  the  learning  process,  learning  increases  (Jensen,  1996).  This  previous  research  lays  a 
foundation  for  hypotheses  about  online  peer  collaboration  using  different  types  of  moderators  and 
strategies;  however,  there  is  a lack  of  empirical  evidence  for  making  strong  recommendations.  Which 
approach  is  more  effective  for  group  collaboration,  the  peer-controlled  online  collaboration  forum  or  the 
structured  and  moderated  online  collaboration  forum?  This  question  is  a difficult  one  to  answer,  but  this 
study  begins  to  answer  the  question  by  investigating  the  quality  of  peer  online  discourse.  This  study 
discusses  possible  activities  to  enhance  web -based  peer  collaboration  activities  as  well  as  the  roles  that 
instructors  and  instructional  designers  can  take  in  the  creation  of  effective  online  peer  collaboration 
activities. 

In  this  study,  complete  electronic  discourse  transcripts  of  college  students  in  a collaborative  problem 
solving  assignment  will  be  analyzed  to  measure  the  quality  of  peer  online  collaboration  with  the  two 
different  methods,  to  help  understand  the  group  dynamics  and  types  and  patterns  of  social  interaction 
which  occur  online. 
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Introduction 


Collaborative  learning  has  been  gaining  increased  attention  in  the  practice  and  research  of  education.  As 
Vygotsky  (1978)  points  out,  social  interactions  play  a very  important  role  in  learning.  Research  shows  that  learning 
to  work  with  a small  group  can  facilitate  group  learning  as  opposed  to  individual  learning.  (Johnson  and  Johnson, 
1975;  Hamm  and  Adams,  1992;  Bosworth  and  Hamilton,  1994;  Bruffee,  1999) 

Computer  technologies,  as  demonstrated  in  research  and  practice,  can  support  collaborative  learning  and  be 
beneficial  to  learners  in  different  ways  (Harisim,  1990;  Bonk  and  King,  1999).  Computer  technologies,  for  example, 
make  collaborative  learning  achievable  even  when  face-to-face  communications  are  less  accessible,  and  more 
importantly,  they  can  be  designed  to  support  delayed  reflection  (Lin  et  al,  1999;  Schwartz  et  al,  1999). 

Statement  of  the  Problem 

As  computers  and  networking  technologies  are  introduced  into  education,  the  form  of  education  is 
changing  dramatically.  A variety  of  computer-supported  education  has  been  widely  used  to  promote  learning  beyond 
the  limits  of  geographic  location  and  time.  With  technologies  available  today,  learners  can  establish  connections  and 
contacts  with  instructors  and  peer  learners  beyond  the  traditional  classroom.  Computer  Supported  Collaborative 
Learning  (CSCL)  Tools,  including  online  forums,  facilitate  learning  by  offering  opportunities  for  interaction  and 
collaboration.  Online  forums,  as  one  of  the  most  widely  used  CSCL  tools,  have  been  introduced  to  today’s  colleges 
to  facilitate  peer  collaboration  (Bonk  and  King,  1999). 

In  large  classes,  students  often  feel  little  attention  to  their  individual  needs,  and  may  sometimes 
consequently  feel  isolated  in  the  learning  process.  Active  learning  in  large  classes  often  occurs  in  group  activities. 
This  group  collaboration  has  been  shown  to  enhance  learning  (Johnson  and  Johnson,  1975;  Hamm  and  Adams, 

1992;  Bosworth  and  Hamilton,  1994;  Bruffee,  1 999).  Some  research  indicates  that  a facilitator  will  smooth  the 
process  of  collaboration  (Johnson  et  al,  1987;  Hooper,  1992;  Moore  and  Taylor,  1996).  Advocates  for  peer- 
controlled  collaboration  claim  that  learners  usually  feel  more  comfortable  discussing  without  the  presence  of  the 
instructor.  This  may  be  explained  by  research  that  shows  when  participants  have  more  active  control  of  the  learning 
process,  learning  increases  (Jensen,  1996).  This  previous  research  lays  a foundation  for  hypotheses  about  online 
peer  collaboration  using  different  types  of  moderators  and  strategies;  however,  there  is  a lack  of  empirical  evidence 
for  making  strong  recommendations.  Which  approach  of  peer  online  collaboration  is  more  effective  for  group 
collaboration,  the  peer-controlled  online  collaboration  forum  or  the  structured  and  moderated  online  collaboration 
forum?  Redundant  with  above,  use  one  place  or  another,  or  re -word.  This  question  is  a difficult  one  to  answer,  but 
this  study  begins  to  answer  the  question  by  investigating  the  quality  of  peer  online  discourse  and  discusses  possible 
activities  to  enhance  web-based  peer  collaboration  activities  as  well  as  the  roles  that  instructors  and  instructional 
designers  can  take  in  the  creation  of  effective  online  peer  collaboration  activities. 

Research  Question 

This  study  begins  to  consider  the  question  of  online  peer  collaboration  structures  by  investigating  the 
quality  of  peer  online  discourse.  More  specifically,  the  study  intends  to  explore  if  there  are  differences  in  the  quality 
of  students’  online  collaboration  in  the  two  types  of  online  collaborative  forums,  and  tries  to  understand  why  such 
differences,  if  any,  occur. 

Deflnition  of  Terms 

In  an  attempt  to  alleviate  any  confusion  due  to  the  use  of  different  terminology,  relevant  terms  are  defined 
based  on  how  they  are  used  in  this  study. 

• Collaboration: 

Mutual  engagement  of  participants  in  a coordinated  effort  to  reach  a shared  goal.  Bruner  (1991)  defines 
collaboration  as  a process  to  complete  tasks  that  cannot  be  fulfilled  individually,  or  cannot  be  done  efficiently 
otherwise.  Mutual  engagement  includes  joint  efforts  to  develop  a set  of  goals  and  directions;  to  share 
responsibilities;  and  to  work  together  to  achieve  those  goals,  utilizing  the  expertise  of  each  collaborator. 

• Computer  Supported  Collaborative  Learning  (CSCL): 
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An  learning  method  that  implements  information  and  communications  technology  to  facilitate  collaborative 
learning  (Wasson,  1997,  1998). 

• Online  forum: 

An  asynchronous  electronic  means  for  communication  and  collaboration,  where  participants  can  post  and 
respond  to  messages  and  discuss  and  collaborate  on  the  Internet. 

• Peer-controlled  online  forum: 

A web-based  bulletin  board  for  peer  learners  to  communicate  with  one  another  as  they  wish  without 
moderation  or  any  other  active  participation  of  the  instructor. 

• Structured  and  moderated  online  forum: 

A web-based  bulletin  board  for  peer  learners  to  communicate  and  collaborate,  where  the  instructor  takes 
active  responsibility  to  structure,  scaffold  and  moderate  the  collaboration. 

• Problem  solving: 

“ In  general,  problem  solving  involves  dealing  with  new  and  unfamiliar  tasks  or  situations  that  present 
some  obstacle,  and  relevant  solution  methods  are  not  known.  In  Gagne’s  conditions  of  learning,  problem  solving  is 
the  skill  of  recalling  and  applying  a set  of  rules  in  the  proper  sequence  to  solve  a problem.  It  is  also  referred  to  as 
higher-order  rule  learning”  (Gredler,  1997,  364). 


LITERATURE  REVIEW 
Sociocultural  Theory 

Vygotsky  states  that  learning  is  a developmental  process,  and  it  occurs  in  social  activities  (Driscoll,  1994). 
One  of  Vygotsky’s  most  influential  contributions  is  to  analyze  human  psychological  development  from  a social- 
cultural  history  method  (Gredler,  1997).  As  Vygotsky  notes  that  higher  ordered,  complex  thinking  is  gradually 
developed  through  social  interactions  with  others  in  the  culture  (Gredler,  1997). 

Consistent  with  sociocultural  theory,  research  shows  that  learning  in  groups  improves  students’ 
achievement  in  various  learning  objectives  and  some  indicate  that  a role  of  facilitator  will  smooth  the  process  of 
collaboration  (Johnson  et  al,  1987;  Hooper,  1992;  Moore  and  Taylor,  1996).  Research  shows  that  successful  group 
learning  promotes  higher  order  thinking  (Blumenfeld  et  al,  1996). 

As  Vygotsky  (1 978)  points  out,  the  learning  environment  is  critical,  as  people  learn  from  people  around 
them  and  are  influenced  by  the  culture  of  the  learning  context.  According  to  Sociocultural  theorists,  people  learn 
from  mediations  and  scaffoldings,  which  are  offered  within  one’s  zone  of  proximal  development  (ZPD)  from  more 
capable  peers  or  expert  (Wertsch,  1985;  Gredler,  1997;  Bonk  and  King,  1999).  Vygotsky  defines  ZPD,  one  of  the 
key  concepts  in  the  sociocultural  theory,  as  the  distance  between  one’s  independent  competency  and  that  obtained 
with  assistance  from  the  expert  or  in  collaboration  with  more  capable  peers  (Wertsch,  1985).  Such  a distance  can  be 
bridged  by  scaffolding,  as  external  assistance  is  gradually  reduced  and  the  learner  finally  achieves  independent 
competency  in  the  task  (Gredler,  1997).  Research  and  scholarship  also  claim  that  computer  supported  collaborative 
learning  environment  can  provide  the  Zone  of  Proximal  Development  (ZPD)  (Salomon,  Globerson  and  Guterman, 
1989;  Newman  et  al,  1993).  In  collaborative  learning  environment,  learners’  ZPD  can  be  reached  and  extended 
through  communications  and  collaborations  with  peers  and  the  instructor.  Also  the  applications  of  computer 
technologies  can  help  scaffold  within  learners’  Zone  of  Proximal  Development  in  collaborative  learning  and 
therefore  to  promote  learning  outcomes. 

Collaborative  Learning 

Collaborative  learning,  consistent  with  sociocultural  theory,  is  gaining  more  and  more  attention  in 
education.  Social  interactions  play  an  important  role  in  learning  (Vygotsky,  1978);  in  fact  peer  group  work  can  have 
significant  impact  on  varied  learning  outcomes  (Jonassen  et  al,  1995;  Berge  and  Collins  1 995). 

Very  often,  independent  and  individual  learning  can  leave  a learner  passive  and  inactive.  Vygotsky  (1978) 
suggests  that  collaboration  should  help  individuals  make  progress  through  their  zone  of  proximal  development  by 
the  activities  in  which  they  are  engaged.  Collaborations  in  peer  group  work  increase  engagement  in  the  learning 
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process,  and  facilitate  cognitive  development.  Collaborative  learning  changes  the  traditional  teaching  and  learning 
practice;  the  instructor  is  no  longer  simply  the  “knov^ledge  giver”  or  the  center  of  pov^er  in  the  learning  process. 
Instead,  peer  collaboration  transformed  traditional  education  to  learner-centered,  active  “learning”  activities.  In  peer 
collaboration,  learners  share  knowledge,  ideas  and  significant  thinking,  and  therefore  learn  from  one  another  and 
achieve  goals  that  may  not  be  obtained  in  isolated  individual  learning.  Interpersonal  communications  enable  and 
encourage  learners  to  confer,  reflect  and  help  to  develop  meaningful  learning  (Johnson  and  Johnson,  1975;  Clements 
andNastasi,  1988). 

As  students  are  led  to  reflect  upon  and  confront  different  ideas  in  peer  collaborations,  to  provide  meaningful 
feedback  and  support  to  one  another,  Cazden  (1988)  suggests  that  learners  therefore  can  learn  in  a constructive  way 
and  benefit  in  cognitive  development.  Research  shows  that  to  learn  and  work  with  a small  group  can  facilitate 
learning  as  opposed  to  individual  learning  (Johnson  and  Johnson,  1975;  Hamm  and  Adams,  1992;  Bosworth  and 
Hamilton,  1994;  Bruffee,  1999). 

Collaborative  learning  environments  provide  a means  to  create  more  engaging  and  dynamic  instructional 
settings.  Research  frequently  shows  that  there  are  clear  educational  advantages  to  be  derived  from  collaborative 
activities  among  students  (Del  Marie  Rysavy  and  Sales,  1991;  Slavin,  1992). 

Computer  Supported  Collaborative  Learning 

Computer  technologies  have  further  extended  the  formats  of  collaborative  learning.  Varied  forms  of 
Computer  Supported  Collaborative  Learning  (CSCL)  have  been  adopted  in  education  of  both  cognitive  and  affective 
domains  (Clements  andNastasi,  1988;  Hoopers,  1992;  Repman,  1993;  Jehng,  1997;  Rada  and  Wang,  1998). 
Researcher  (Dede,  1996)  suggests  that  collaborative  online  learning  should  be  integrated  into  higher  education  in  the 
twenty  first  century. 

Asynchronous  online  forums  further  facilitate  self-paced  and  self-controlled  learning  and  collaboration. 
Asynchronous  online  forums  extend  peer  collaboration  beyond  the  classroom,  expand  learners’  control  over  the  time 
for  collaboration,  and  increase  the  time  available  to  read,  to  reflect  and  to  reply  to  a message  and  formulate  ideas  in 
writing.  Scholars  (Vygotsky,  1978;  Harasim,  1990)  believe  that  the  change  from  oral  to  verbal  communication  also 
contributes  to  learning  effectiveness  when  learners  have  to  concrete  and  articulate  their  thinking  in  writing. 
Asynchronous  online  forums  also  extend  the  time  span  for  peer  collaboration,  making  it  more  flexible.  Learners  can 
conduct  the  collaboration  online  as  long  as  needed,  and  therefore  can  possibly  improve  in-depth  investigation  of  the 
collaborative  task  (Harasim,  1990). 

Virtual  discourse  in  online  forums  can  be  stored  and  easily  retrieved,  and  hence  can  facilitate  delayed 
reflection,  and  is  always  open  to  review  (Scardamalia  and  Bereiter,  1994;  Blumenfeld,  et  al,  1996).  Technologies 
also  make  participants’  thinking  and  reasoning  visible  (Lin  et  al,  1999).  By  requiring  students  to  articulate  their 
thinking,  by  facilitating  comments  and  communications  among  learners,  and  by  making  it  easy  to  view  by  thread, 
date  or  author,  instructors  can  use  asynchronous  online  forums  to  support  delayed  reflection.  In  addition, 
experiences  indicate  that  some  students  that  are  not  comfortable  in  classroom  collaboration  can  be  very  active  and 
engaged  online,  as  there  is  no  time  restriction  or  competitions  or  interruption  in  the  course  of  peer  collaboration 
(Harasim,  1990).  In  asynchronous  online  collaborative  environment,  even  less  active  learners  can  have  the  same 
time  and  opportunity  to  express  themselves,  without  fear  of  being  interrupted  by  more  aggressive  peers.  In  this 
sense,  it  can  ensure  equal  opportunity  in  peer  collaboration. 

However,  online  collaboration  forums  do  not  simply  make  peer  collaboration  easier.  First  of  all, 
asynchronous  online  collaboration  is  very  different  from  face-to-face  meetings.  Participants  communicate  not  in  the 
real  time  but  in  the  times  at  their  individual  choice.  Yet  the  lack  of  non-verbal  communication  in  an  online  forum 
makes  misunderstanding  and  miscommunications  less  detectable,  and  thus  requires  higher  communication 
competency  in  writing.  Concerns  for  miscommunications  can  make  participants  more  conscious  and  nervous  about 
writing  and  diction  in  their  online  collaboration  efforts.  Second,  learners  may  feel  frustration  with  the  technology 
itself.  They  may  also  feel  like  they  are  “talking  to  a vacuum”  when  no  immediate  response  is  available.  In  addition, 
consensus  can  be  more  time-consuming  and  is  generally  more  difficult  to  achieve  in  online  collaborations. 

Moreover,  the  quality  of  decision  may  be  sacrificed  to  compromise  delays  caused  in  online  collaboration  (Harasim, 
1990). 

Scaffolding 

Closely  related  to  the  Zone  of  Proximal  Development  is  scaffolding.  Scaffolding  refers  to  varied  forms  of 
external  support  and  assistance  to  learners  to  complete  a task  that  is  beyond  their  individual,  independent  efforts 
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(Gredler,  1997;  Bonk  and  Cunningham,  1999).  Scaffolding  is  also  a process  to  gradually  reduce  expert  assistance  as 
the  learner  gets  more  and  more  competent  at  the  task  in  the  learning  process  (Gredler,  1 997).  Scaffolding  efforts  are 
to  provide  necessary  assistance  and  therefore  to  help  learners  achieve  the  zone  of  unknown  gradually. 

Scaffolds  can  be  provided  by  different  external  agents,  teachers,  peer  learners  or  supporting  materials 
(Wertsch,  1985;  Gredler,  1997;  Bonk  and  King,  1999).  With  computer  technologies,  several  design  features  can  be 
embedded  to  provide  scaffolds  such  as  process  display,  process  prompts,  process  models  and  reflective  social 
discourse  (Lin  et  al,  1 999). 

As  suggested  by  research  and  scholarships,  the  role  of  facilitator  can  smooth  the  process  of  collaborative 
learning  (Johnson  et  al,  1987;  Hooper,  1992;  Moore  and  Taylor,  1996).  In  collaborative  learning  environment,  the 
role  of  instructor  changes  from  the  center  of  authority  to  a facilitator  or  co-conspirator  ( Hamm  and  Adams,  1992; 
Flannery,  1994).  In  the  peer  collaborative  learning  environment,  the  instructor  does  not  serve  as  an  information 
giver,  but  rather,  learners  have  more  active  control  over  the  learning  process.  In  addition,  students  need  to  be 
engaged  in  a very  rich  context  to  collaborate  and  develop  higher  order  thinking  (Bosworth  and  Hamilton,  1994). 
Similar  to  traditional  classroom  collaboration,  in  online  collaborative  learning  environment,  the  instructor  can  take 
the  role  of  a moderator  and  facilitator  as  needed  in  the  course  of  peer  online  collaboration.  Supports  that  the 
instructor  can  provide  in  the  learning  process  include  motivating  students,  monitoring  and  regulating  learners 
performance,  provoking  reflection,  modeling,  moderations  and  scaffolding  (Brown  & Palinscar,  1989;  Hamm  and 
Adams,  1992;  Bosworth  and  Hamilton,  1994;  Brandon  & Hollingshead,  1999;  Jonassen,  1999). 

Forms  of  scaffolding  can  include  hinting,  elaborating,  guiding,  questioning,  prompting,  probing, 
simplifying,  or  other  similar  learning  supports  (Bonk  and  Cunningham,  1 999).  The  scaffolding  efforts  can  also  be 
classified  as  conceptual,  metacognitive,  procedural  and  strategic  scaffolds,  as  Hannafin  et  al  (1999)  suggest. 
Conceptual  scaffolds  are  to  guides  students  in  what  to  consider;  metacognitive  scaffolds  are  to  guide  how  to  think 
during  learning  process;  procedural  scaffolds  are  to  guide  students  how  to  utilize  the  available  features  in  the 
learning  environment;  strategic  scaffolds  are  to  provide  macro -strategy  initially  or  ongoing  as  needs  or  requests  arise 
so  to  guide  students  in  analyzing  and  approaching  learning  tasks  or  problem  (Hannafin  et  al,  1999).  The  various 
scaffolds  can  be  embedded  in  a well -designed  computer  supported  collaborative  learning  environment  and  therefore 
to  enhance  learning  and  improve  cognitive  performance. 

Structuring  and  Moderating  Online  Forums 

Some  research  documents  that  when  participants  have  more  active  control  over  the  learning  process, 
learning  increases  (Jensen,  1 996).  Therefore  less  participation  of  the  instructor  may  foster  more  active  learning.  But 
as  some  research  indicates,  participation  in  electronic  discussion  is  often  passive  without  instructor’s  participation. 

As  Aviv  and  Golan  (1998)  report,  most  students  in  distance  learning  environments  participate  in  electronic 
discussions  by  passively  reading  some  peers’  postings,  and  only  responding  to  a few  of  others’  messages,  also  very 
few  students  raise  questions.  Aviv  and  Golan  (1998)  also  found  that  when  preplanned,  and  focused,  and  students  are 
led  through  the  learning  process,  electronic  discussions  can  lead  to  a highly  successful  learning  experience.  Scholars 
(e.g.  Slavin,  1 995)  recommend  the  use  of  structured  protocol  to  direct  interactive  discourse  among  peer  learners  so 
to  reduce  off-track  and  passive  behaviors  while  ensuring  opportunities  for  equal  participation. 

When  shifting  from  the  “center  of  authority”  to  “co-conspirator,”  the  instructor  needs  to  take  substantial 
responsibilities  in  the  course  of  collaborative  learning  in  order  to  foster  a learner-centered  peer  collaborative 
learning  environment.  As  for  students’  participation  in  online  collaborations,  strong  social  skills  and  good  group 
dynamics  are  essential  to  effective  and  productive  learning  outcomes.  Group  dynamics  contribute  to  students’ 
performance  in  collaborative  leamin  g and  their  satisfaction  for  the  learning  experience  (Bosworth  and  Hamilton, 
1994).  Some  participants’  actions  of  “free  riding”  and  “social  loafing”  and  failure  to  contribution,  however,  can 
damage  others’  enthusiasm  and  motivation  in  the  course  of  collaborative  learning.  In  addition,  the  feeling  of  “talking 
in  a vacuum”  with  online  collaboration  and  other  frustrations  with  technology  and  many  other  factors  make  online 
collaboration  a challenge  to  many  participants.  Research  indicates  that  even  in  learner-centered  learning 
environment,  as  in  online  collaborative  learning,  moderations  and  structuring  of  the  learning  process  are  needed  for 
successful  learning  experiences  (Flannery,  1994).  Online  collaborations  need  to  be  well  organized,  facilitated,  and 
moderated  to  be  effective  and  successful  (Hamm  and  Adams,  1992;  Flannery,  1994). 

Structuring 

Little  research  has  been  focused  on  how  to  structure  peer  online  collaboration.  In  practice,  peer 
collaborations  are  often  structured  by  the  learners  themselves  by  chunking  the  group  assignment  into  “detailed 
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division  of  labor”  (Althauser  and  Matuga,  1999).  In  order  to  reduce  “free  riding”  in  teamwork,  instructors  also  often 
structure  the  online  peer  collaborations  by  assigning  a set  of  questions  to  each  member  of  a group.  Therefore  every 
group  member  must  participate  by  taking  charge  of  the  part  of  the  task  to  which  they  are  assigned.  Such  structuring, 
however,  has  drawbacks,  as  it  tends  to  lead  students  to  work  on  the  group  assignment  in  a cooperative  way  and  thus 
miss  the  opportunity  for  collaborative  learning.  Individuals  can  execute  their  own  part  of  the  assignment  without 
collaborative  efforts  from  peers,  and  they  may  not  contribute  to  other  part  of  the  assignment  either. 

Bosworth  and  Hamilton  (1994)  suggest  that  instructors  create  and  develop  requisite  structure  and  process 
of  the  group  to  achieve  better  collaborative  learning.  Similarly,  Slavin  (1995)  suggests  using  structured  protocols  to 
direct  student  interactions.  Efforts  of  structuring  peer  online  collaborations  in  this  study  are  two  fold,  one  is  to 
structure  group  work,  and  the  other  is  to  structure  the  collaboration  in  online  forums. 

Moderation 

The  importance  of  moderating  peer  collaborative  learning  has  been  recognized  in  practice  and  research 
(Harasim,  1990;  Hamm  and  Adams,  1992;  Bosworth,  and  Hamilton,  1994).  Bemnard  et  al  (2000)  also  suggest  that 
instructors  assume  a facilitator’s  role  in  an  online  collaborative  learning  environment.  Strategies  for  online  forum 
moderation  include,  but  are  not  limited  to  the  following:  to  maintain  the  discourse  focused  on  the  topic,  to  check 
team  progress,  to  promote  equal  participation,  to  provide  individual  support  as  needed. 

Research  Design  and  Implementation 

The  study  was  conducted  in  a large  undergraduate  introductory  statistics  course  at  a major  university  in  the 
northeastern  United  States.  As  a required  course  by  many  majors  at  the  university,  this  course  included  students 
from  all  majors  and  class  standings.  Forty -one  groups,  consisting  of  one  hundred  and  forty  eight  students 
volunteered  to  participate  in  this  study.  All  the  participants  were  already  experienced  with  teamwork  in  this  course 
before  the  study  began.  Participants  were  given  the  opportunity  to  re-form  teams  of  four  or  five  each,  based  on  their 
past  team  work  experience  with  others  in  this  course.  The  forty-one  groups  were  then  randomly  assigned  to  either  of 
the  two  types  of  forums  (peer  controlled  or  structured  and  moderated)  and  asked  to  collaborate  on  twelve  problem 
solving  scenarios  in  the  online  forums  as  a team  assignment.  As  a result  of  the  random  assignment,  twenty-one 
groups  were  using  the  peer-controlled  online  collaborative  forum,  and  twenty  groups  were  using  the  structured  and 
moderated  online  forum.  Each  team  was  provided  a private  online  forum,  which  only  team  members  had  access  to. 
For  each  scenario,  participants  were  asked  to  recommend  an  appropriate  statistical  technique  to  address  the  problem 
and  provide  justifications  for  the  recommendation.  The  study  took  place  in  the  last  few  weeks  of  the  semester  when 
students  were  extremely  busy  with  finals,  projects  and  exams  for  other  course  work,  and  participants  had  two  weeks 
to  work  on  the  assignment  before  final  submission.  Before  the  study  began,  the  participants  had  already  experienced 
many  technology -enhanced  learning  opportunities  in  this  course,  such  as  a course  website,  the  password -protected 
web-based  electronic  textbook,  online  quizzes  for  individual  assessment,  and  online  forum  for  group  and/or  class 
bulletin  board. 

Teams  assigned  to  the  peer-controlled  collaboration  forums  worked  with  their  team  members  on  the 
problem  solving  assignment  without  any  intervention  from  the  instructor  or  anyone  else.  Teams  assigned  to  the 
structured  and  moderated  collaboration  forums,  on  the  other  hand,  were  closely  monitored  by  the  instructor  and 
received  scaffolding,  moderation  and  structuring  prompts  from  the  instructor  as  needed. 

Data  Collection  And  Analysis 

All  the  electronic  transcripts  of  each  group’s  discourse  on  the  online  forums  were  collected  in  a previous 
experimental  study  conducted  by  the  lead  researcher  and  analyzed  with  multiple  -party  discourse  analysis  principle 
applied.  Since  this  research  was  interested  in  teams  rather  than  individual  behaviors,  each  team’s  electronic  postings 
on  the  private  forum  were  saved  and  analyzed  as  one  single  unit. 

Data  were  stored  and  organized  in  NVivo,  a Qualitative  Data  Analysis  (QDA)  software.  The  researcher 
also  wrote  memos  for  each  group  to  record  general  impression  and  important  thoughts  each  time  reading  and/or 
coding  the  transcripts,  and  those  memos  were  then  analyzed  together  with  the  electronic  transcripts. 

The  data  analysis  was  carried  out  as  an  ongoing  process  as  the  research  proceeded,  when  the  teams  were 
using  the  online  forums  for  the  problem  solving  assignment,  the  first  author  read  the  electronic  transcripts  as  they 
became  available  everyday  and  wrote  brief  memos  to  record  ideas  and  impressions  constantly.  Reading  the 
transcripts  and  recording  initial  ideas  as  an  ongoing  process  helped  the  researcher  to  understand  the  dynamic  nature 
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of  teamwork  and  online  collaborations.  It  felt  like  the  researcher  were  experiencing  the  team  process,  learning  and 
observing  collaboration  real  time  as  the  participants  did.  It  enabled  the  researcher  to  capture  the  dynamic  nature  of 
teamwork  without  meeting  any  of  the  participants  in  person  or  interrupting/disturbing  the  team  dynamics. 

Some  teams  used  the  online  forums,  which  were  originally  designed  to  be  synchronous  communication 
tools,  for  real  time  team  collaborations.  They  set  up  a time  that  all  members  logged  on  and  conducted  the 
collaboration  as  if  in  a real  time  face-to-face  meeting.  In  those  incidences,  the  lead  researcher,  when  she  happened  to 
be  online  exploring  those  forums,  or  when  she  knew  the  meeting  time  from  their  previous  communications,  then 
observed  the  team’s  online  collaboration  as  if  in  a real-time  observation.*^  While  reading  the  team  dynamics,  the 
lead  researcher  took  notes  as  if  in  a synchronous  observation,  and  those  notes  were  also  analyzed  together  with  the 
original  transcripts. 

When  the  team  online  collaborative  assignment  was  due,  all  teams’  online  discourse  was  saved  as 
electronic  transcripts.  Those  electronic  transcripts,  which  were  collected  in  a previous  experimental  study  (Zhang, 
2000;  Zhang  & Peck,  2001)  and  now  analyzed  for  an  alternative  perspective  on  and  deeper  understanding  of  the 
original  questions,  could  be  considered  as  secondary  data  (Hinds,  et  al  1997,  Szabo  and  Strang  1997).  Researchers 
(e.  g.  Hinds  et  al,  1997,  Sandelowski  1997,  Szabo  and  Strang  1997,  Thorne  1994)  argue  that  secondary  data  analysis 
can  be  deployed  to  revisit  existing  theories  and/or  generate  new  knowledge.  The  memos  the  lead  researcher  took, 
either  when  observing  team  collaborations  real  time  or  when  coding  and  re -coding,  were  primary  data.  Both  primary 
data  and  secondary  data  were  analyzed  together  in  this  study. 

As  the  research  question  was  particularly  interested  in  comparison,  which  is  a basis  of  discourse  analysis 
(Lemke,  199?),  discourse  analysis  principles  were  very  appropriate  as  an  attempt  to  understand  the  team  interaction 
pattern.  As  researchers  (e.g.  Lemke,  199?)  believe  that  language  written  or  spoken,  as  research  data  should  be 
translated  from  the  activity  in  which  it  originally  functions  to  the  activity  in  which  we  are  analyzing  it.  Also  human 
discourse  is,  by  nature,  highly  contextual,  online  or  in  person,  thus  it  was  critical  for  the  researchers  to  visit  and  re- 
visit the  electronic  transcripts  frequently  through  the  study.  And  the  real  time  observations  played  an  important  role 
in  helping  the  researchers  understand  the  context  of  the  collaborations  when  they  happened  and  hence  enabled  the 
researchers  to  interpret  the  language  and  interactive  rhetoric  as  close  as  we  could. 

Researchers  identity 

The  lead  researcher  was  also  the  primary  researcher  of  the  study  from  which  the  electronic  data  was 
gathered  so  she  understood  the  data  collection  process  and  the  context  in  which  it  was  generated.  During  the 
experimental  study,  where  participants  were  first  introduced  to  the  online  forum  as  a collaborative  learning  tool,  the 
lead  researcher  was  also  working  closely  with  the  students  and  instructor  of  the  course  as  the  technical  supporter  and 
was  actively  involved  in  setting  up  the  forums,  facilitating  re -grouping  and  provide  resolving  technical  issues.  Yet 
most  of  the  contacts  between  the  lead  research  and  the  participants  were  through  email,  the  online  forum  or 
indirectly  through  the  instructor.  Since  the  experimental  study  only  lasted  for  two  weeks,  lead  researcher  did  not 
become  to  know  any  of  the  participants  in  person,  nor  did  she  become  familiar  with  any  of  them.  The  physical  and 
psychological  distances,  which  were  generated  from  the  little  personal  knowledge  and  only  electronic 
communications,  served  both  good  and  bad  for  the  research.  On  one  hand,  the  distances  enabled  the  researchers  to 
be  closely  engaged  with  the  data,  the  online  discourse  transcripts,  while  staying  detained  from  the  participants.  Thus 
the  researchers  were  able  to  study  the  groups  with  little  personal  bias  on  particular  participants.  Yet  on  the  other 
hand,  in  the  data  analysis  process,  the  researchers  often  felt  the  strong  curiosity  to  get  to  know  the  participant  in 
person  as  they  came  out  vividly  as  live  characters  fromtheir  online  discourse.  Also  the  nature  of  the  medium  had 
limited  the  communication  cues  (Daft  & Lengel,  1984;  Fulk  et  al,  1990),  and  the  researchers  felt  some  personal 
contacts  with  the  participants  might  have  helped  interpretation  of  the  discourse,  or  might  have  led  us  to  a more  in- 
depth  understanding  of  the  group  dynamics  as  well  as  the  problem  solving  process. 

We  did  line-by-line  coding  to  generate  initial  ideas  and  categories  and  tired  to  discover  the  relationship 
among  concepts,  or  nodes  as  referred  to  in  NVivo.  The  researcher  first  coded  five  files  line  by  line,  using  only  free 
nodes  to  get  a sense  of  the  data.  As  inducted  from  the  data,  we  noticed  that  the  transcripts  could  be  categorized  into 
four  major  groups  of  codes,  team  process  related,  task  related,  media  related  and  relationship  related,  which  were 
referred  to  as  tree  nodes  in  NVivo.  The  four  categories  were  also  consistent  with  literature  on  teams  in  organizations 


We  think  iiis  observation,  which  happened  to  be  occurring  online.  Having  obtained  participants’  consent,  the  lead 
researcher  still  felt  facing  the  ethical  issue  of ‘lurking”  while  observing  the  online  collaborations. 
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(e.g.  Guzzo  & Dickson,  1996)  and  information  and  communication  technology  (e.g.  Daft  & Lengel,  1984;  Fulk  et  al, 
1990;  Fulk,  1993;  Gunawardena,  1995).  So  we  re-organized  most  of  the  free  nodes  into  the  four  major  themes  as 
tree  nodes  in  NVivo,  task-related,  team  process,  relation  related  and  media -related,  each  with  extended  nodes  under 
it.  In  the  coding  process,  we  used  the  tree  nodes  to  help  categorize  ideas  and  identify  the  team  collaborative  process 
and  quality.  In  the  coding  and  recoding  process,  we  constantly  needed  to  add,  change  or  modify  the  nodes  and  their 
relationships  as  indicated  in  the  tree  nodes  structure,  since  our  interpretation  and  understanding  of  data  was 
developing  and  merging  throughout  the  process. 

As  the  data  indicated  that  the  major  theme  was  task-related,  in  the  late  stage  of  data  analysis  we  started 
from  a key  node,  interactive  collaboration,  which  was  identified  as  the  most  important  node  from  task  related  tree 
node,  and  tried  to  figure  out  the  possible  relationships  among  the  nodes.  The  researchers  also  used  the  concept 
mapping  software,  Inspiration  5 and  the  modeling  feature  embedded  in  Nvivo  to  untangle  and  present  the 
relationship  among  the  key  nodes.  A model  of  team  discourse  was  developed  later  for  each  approach  of  the  online 
collaborative  forums. 

Scholars  (Lincoln  & Guba,  1985;  Moustakas,  1994;  Stake,  1995;  Creswell,  1998)  suggest  the  application  of 
various  techniques  to  ensure  the  quality  of  qualitative  research.  We  invited  colleagues  and  peer  researchers  for 
review  and  debriefing  throughout  the  research.  In  addition,  the  first  author  met  with  a colleague  twice  a week  in  the 
data  analysis  process  to  discuss  and  reflect  upon  the  research  process  and  to  ask  for  an  outsider’s  opinion  on  the 
interpretation  and  coding  of  the  data.  Also  the  researchers  were  constantly  engaged  in  academic  discourse  with  peers 
for  alternative  interpretation  and  representation  of  the  data  in  lively  discussions.  The  scholarly  communications 
between  the  researchers  and  with  peer  researchers  helped  to  validate  and  establish  trustworthiness  of  the  research. 
Also  with  the  concept  mapping  and  the  QDA  software,  the  researchers  were  able  to  constantly  refine  the  coding  and 
analysis  through  the  research. 

Findings 

A model  was  developed  for  each  type  of  collaboration  forum  to  represent  the  major  findings  of  this  study.  ( 
figure  1,  figure  2 inserted  here.)  As  indicated  in  the  model,  there  are  two  major  differences  between  the  online 
discourses  in  the  two  types  of  forum,  peer-controlled  and  structured  and  moderated.  One  was  the  presence  of 
strategy  that  guide  through  the  four  major  themes  (i.e.  task-related,  interaction  pattern,  relationship-related  and 
media-related).  The  other  was  the  strength  of  interactive  collaboration. 

Discourse  occurred  in  the  structured  and  moderated  forums  started  with  a strategy,  in  the  form  of  a 
proposal  for  or  discussion  of  a strategy,  which  then  generated  shared  agreement  on  how  to  process  the  problem 
solving  assignment  as  a team.  Such  strategies  often  included  two  aspects,  one  about  team  processing,  (which  was 
reported  in  the  models  as  interaction  pattern,  because  the  researchers  decided  that  the  two-week  discourse  was  too 
short  and  not  rich  enough  to  determine  the  team  processing  pattern)  and  another  for  the  team  task.  Many  teams 
started  with  a clarification  of  membership,  since  they  were  re-forming  a self-selected  group,  and  started  with 
questions,  suggestions  or  statements  on  how  to  collaborate  on  this  assignment. 

Interactive  collaboration  as  merged  from  the  data  was  conducted  through  simple  agreement,  agreement 
with  elaboration,  question  and  answer,  and  disagreement.  In  forms  of  information  seeking,  asking/providing 
clarification,  challenging  other’s  ideas  and  further  questions  that  led  to  active  learning  beyond  the  scope  of  the 
original  task.  Through  interactive  collaboration,  participants  experienced  self-reflection  and  critique,  made 
connections  and  comparisons  with  previous  learning,  and  searched  and  utilized  other  resources.  Such  interactive 
collaborations  did  happen  in  both  types  of  forum,  yet  it  was  evident  that  in  the  structured  and  moderated  forums, 
groups  had  more  intense  and  orderly  interactive  collaborations  and  did  achieve  active  learning  through  the 
collaborative  inquiries,  while  in  peer-controlled  forums  groups  often  stayed  with  individual  and/or  fragmental 
reasoning  and  many  left  the  forums  without  any  interaction  or  collaborations.  The  one-way,  individual  reasoning, 
together  with  simple  agreement  without  reflection  or  elaboration  on  it,  fail  to  lead  the  teams  experience  real 
collaborative  learning  and  thus  lost  opportunities  for  shared  active  learning.  Simple  agreement,  which  happened 
very  often  in  the  peer-controlled  forums,  indicated  only  shallow,  superficial,  interactive  collaboration,  if  we 
considered  it  as  collaborations  at  all. 

Starting  with  a strategy,  or  the  efforts  to  build  some  strategy  in  the  early  stage  of  the  task  also  helped  the 
teams  to  establish  a mutually  agreed  interaction  pattern  with  people  taking  the  leadership  role  and  organize  the 
collaboration  in  a well-accepted  fashion.  Such  efforts  were  evident  in  the  structured  and  moderated  forums  and  the 
interactions  thus  were  more  clearly  organized  and  processed.  Yet  in  the  peer-controlled  forums,  teams  either  did  not 
interact  at  all  or  did  it  in  a more  or  less  chaos  or  similar  manner. 
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Closely  related  to  the  interaction  pattern  and  task-related  themes  were  the  relationship -related  issues  as 
indicated  in  the  data.  Teams  that  showed  mutual  respect  and  provide  support  and  encouragement  to  one  another 
were  also  showing  interaction  patterns  in  an  orderly  manner,  and  had  more  frequent  and  in-depth  interactions  with 
one  another.  In  peer-controlled  forums,  teams  either  went  off-topic  and  spent  a lot  of  time  in  the  online  forums  on 
relationship -related  issues  or  did  not  show  any  evidence  at  all  on  team  member  relationships. 

Online  forums  at  the  time  the  data  was  collected  were,  and  probably  still  are,  a new  medium  for  learning  as 
well  as  communication.  Part  of  the  task-related  and  interaction  patterns,  as  shown  in  the  data,  was  about  the  team 
collaboration  media.  Teams  chose  different  media  to  meet  their  needs  for  collaborations  at  different  stage  of  team 
problem  solving.  Those  choices  varied  from  team  to  team.  Yet  it  was  clear  to  the  researchers  that  the  choice  of 
medium  that  matched  their  needs  was  critical  for  the  successful  of  team  collaboration  process  and  relationship - 
related  issues. 


160 


649 


Courseware  & Copyright:  Who’s  Rights  Are  Right? 


Ke  Zhang 
Ali  Carr-Chellman 
Penn  State  University 


Short  Description 

This  session  addresses  the  difficult  issues  of  balancing  faculty  and  university  rights  and  responsibilities 
regarding  courseware  and  courseware  modules  developed  as  both  commissioned  and  non-commissioned  works. 

The  ways  in  which  this  debate  affect  those  who  are  engaged  in  a)  building  their  own  online  courses  for  their 
universities;  b)  courseware  development  as  consulting  practice,  and  c)  work  for  universities  as  faculty  development 
web  development  experts  will  be  addressed. 

Introduction 

The  application  of  the  web  to  higher  education  has  brought  with  it  many  new  and,  at  moments,  disturbing 
systemic  impacts  that  significantly  effect  the  work  of  instructional  design  faculty  and  instructional  designers  in 
higher  education.  On-line  learning  is  a considerably  different  approach  to  higher  education  (Daniel,  1996; 

Gubemick  & Ebeling,  1997;  Hall,  1995;  Chea,  2000)  and  creates  important  new  challenges  including  what  counts  as 
digital  courseware,  how  is  courseware  different  from  textbooks,  who  owns  courses  that  are  designed  for  online 
learning,  under  what  circumstances,  and  who  controls  those  courses  into  the  future?  Most  of  these  questions  have 
been  dealt  with  substantially  through  a number  of  publications  of  policies  for  various  institutions  (University  of 
Wisconsin,  1997;  University  of  Pennsylvania,  1997;  University  of  Texas  System,  1998;  University  of  Illinois,  1999; 
University  of  California,  1999;  University  System  of  Maryland,  2000;  Penn  State,  2000)  and  through  a number  of 
think  tank  reports  (CETUS,  1997;  AAU,  2001;  Twigg,  2000;  AAUP).  While  a good  deal  of  work  has  already  been 
accomplished  examining  courseware  and  copyright  issues,  none  has  linked  these  questions  and  policy  discussions 
specifically  to  the  work  of  instructional  designers. 

Thus  this  work  is  an  extension  of  the  work  begun  by  many  policy  makers,  administrators,  lawyers,  faculty 
members,  and  intellectual  property  scholars.  Specifically,  this  paper  will  look  at  the  potential  impacts  of  a variety  of 
copyright  issues  on  the  work  that  instructional  designers  do — particularly  in  the  online  educational  development 
enterprise. 

There  are  two  primary  roles  that  instructional  designers  hold  in  the  university,  as  service  professionals  and 
as  faculty.  As  instructional  design  professionals,  the  impact  of  policies,  which  may  serve  to  encourage  or 
discourage  faculty  participation  in  the  creation  of  online  courseware  may  have  broad  and  sweeping  impacts.  For  the 
first  time  since  the  inception  of  the  field,  courseware  has  brought  the  field  of  instructional  design  into  the  university 
administrations'  radar  screens.  There  is  a sudden  realization  that  in  order  to  effectively  move  the  traditional  face-to- 
face  course,  which  always  required  a human  professor  to  make  the  pedagogical  connection  to  extant  materials  (Hiltz 
& Wellman,  1997;  Twigg,  2000)  into  the  digital  format  requires  these  people  called  instructional  designers.  This 
has  been  a serious  boon  for  our  field,  although  at  times  there  is  a perception  of  our  academic  programs  as  being 
primarily  service  oriented  rather  than  content  oriented  or  research  based—  a common  problem  for  our  field 
addressed  by  Heinich  (1984).  If  policies  are  formulated  which  intimidate  or  worry  faculty,  there  is  likely  to  be  a 
chilling  effect,  and  a lack  of  interest  in  pursuing  the  creation  of  courseware  on  the  part  of  faculty.  Without  the 
faculty,  who  are  essentially  the  SME's  or  content  providers,  the  instructional  designers  would  certainly  have  a 
difficult  time  putting  courses  online.  And  some  would  question  the  ethicality  of  pursuing  online  learning  in  the 
absence  of  generalized  faculty  support. 

In  addition,  instructional  design  faculty  members  are  very  interested  in  many  questions  associated  with  the 
new  online  higher  education  enterprise.  It  is  an  opportunity  for  consulting  as  well  as  research  and  scholarship.  Here 
again  if  policies  constructed  by  the  institutions  are  narrow  in  their  understanding  of  the  work  that  instructional 
design  faculty  do  in  this  regard,  they  could  find  themselves  asking  permission  for  each  project  from  upper 
administration  and  constantly  defending  their  work  in  an  effort  to  show  it  is  not  a conflict  of  interest  or  infringement 
of  narrowly  construed  intellectual  property,  courseware  or  copyright  policies.  This  paper  considers  the 
specifications  of  three  policies  being  considered  at  three  institutions:  Wisconsin,  Madison,  University  of  Texas,  and 
Penn  State  as  examples  and  as  a contextualized  basis  for  further  discussion  of  the  potential  impacts  of  such  policies 
on  our  work  as  instructional  designers  and  instructional  design  faculty. 


161 


;650 


Instructional  designers  have  been  actively  involved  in  design,  development  and  delivery  of  online 
education,  when  building  their  own  online  courses  for  their  universities;  developing  courseware  as  consulting 
practice,  and/or  serving  as  subject  matter  experts.  Thus  university  courseware  policies,  both  directly  and  indirectly, 
have  significant  impacts  on  instructional  designers  working  in  higher  education.  Considering  the  varied  roles  that 
instructional  designers  play  in  a university,  we  will  be  using  university  personnel  in  this  paper  to  include  faculty 
members,  staff,  and  professionals  engaged  in  courseware  development.  First  we  will  examine  some  of  the  concerns 
that  are  raised  initially  as  one  considers  courseware  copyright  policies,  then  we  will  turn  our  attention  to  more 
deeply  understanding  three  contexts  in  which  courseware  copyright  policies  are  either  being  considered  or  have 
recently  been  passed  into  campus  legislation.  Finally,  we  will  reconsider  the  concerns  that  evolve  from  that  deeper 
understanding  of  these  new  policies. 

CONCERNS 

Academic  Freedom 

Perhaps  one  of  the  most  prevalent  concerns  voiced  by  faculty  members  in  many  subject  areas  is  the  extent 
to  which  the  rules  are  somehow  changing  in  this  new  technological  world.  Faculty  are  concerned  that  if  courseware 
policies  are  passed,  it  may  lead  to  infringements  on  academic  freedom.  These  concerns  actually  take  the  form  of 
several  specific  worries.  First,  faculty  members  are  very  concerned  about  their  mobility  as  scholars,  designers,  and 
creators  if  university  claims  strong  ownership  and  rights  on  courseware.  If  a faculty  member  leaves  their  home’ 
institution  for  another  position,  but  their  courses  were  online,  to  what  extent  will  they  be  able  to  take  their  courses 
with  them  to  their  new  place  of  employment?  While  it  is  clear  that  a faculty  number  takes  their  knowledge  with 
them,  and  also  that  they  would  not  be  allowed  to  take  the  specific  software  solution  designed  for  a given  university 
with  them,  the  questions  of  the  gray  areas  are  much  more  confusing.  For  example,  let’s  suppose  that  a faculty 
member  designs  a course,  an  online  course,  with  substantial  support  from  their  institution.  They  work  with 
instructional  designers,  graphic  artists,  and  software  experts  and  create  a really  wonderful  course  that  utilizes  a very 
innovative  method  for  delivering  the  instruction.  The  given  method  is  the  original  idea  of  the  faculty  member,  not 
the  collaborative  outcome  of  the  work  with  instructional  designers.  The  home  institution  could,  under  certain 
circumstances  claim  ownership  over  not  only  the  specific  course  solution  that  was  built  utilizing  their  substantial 
resources,  but  also  the  innovation  of  the  method  itself,  something  that  the  faculty  member  originated  and  certainly 
planned  to  replicate  perhaps  in  another  medium  at  their  new  position. 

Control 


Within  the  academic  freedom  area  of  concerns  is  another  important  general  category — control.  To  what 
extent  will  the  faculty  member  control  the  frequency,  updating,  and  actual  implementation  of  the  course  they  have 
built.  Here  again,  given  certain  policies,  there  is  significant  danger  that  courses  will  be  under  the  control  of 
administrators  who  may  have  more  interest  in  profit  than  in  fidelity  to  a given  subject  area.  Faculty  worry  that  their 
work  may  be  changed  without  their  approval,  utilized  without  proper  attribution,  or  offered  far  into  the  future  under 
the  direction  of  instructors  without  a significant  commitment  to  the  field  being  taught.  Naturally,  t is  frightening  to 
consider  the  possible  replacement  of  professors  with  courseware  without  recognizing  the  creators,  to  use,  modify 
transfer  or  commercialize  courseware  without  consultation  with  or  permission  from  the  original  faculty  authors  ’ 

(Farrell,  2001;  Twigg,  2000;  AAUP,  2001). 

Defining  Roles 

Clearly  one  of  the  fuzziest  areas  to  many  faculty  are  the  understandings  of  the  roles  of  faculty  and  support 
staff  in  the  creation  of  online  learning  resources.  In  general,  the  administration  at  most  institutions  focus  on  the 
commitment  of  the  faculty  member  to  the  institution,  suggesting  that  it  is  a normal  part  of  their  job  to  create  online 
learning  materials  and  that  they  should  not  expect  either  ownership  or  additional  pay,  though  they  should  be  given 
the  tools  to  enable  the  creation  process.  On  the  other  hand,  based  on  a history  of  textbooks  and  prior  use  of  course 
materials  m workshops,  consulting,  mini-courses,  classes  on  tape  etc,  the  faculty  member  understands  the  course  as 
a basic  unit  owned  by  them.  It  is  their  content  knowledge,  their  creation  and  has  typically  been  their  property  in  the 
past.  While  many  institutions  have  long  standing  policies  indicating  that  the  syllabus  for  a given  course  is  owned  by 
the  university,  this  is  so  rarely  enforced  in  terms  of  mobile  faculty  or  consulting  usage  that  most  faculty  are  unaware 
of  such  policies  where  they  exist.  In  essence,  many  faculty  see  that  the  institution  is  there  to  serve  them  in  their 
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quest  to  advance  the  knowledge  in  their  given  field,  while  the  administrators  see  the  faculty  as  there  to  serve  the 
advancement  of  their  home  institution.  The  way  that  one  interprets  this  conflict  typically  speaks  clearly  to  the  sorts 
of  policies  that  are  likely  to  result.  Faculty  members  typically  want  very  faculty-friendly  policies,  while 
administrators  are  looking  for  protection  for  the  institution  fro  m competition  in  this  new  cyber-education  world. 

How  to  define  substantial? 

Related  to  the  understanding  of  roles  is  the  subsequent  understanding  of  the  term  “substantial”.  This  term 
is  used  extensively  throughout  the  copyright  literature  and  the  resulting  copyright  policies.  The  use  of  substantial 
university  facilities  and/or  personnel  is  often  one  of  the  most  important  or  even  determining  factors  in  sorting  out 
university  ownership  of  courseware.  Yet  the  concept  of  substantial  is  far  from  clearly  defined.  Partially  because  of 
the  distinctions  in  roles  discussed  above,  the  concept  of  substantial  resources  is  extraordinarily  open  to 
interpretation.  The  State  University  of  Pennsylvania  system,  for  example  defines  it  as  at  least  $40,000  per  project. 
This  seems  like  perhaps  the  clearest  definition  of  substantial  that  we  have  found  in  the  literature.  In  general  the  term 
is  defined  in  much  more  nebulous  ways  such  as  the  “use”  of  instructional  designers,  graphic  artists,  working  with  a 
particular  support  arm  of  the  university  or  the  like.  How  much  or  how  little  is  rarely  as  clearly  defined  as  it  has  been 
at  U Penn.  However,  even  with  this  clear  financial  figure,  still  it  is  unclear  precisely  how  that  number  is  calculated. 
What  “counts”  and  what  doesn’t  is  not  as  clear  as  everyone  would  like.  The  U Wisconsin  has  a relatively  detailed 
definition  of  “substantial”  (See  Table  1),  yet  the  “extraordinary”  use  of  computing  resources  remains  a nebulous 
term  and  therefore  unclear  in  terms  of  how  it  will  be  measured. 

Given  the  varying  roles  the  institutions  and  individual  professors  may  play  in  the  creation  and  development 
of  courseware,  it  is  not  likely  that  a single  principle  of  law  can  clearly  allocate  copyright  interests  in  all  cases 
(AAUP  1999).  Generally  the  following  instruments  are  deployed  to  address  the  courseware  copyright  issues: 
individual  negotiations  and  signed  agreements  between  individual  university  personnel  and  institutions;  collective 
bargaining  agreements  between  the  university  and  personnel;  and  university  copyright  policies  (Morgan,  2000). 
Many  universities  (e.g..  University  of  Texas,  University  of  Wisconsin,  and  Penn  State  University)  have  made  efforts 
to  develop  a courseware  policy  to  provide  a framework  on  how  to  address  courseware  copyright.  Generally  three 
models  of  ownership  are  applicable,  i.e.  university  ownership,  university  personnel  ownership  and  Joint  ownership. 
The  most  common  factors  used  to  determine  ownership  include,  but  are  not  limited  to  the  following:  (1)  the 
conditions  under  which  the  courseware  is  developed;  (2)  the  scholarly  nature  of  the  product;  (3)  the  commercial 
character  of  the  work;  and  (4)  the  scope  of  employment. 

Characteristics  of  AAU  (1999)  Model 

The  current  debate  on  whether  to  treat  courseware  as  traditional  textbook  or  as  patentable  innovations,  as 
AAU  (American  Association  of  Universities)  interprets  (AAU,  2001),  focuses  only  on  the  product,  and  AAU  calls 
for  a shift  of  attention  from  the  product  to  the  process  when  addressing  courseware  related  copyright  issues.  The 
AAU  model  primarily  represents  the  interests  of  the  institution,  or  the  administrators  within  the  institution.  AAU 
(1999)  argues  that  courseware  is  a ’’collaborative  creation"  at  a university.  Based  on  this  assumption,  AAU  further 
proposes  that  "the  university  should  own  the  intellectual  property  that  is  created  at  the  university  by  faculty,  research 
staff,  and  scientists  and  with  substantial  aid  of  its  facilities  or  its  financial  supports.”  (AAU  1999). 

Being  very  aware  of  how  difficult  it  could  be  due  to  the  complexity  of  new  media  products,  AAU  still 
proposes  that  generally  applicable  formulas  should  be  available  to  allocate  the  sharing  of  returns  on  digital  media 
products  among  individual  faculty,  departments,  schools  and  the  university. 

For  the  products  that  do  not  rely  on  the  university  resources  and  not  for  use  at  the  university  (e.g.  in 
courses),  the  university  may  not  claim  interests  in  them,  yet  the  definition  of  “reliance  on  university  resources” 
remains  less  than  completely  defined.  Here  is  raised  a particularly  interesting  set  of  questions.  While  it  may  be  clear 
to  many  faculty  that  the  use  of  $40,000  or  more  of  university  resources  Justifies  university  ownership  of  courseware, 
if  the  faculty  member  does  the  work  on  his  or  her  own  time,  then  it  would  seem  intuitively  obvious  that  the 
university  ought  not  to  interfere  with  the  use  of  these  materials.  This  is  particularly  the  case  if  the  materials  are  not 
in  substantial  overlap  with  the  faculty  members’  duties  at  the  university.  However,  university  administrators  are 
concerned  about  increases  in  competition  for  almost  any  online  product  that  a faculty  member  at  their  institution 
may  produce  for  a mass  market.  From  the  institution’s  perspective,  if  people  can  gain  access  to  famous  faculty 
members  online  cheaply,  or  potentially  cheaper  than  through  their  own  university  tuition  and  fees,  then  what  could 
compel  them  to  spend  more?  And  then  what  protects  the  university  under  those  circumstances  from  a substantial 
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loss  of  income  because  the  faculty  member  is  essentially  competing  against  his  or  her  own  institution.  We  will 
revisit  this  question  in  more  depth  during  the  final  section  of  this  paper. 

Portability  of  intellectual  property 

AAU  questions  if  courseware  should  be  treated  differently  fro  m patented  discoveries,  which  belong  to  the 
university  and  are  not  portable  with  the  faculty  who  invent  them.  Here  we  sense  the  blurred  distinction  between 
teaching  and  research,  knowledge  and  products.  The  patent  for  an  invention  remains  with  the  university  who  is 
granted  ownership  in  the  traditional  custom,  yet  courseware  is  not  a simple  product,  it  is  more  likely  to  be  a 
representation  of  knowledge  and  scholarly  thinking,  which  is  always  expected  to  be  continuously  constructed 
through  the  journey  of  delivery  as  well  as  constant  updates.  It  is  potentially  dangerous  to  view  courseware  as  a one- 
time invention  and  thus  to  limit  the  portability  of  it.  Also  the  importability,  if  defined  in  university  courseware 
policy  as  suggested  by  AAU,  will  cause  cautiousness  among  faculty  members  about  their  academic  freedom,  and  is 
quite  likely  to  reduce  their  motivation  to  design,  develop,  maintain,  or  update  courseware. 

Characteristics  of  AAUP  Model 

If  the  AAU  model  represents  the  institutional  perspective,  then  the  AAUP  (American  Association  of 
University  Professors)  model  represents  the  faculty  interests.  AAUP  (2001)  proposes  an  informed  allocation  of 
rights  between  faculty  and  universities  in  forms  of  ownership,  control,  use  and  compensation,  AAUP  (2001) 
classifies  the  general  projects  into  three  categories:  works  that  are  properly  considered  “made  for  hire,*’  negotiated 
contractual  transfer,  and  joint  works  as  defined  by  the  Copyright  Act.  AAUP  (2001)  further  stresses  that  regardless 
of  the  ownership  of  the  courseware,  faculty  members  should  be  given  rights  in  connection  with  future  use  of 
courseware  that  they  have  been  part  of  in  the  creation  and  development.  Not  only  through  compensation,  faculty 
should  be  given  the  right  of  “first  refusal”  in  making  new  versions  or  at  least  of  the  right  to  be  consulted  in  good 
faith  on  re-use  and  revisions  (AAUP,  2001).  Those  points  certainly  voice  faculty  members’  concerns  and  are 
attempts  to  protect  faculty  members’  rights  as  scholars,  researchers,  authors,  and  creators  while  promoting 
development  of  courseware.  For  instructional  designers  and  faculty  members  this  is  really  important,  particularly  as 
regards  quality  control  and  academic  freedom.  The  AAUP  plan,  while  it  is  not  as  faculty-friendly  as  some  models, 
certainly  represents  the  interests  of  faculty  and  tends  to  encourage  faculty  to  engage  in  the  creation  of  online 
learning  materials. 

Three  Institutional  Policies 

We  now  turn  our  attention  from  models  to  specific  policies.  The  policies  of  three  institutions  (University 
of  Texas,  Penn  State,  and  the  University  of  Wisconsin,  We  chose  these  three  because  they  have  all  been  recent 
negotiations  and  because  we  believe  that  they  represent  significantly  different  types  of  institutions.  While  all  are 
relatively  large  public  institutions,  politically  they  are  quite  diverse  in  their  histories  and  past  policies. 

University  of  Texas’  courseware  policy  has  been  recognized  as  the  model  for  electronic  course  materials 
ownership  (Farrell,  2001).  And  the  American  Association  of  University  Professors  uses  University  of  Texas’ 
language  as  a template  for  its  own  policies  and  guidelines  on  the  matter  (AAUP,  2001).  Thus  the  UT  policy  is 
perhaps  the  benchmark  that  other  universities  are  typically  considering  as  they  create  their  own  policies.  Penn  State 
after  some  lengthy  debate  in  the  faculty  senate  has  just  passed  a courseware  policy  recently.  And  we  think  it  is  very 
telling  to  share  their  view  and  struggle  with  this  issue.  University  of  Wisconsin  is  historically  a very  politically 
progressive  institution,  so  it  would  be  interesting  to  see  how  they  dealt  with  this  in  comparison  to  say  PSU  which  is 
much  more  corporate  in  nature  and  does  not  have  a history  of  progressive  liberal  policies. 


In  the  following  table  we  try  to  summarize  the  current  university  courseware  policy  at  Penn  State 
University,  University  of  Texas  System,  and  University  of  Wisconsin  System. 


TABLE  1 


Ownership  Institution  | Penn  State  University 


University  of  Texas  System  University  of  Wisconsin 
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fPSU) 

(UTS) 

Svstem  (UWS) 

The  university  initiates  the 

Any  of  the  following: 

Condition  1:  with 

development  as  a normal 

“substantial”  institutional 

duty  or  special  project  of  a 

Products  are  created  within 

support  or  involvement 

faculty  member  for  which 

the  scope  of  employment 

extra  compensation  is 

In  such  cases  the  university 

provided 

Using  UTS  time,  facility  or 

system  may  assert 

state  financial  support 

ownership  or  other  property 

(applicable  to  both 

interests,  which  should  be 

commissioned  courseware 

Commissioned  by  the 

addressed  through  specific 

and  commissioned 

university  of  a component 

agreements  with  the  authors 

courseware  modules) 

institution,  either  as  a 

and  producers  of  the 

contract  or  work  for  hire 

materials  (University  of 

Wisconsin  System,  1997, 

c 

o 

Research  supported  by 
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federal  funds  or  third  party 

c/3 
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sponsorship 

Condition  2: 

o 

As  assigned  duty  of 
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s 

6 

employment 

c/3 

(L) 
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13 

Condition  3:  pursuant  to  a 

‘S 
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work-for-hire  agreement 
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Definition  of  substantial 


bO 


Generally,  sale  or  use  of 
courseware/  courseware 
modules  in  areas  that 
substantially  compete  (how 
is  it  defined  remains 
unclear)  with  PSU  is  not 
allowed  without  prior 
approval. 

For  commissioned 
courseware: 

Written  agreement  between 
university  personnel  and  the 
university  prior  to  the 
project  starts  will  establish 
the  extent  to  which  the 
materials  may  be  used  in 
derivative  works  published 
outside  PSU  and  will  also 
formalize  the  relationship 
with  authors  outside  PSU,  if 
any,  and  the  procedure  for 
the  use  of  existing 
materials. 

If  distributed  outside  of 
PSU,  authors  will  receive 
50%  of  royalty  or  other 
consideration  received  by 
the  university 

For  commissioned 
courseware  modules: 
University  give  proper 
credit  to  author(s). 


For  Condition  1 : Subject  to 
a written  agreement 
between  U personnel  and 
chief  administrative  officer 
of  the  institution  involved 
to  determine  copyright  and 
ownership 

For  condition  2(assigned 
duty): 

U owns  all  rights,  including 
copyrights  and  royalty  and 
fees,  unless  a contrary 
agreement  reached  prior  to 
the  beginning  of  the  project 
specifies  otherwise. 

For  condition  3 (work  for 
hire): 

U receive  all  rights, 
including  royalties  and  fees, 
and  a written  work-for-hire 
agreement  shall  be  executed 

Fair  payment  shall  be  made 
to  the  author  of  the 
copyrightable  instructional 
materials  under  the  work- 
for-hire  agreement 


Released  time  from 
regularly  assigned  duties; 

direct  investment  of  funds 
or  staff; 

purchase  of  special 
equipments; 

use  of  multimedia 
production  personnel  and 
facilities; 

extraordinary  use  of 
computing  resources 

(University  of  Wisconsin 
System,  1997,  footnote  3) 


er|c 


5 


166 


University  personnel 

General  rule:  author  is  the 

initiated,  noncommissioned 

owner 

courseware  and  courseware 

modules  which  do  not 

University  of  Texas  System 

compete  with  any  PSU 

intellectual  property  policy 

programs 

permits  faculty  ownership 

of  scholarly,  artistic. 

Yet:  the  use  of  university 

literacy,  musical  and 

resources  to  make 

educational  materials 

university-personnel-owned 

within  the  author’s  field  of 

courseware  modules  should 

expertise. 

use  care  to  ensure  that 

applicable  copyright  laws 

Product  unrelated  to 

and  policies  are  followed. 

university  personnel’s  job 

responsibilities  and  no  more 

No  use  of  university 

than  incidental  use  of  UTS 

resources,  equipment. 

resources 

support  staff,  materials  is 

c/3 

allowed 

Invention  that  has  been 

G 

O 

released  to  the  inventor 

Uh 

D 

G 

> 

G 

O 

Intellectual  property  is 

> 

O 

o 

4-» 

embodied  in  a professional. 

13 

G 

D 

G 

faculty,  researcher  or 

G 

O 

G 

o. 

o 

student-authored  scholarly. 

13 

artistic,  musical,  literary  or 

CU| 

p 

u 

architectural  work  in  the 

1— 1 

author’s  field  of  expertise 
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No  or  minimal  University 
support  or  involvement 

“Minimal”,  as  used  in  this 
policy,  includes  the  use  of 
university  lab  or  equipment 
but  does  not  include  release 
time  from  regularly 
assigned  duties.  (University 
of  Wisconsin  System,  1997, 
footnote  2) 

it  further  defines  “author”, 
who  has  solely  ownership 
for  courseware  produced 
under  the  above  condition, 
as  “creators  of  instructional 
materials... may  include 
faculty,  staff  or  students 
who,  as  a regular  part  of  the 
instructional  program, 
become  participants  in  the 
creation  of  copyrightable 
materials”  (University  of 
Wisconsin  System,  1997, 
footnote  4). 
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Control  and  use  of 

In  case  of  educational 

personnel  initiated 

materials  that  involves 

courseware  modules  at  no 

significant  institutional 

or  minimum  cost  remain 

resources,  UTS  retains 

with  the  author(s) 

rights,  for  example,  to  use 
the  work  and  recover  its 

University  claims  royalty- 
free  nonexclusive  right  to 
use  the  personnel-initiated 

investment. 

noncommissioned 

University  personnel 

courseware  in  the  university 

owners  must  share  benefits 

program 

with  the  UTS  from 
commercializing  a UTS 

Use  of  university- 

invention  released  to 

personnel-owned 

him/her;  or  if  the  work 

courseware  in  the  university 

embodying  the  intellectual 

educational  programs  will 

property  required 

only  be  allowed  under  the 

significant  resources 

university’s  supervision. 

contribution  from  the  UTS 
or  s component  to  create  or 

Courseware  provided  at  no 

develop  the  intellectual 

or  minimum  cost  can  be 

property  (parties  should 

supervised  at  the 

execute  an  agreement 

department  level. 

courseware  modules 
provided  at  minimum  cost 

regarding  sharing  before  the 
project  starts). 

CA 

can  be  supervised  at  the 

.2? 

'u 

department  level 
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Sale  or  use  of  university- 
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Multimedia  courseware  and 
distance  learning  materials 
are  very  likely  to  be  jointly 
developed  and  thus 
protected  by  joint 
ownership. 

When  the  university 
personnel’s  expression  is 
added  to  a faculty 
member’s  contribution,  it  is 
then  considered  jointly- 
authored  work,  and  owned 
by  the  university  and 
faculty  member. 

When  substantial  UTS 
resources  are  used  in  the 
development  of  a 
courseware,  the  university 
may  claim  joint  ownership. 


Written  agreements  among 
parties  involved  prior  to  the 
project  starts  is 
recommended  to  clarify  the 
relationships  and  rights. 


External  /extramural 
support 


We  only  find  policy 
regarding  sponsored 
inventions(PSU,  2001)  but 
nothing  about  sponsored 
courseware  policy 


Agreement  with  the 
extramural  sponsor  shall  be 
considered  in  determining 
the  copyright  and 
ownership  rights  of  the 
parties. 


Claims  on  /interest  in 
traditional  course- 
related  materials 


Not  specifically  mentioned 
in  current  courseware 
policy  yet  no  evidence  that 
the  university  would  claim 
for  ownership  for 
traditional  course  materials 


Not  specifically  mentbned 
in  current  courseware 
policy  yet  no  evidence  that 
the  university  would  claim 
for  ownership  for 
traditional  course  materials 


Does  not  assert  interests  in 
materials  resulted  in 
traditional  teaching, 
research  and  scholarly 
activities 
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ERIC 


Conflict  of 

interest/commitment  is 
defined  as:  for  personal 
financial  gain 


Written  agreement  among 
parties  prior  to  the 
beginning  of  the  project  is 
recommended 


Written  agreement  among 
parties  prior  to  the 
beginning  of  a project  is 
highly  recommended  to 
clarify  rights  and 
responsibilities  of  each 
party  involved. 

responsible  individuals  are 
responsible  for  copyright- 
related  clearance,  waivers, 
agreement,  etc. 

The  policy  encourages 
internal  use  and  sharing 
based  on  agreements 
developed  under  this 
policy,  and  charges  for 
internal  use  should  be 
limited  to  actual,  direct 
cost. 

Release  to  external 
distribution  should  be 
negotiated  between  the 
institution  and  the  author. 

Revision  or  withdrawal: 
Shall  not  be  altered  or 
revised  without 
consultation  with  the  author 

When  contrary  to 
recommendation  of  the 
author,  use,  or  authorization 
for  use  by  others  shall  not 
be  advertised  or  presented 
as  the  work  of  that  author, 
except  appropriate 
acknowledgement  of  the 
author’s  original  production 

Author  shall  be  offered 
opportunity  for  revision 

When  considered  for 
different  uses  than  the 
original  ones,  author  shall 
be  consulted. 
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University  of  Texas  System  Courseware  Policy 

University  of  Texas  System  (UTS)  standards  automatically  grant  ownership  of  scholarly,  electronic 
learning  materials  to  authors  who  create  them,  reinforcing  the  traditional  presumption  for  scholarly  writings,  books 
and  teaching  materials  in  universities. 

As  Georgia  Harper,  section  manager  for  intellectual  property,  office  of  general  counsel  UTS  points  out 

(Farrell,  2001),  “Faculty  certainly  own  their  course  material  in  a traditional  environment,  and  so  to  change  it to 

take  away  something  they  and  we  believe  they  own  — would  have  precipitated  an  immense  struggle.**  And  the  merit 
of  UTS  policy  is  that  it  recognizes  the  presumption  of  faculty  ownership  of  work  they  create,  and  faculty  members 
feel  secured  (Farrell,  2001).  To  keep  a balance  between  the  faculty  members  and  the  university,  UTS  policy 
recognizes  faculty  members  as  owner  of  software  and  other  materia  Is  they  create,  and  allows  the  university  to 
reserve  the  right  to  recover  cost  if  significant  university  resources  have  been  used  in  the  development  process.  Now, 
as  a general  rule,  teaching  materials  created  by  faculty  members,  regardless  of  the  medium,  leave  with  the  faculty 
creators  if  they  leave  the  university  (Farrell,  2001). 

According  to  UTS  policy,  university  personnel  ownership  is  granted  when  courseware  is  unrelated  to 
employee’s  job  responsibilities  AND  is  developed  with  no  more  than  incidental  use  of  system  resources. 
Professional-,  faculty-,  researcher-,  or  student-authored  scholarly,  educational,  artistic,  musical,  literacy  or 
architectural  work  in  the  author’s  field  of  expertise,  unless  is  for  hire,  or  required  to,  or  commissioned  by  the  UT 
system,  will  be  granted  university  personnel  ownership  as  well,  in  case  of  substantial  use  of  institutional  resources, 
the  university  retains  rights  to  use  the  work  and  recover  its  investment,  from  royalty,  for  example,  and  may  claim 
joint  ownership. 

The  UTS  policy  also  points  out  that  multimedia  courseware  products  and  distance  learning  materials,  since 
are  more  than  often  jointly  developed,  are  very  likely  to  be  jointly  owned  by  the  university  personnel  and  the 
university. 

Penn  State  University  Courseware  Policy 

After  some  lengthy  debate,  the  faculty  senate  at  PSU  passed  an  administration-friendly  policy  which  was 
promptly  accepted  without  revision  by  the  university  president.  The  senate  debate  did  raise  many  concerns  for  some 
faculty,  but  the  bulk  of  faculty  felt  they  would  not  be  affected  by  the  new  policy  and  were  willing  to  work  with  the 
administration  to  protect  the  institution  from  outside  competition. 

As  the  chair  of  the  courseware  copyright  committee  stated,  one  of  the  purposes  of  the  efforts  is  to  protect 
Penn  State,  a large  traditional  higher  education  institution  from  the  commercial  competitors  such  as  the  University 
of  Phoenix. 

The  policy  basically  allows  that  the  university  has  copyright  ownership  on  online  courses  whenever  the 
university  initiates  the  development  as  a normal  duty  or  special  project  of  a faculty  member  for  which  extra 
compensation  is  provided. 

The  policy  claims  that  for  non-commissioned  courseware,  the  university  makes  no  claim  to  copyright 
ownership,  however,  it  DOES  claim  the  royalty-free,  non-exclusive  right  to  use  the  courseware  in  any  university 
programs.  Faculty  have  no  right  of  first  refusal.  This  part  of  the  policy  certainly  does  not  incline  faculty  members  to 
try  to  create  non-commissioned  online  courses  particularly  in  combination  with  the  conflict  of  commitment  clause 
discussed  below. 

In  cases  that  the  courseware  is  a full  replacement  of  a university  educational  program,  the  use  of  such 
software  is  allowed  only  under  university  supervision  and  any  courseware  module  that  imposes  a cost  to  students 
also  requires  university  approval. 

In  order  to  protect  the  university  from  competition,  faculty  members  are  not  allowed  to  create  online 
courses  for  sale  to  private  online  institutions,  corporations,  workshops,  or  any  other  form  which  would  in  any  way 
compete  with  Penn  State.  One  important  thing  to  understand  about  this  part  of  the  policy,  called  conflict  of 
commitment  clause,  is  that  it  also  applies  to  areas  far  from  the  faculty  members’  duties  to  the  university.  In  other 
words  a Shakespearean  scholar  who  chooses  to  create  a course  on  gardening  might  run  aground  of  this  policy 
because  Penn  State,  being  a wholely  comprehensive  university,  has  an  agricultural  program  that  looks  at  gardening. 
Thus,  if  a faculty  member  decides,  on  their  own,  to  create  an  online  course,  it  may  not  in  any  way  compete  with 
pu,  if  it  does,  it  is  not  allowed  without  prior  permission.  So,  a non-commissioned  work  is  owned  by  the  author, 
but  PSU  maintains  royalty  free  non-exclusive  right  to  utilize  the  materials  and  the  faculty  member  is  not  likely  to’be 
able  to  sell  the  materials  outside  as  it  may  be  considered  a conflict  of  commitment.  In  this  case,  ownership  is  not 
particularly  advantageous  to  most  faculty  members  which  has  caused  some  of  them  to  chill  to  the  idea  of  engaging 
in  online  course  development.  & & & 
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In  all  cases  written  agreement  signed  by  both  the  university  and  faculty  members  are  recommended  at  the 
start  of  the  project  development. 

University  of  Wisconsin  System  Courseware  Policy 

The  UWS  policy,  as  stated  in  the  background  (University  of  Wisconsin  System,  1997,  pi),  was  developed 
to  address  the  increased  institutional  involvement  in  development  of  copyrightable  instructional  materials.  It  was 
thus  written  with  special  focus  on  courseware  developed  with  substantial  support  form  the  university, 

University  of  Wisconsin  System  claims  university  ownership  on  courseware  developed  as  a)  work-for-hire 
or  assigned  duty  or  b)  with  substantial  use  of  institutional  support  or  involvement,  and  the  university  reserves  all 
rights  including  royalty  and  fees  for  work-for-hire  and  assigned  duty  and  rights  allocations  are  subject  to  written 
agreement  otherwise.  UWS  has  provided  a relatively  detailed  definition  for  “substantial”  which  helps  to  reduce 
conflict  or  ambiguous  interpretation  of  the  policy. 

For  courseware  that  is  developed  with  no  or  minimal  university  support  or  involvement,  university 
personnel  ownership  is  granted.  And  there  are  no  further  specifications  on  allocations  of  rights  for  those  courseware 
products. 

For  projects  developed  with  external  sponsorship,  UWS  specifically  states  that  agreement  with  external 
sponsor  shall  be  considered  in  determining  the  ownership  and  copyrights  of  all  parties  involved.  Nothing  is  specified 
regarding  joint  ownership. 

Interestingly,  UWS  is  the  only  one  among  the  three  institutions  we  investigated  that  has  specifically 
addressed  the  issues  of  responsibilities  of  the  institution  and  university  personnel,  external  distributions,  revision  and 
withdrawal.  The  policy  specifies  that  responsible  individuals  for  each  project,  who  are  assigned  by  sponsoring 
departments,  are  responsible  for  copyright-related  clearance,  waivers,  and  legal  agreements.  The  policy  encourages 
internal  use  and  sharing  based  on  agreements  developed  under  this  policy,  and  charges  for  internal  use  should  be 
limited  to  actual,  direct  cost.  External  distributions  may  occur  as  direct  rental,  sale,  or  licensing  by  UWS  or  as 
commercial  rental,  sale  or  distribution  by  a third  party  under  agreement  for  payment  of  royalties. 

Release  to  external  distribution  should  be  negotiated  between  the  institution  and  the  author. 

The  faculty  are  clearly  to  be  consulted  whenever  a course  they  authored  is  to  be  altered  or  revised.  When 
contrary  to  recommendation  of  the  author,  use,  or  authorization  for  use  by  others  shall  not  be  advertised  or  presented 
as  the  work  of  that  author,  except  appropriate  acknowledgement  of  the  author’s  original  production.  Author  shall  be 
offered  opportunity  for  revision  unless  he/she  declines.  When  courseware  is  considered  for  uses  different  from  the 
original  ones,  author  shall  be  consulted.  Thus  these  policies  clearly  have  a high  regard  for  the  importance  of  faculty 
consultation  and  consequently  academic  freedom.  The  UWS  policy  shows  respect  to  original  authors  of  courseware, 
and  ensures  that  authors  shall  be  consulted  for  revision,  alteration  and  different  uses,  regardless  of  ownership. 

Discussion: 

As  we  outlined  above,  there  are  many  critical  issues  associated  with  current  policy  making  provisions  for 
copyright  and  courseware  ownership.  In  our  view,  from  our  perspective  on  the  front  lines  of  the  Penn  State  debates, 
the  copyright  interests  are  at  the  nexus  of  faculty  versus  administration  concerns.  For  the  faculty  member  there  is 
concern  over  academic  freedom,  future  use,  consulting  restrictions,  and  even  job  security.  For  administrators,  there 
is  fear  of  competition,  lost  revenues,  and  even  their  own  job  security.  This  particular  moment  of  conflict  between 
faculty  and  administration  is  an  important  area  for  further  exploration  not  only  of  the  impact  of  specific  copyright 
and  courseware  ownership  issues  and  policies,  but  also  of  the  basic  working  relationships  between  faculty  and 
administrators — particularly  administrators  of  distance  education  enterprises  on  traditional  campuses. 

What  are  the  specific  issues  as  they  relate  to  the  instructional  designer?  Certainly  we  have  already  hinted  at 
one  of  the  biggest  concerns  for  service  instructional  designers  who  work  with  faculty  to  create  online  learning 
materials — that  of  disincentives  for  faculty  and  eventual  extinguishment  of  new  online  courses  to  work  on... concern 
over  their  own  job  security  too.  However,  a different  perspective  on  the  issue  might  help  instructional  designers  to 
support  a pro -administration  stance  on  copyright  policies.  If  the  institution  is  not  able  to  recoup  costs  and  make  a 
profit  on  the  online  enterprise,  their  jobs  are  equally  endangered.  Perhaps  the  instructional  designer  can’t  win  either 
way  they  go,  but  they  have  seen  that  the  most  immediate  concern  is  over  whether  or  not  faculty  will  continue  to 
engage  in  activities  that  are  not  financially  or  institutionally  rewarded  at  the  same  levels  as  competing  activities  such 
as  research,  scholarship  or  private  consulting. 

If  we  can  separate  ourselves  from  the  specific  personal  interests  of  each  of  these  groups  for  their  own  job 
and  income  concerns,  we  can  begin  to  look  at  the  problem  a bit  more  holistically,  perhaps  even  a bit  more 
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systemically.  All  of  the  concerns  voiced  by  all  of  the  different  parties  are  entirely  legitimate.  The  question  is  what 
is  most  in  alignment  with  the  goals  (historical  or  contemporary)  of  the  institution  of  higher  education.  Advocates  of 
online  learning  typically  cite  the  importance  of  democratizing  the  university  (CITE),  opening  access,  and,  in  the  case 
of  land  grant  public  institutions,  the  mission  of  outreach  as  primary  goals  for  institutions  of  higher  education.  These 
are  indeed  important  missions,  however,  they  are  very  tentatively  linked  to  the  interests  of  administrators  which  are 
primarily  around  competition  and  income  streams.  While  the  rhetoric  may  focus  on  democracy,  the  policies  do  not. 
Policies  that  would  focus  on  democratization  would  emphasize  changes  in  incentive  and  reward  systems  within  the 
university  to  more  fully  legitimize  online  learning  development  as  important  to  the  promotion  and  tenure  process. 
Policies  interested  in  furthering  the  reach  of  the  university  through  online  learning  would  not  restrict  faculty  from 
offering  courses  online  to  any  and  all  comers  through  whatever  outlets  were  available  and  willing  to  invest  the 
necessary  resources.  Instead,  most  administration  policies  assert  the  right  of  the  institution  to  own  the  materials 
developed,  essentially  discouraging  faculty  from  participating  and  lay  out  conflict  of  commitment  clauses  to 
minimize  competition.  This  is  borne  of  the  self-interest  of  maintaining  revenue  streams  within  the  institution.  This 
may  be  seen  by  faculty  as  particularly  aggregious  in  the  context  of  large  salary  increases  taken  by  upper 
administrators  within  their  home  institutions. 

On  the  other  hand,  the  faculty  members  are  rarely  working  out  of  purer  motives,  even  from  a systemic 
perspective.  The  rhetoric  many  faculty  members  use  to  forward  the  agenda  of  creating  online  courses  for  competing 
organizations  is  the  importance  of  academic  freedom.  The  stronger  rhetoric  is  actually  founded  on  the  basic  goal 
and  mission  of  higher  education.  In  this  case,  the  mission  that  is  emphasized  is  the  advancement  of  knowledge 
throughout  the  world.  The  open  access  through  MIT’s  recent  agreement  to  put  learning  materials  online  free  of 
charge  is  an  interesting  case  of  living  this  mission  out.  No  one  is  making  money  off  of  that  initiative  unless  it  is 
through  the  good  will  that  is  spread  by  returning  to  a nobler  mission  for  higher  education.  It  is  quite  possible  that 
such  an  open  access  initiative  may  engender  significant  public  support  for  public  institutions  of  higher  education, 
making  the  entire  enterpris  e far  more  relevant  than  it  has  been  for  many  years.  When  the  mission  is  seen  as 
advancement  of  knowledge,  and  sharing  of  learning  and  information  as  widely  as  possible,  non-compete  clauses 
come  up  far  short  and  administrator  copyright  policies  appear  to  be  more  interested  in  furthering  the  goals  of  a 
corporation  than  an  educational  institution. 
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Creating  “Technology  Intensive”  Courses  Through  Faculty  Mentoring 
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Abstract 

The  College  of  Education  (COE)  at  the  University  of  Hawaii  (UH)  has  already  begun  to  create  an 
infrastructure  to  systematically  infuse  technology  into  its  curriculum  through  a new  designation  of 
“Technology  Intensive*'  courses.  The  primary  goal  of  this  project  was  to  prepare  future  teachers  to 
integrate  technology  into  instruction  through  systematic  reformation  in  the  teacher  preparation  programs. 
To  meet  this  goal,  faculty  members  in  the  COE  and  their  pre-service  students  are  being  mentored  by 
graduate  students  to  model  technology  integration.  The  newest  generation  of  this  project  is  referred  to  as 
“LEI  Aloha,  “ which  stands  for  Learning  Enhancement  through  Innovation, 

Project  Need 

The  electronic  revolution  has  created  new  means  to  present  information  with  equipment  such  as  digital 
video  and  multimedia  authoring  software.  What  is  currently  lacking,  however,  is  the  professional  development  and 
expertise  in  the  usage  of  these  tools  in  today's  classrooms.  Faison  states  that  United  States  (US)  Office  of 
Technology  Assessment  (OTA)  has  concluded  that  the  most  direct  and  cost-effective  way  to  train  teachers  about 
technology  is  through  pre -service  teacher  education  courses.  The  question  then  is  how  do  we  train  faculty  to  be 
technology  role  models? 

A technology  needs  assessment  of  the  University  of  Hawaii  at  Manoa  (UH)  College  of  Education  (COE) 
faculty  was  conducted  by  Ho,  Sherry,  Speitel,  & Walton  (1999)  to  find  out  how  technology  was  being  used  to 
identify  the  training  needs  of  faculty,  and  to  provide  recommendations  for  the  future  growth  of  technology 
applications  in  the  COE.  The  survey  was  based  on  the  guidelines  of  the  National  Council  on  Accreditation  of 
Teacher  Education  (NCATE)  and  International  Society  for  Technology  in  Education  (ISTE)  standards  for 
technology  within  education. 

Overall,  it  was  found  that  there  was  a discrepancy  between  the  use  of  technology  for  teaching  and  the  use 
of  technology  for  professional  activities.  Faculty  and  students  commonly  used  email,  word -processing,  and  on-line 
information  for  their  scholarly  work  but  much  less  frequently  for  teaching  and  learning. 

It  was  interesting  that  students’  use  of  technology  mirrored  that  of  faculty,  but  they  used  technology  less 
frequently  than  faculty.  This  result  demonstrates  how  important  faculty  are  as  role  models  to  students.  “After  all, 
faculty  model  the  use  of  technology  applications  in  their  courses.  Also,  faculty  provide  opportunities  for  students  to 
use  technology...”.  Since  students  were  rated  lower  on  technology  use,  it  suggests  that  faculty  are  still  not 
providing  adequate  technological  opportunities  when  designing  their  courses.  This  may  be  because  the  integration 
of  technology  into  teaching  is  more  difficult  than  personal  use  of  technology. 

This  report  confirmed  the  need  for  a continuing  effort  to  solve  this  problem.  Five  years  ago  the  authors 
developed  such  an  effort  in  a program  to  provide  professional  development  for  a new  designation  of  “technology 
intensive”  courses.  The  program  provides  focuses  primarily  on  technology  workshops  and  one-on-one  mentoring 
assistance  for  faculty. 

Standards  and  Guidelines 

A 1996  pilot  project  was  funded  by  the  University  to  develop  a general  designation  of  “technology 
intensive”  courses  that  could  be  used  university-wide.  The  type  of  impact  desired  called  for  an  expansive,  long-term 
project  thinking  on  the  question  of  “what  should  our  graduates  know?”  With  the  leadership  of  Fulford  and  Hines 
(1997),  a committee  of  faculty  members  from  a broad  range  of  subject  areas  developed  a set  of  technology  intensive 
standards  for  the  development  of  the  courses. 

The  Technology  Intensive  Standards  were  derived  from  three  sources:  the  UH  General  Education 
Standards,  the  United  States  Education  Secretary's  Commission  on  Achieving  Necessary  Skills  (SCANS)  Report, 
and  the  International  Society  for  Technology  in  Education  (ISTE)  Standards  for  Basic  Endorsement  in  Educational 
Computing  and  Technology  Literacy. 


The  standards  include  six  categories:  ethics,  operations,  analysis,  retrieval,  application,  and  attitudes.  The 
six  categories  cover  35  specific  standards.  The  standards  development  team  provided  an  explanation  of  each 
category’s  purpose. 

The  technology -intensive  guidelines  were  established  by  Fulford  and  the  committee  (1997)  to  provide 
information  for  faculty  on  course  development.  The  idea  is  that  technology  should  help  facilitate  communication, 
problem  solving,  analysis,  research,  and  presentation.  However,  the  course  subject  matter  would  remain  the  same. 

Faculty  were  expected  to  use  at  least  one  standard  from  each  category  since  one  course  could  not  cover  all 
the  standards.  The  premise  was  that  among  the  variety  of  courses  students  take,  they  would  upon  graduation  have 
mastered  these  standards. 

Faculty  should  model  ethical  standards  as  well  as  visual  and  instructional  design  techniques  fundamental  to 
learning  with  technology.  Essential  to  all  courses  is  the  student  use  of  technology  including  opportunities  to  create 
multi-media,  web  pages,  videos,  and  electronic  slide  shows.  Faculty  are  encouraged  to  discuss  need  for  growth  in 
technology  with  their  students  and  to  acknowledge  issues  of  access,  skill  level,  and  anxiety. 

Professional  Development  Workshops  and  Mentoring 

Faculty  in  the  project  have  been  provided  with  ongoing  professional  development  to  effectively  integrate 
technology  into  their  courses.  Each  year  the  project  began  with  professional  development  workshops.  These  efforts 
received  help  from  the  graduate  students  in  an  Educational  Technology  Practicum.  In  each  session,  experienced 
faculty  and  teachers  in  kindergarten-1 (K.-12)  schools  showed  examples  of  how  they  were  integrating  technology 
into  their  courses.  These  were  followed  by  specific  demonstrations  of  hardware  and  software. 

The  next  half  of  each  session  was  devoted  to  hands-on  practice  with  the  technology.  Faculty  professional 
development  has  included  topics  such  as:  course  design,  visual  design,  electronic  conferencing,  web  page  design, 
multi-media  and  video  production,  electronic  presentations  and  portfolios,  technology  ethics  and  copyright, 
assessment  of  technology  projects,  one  computer  classroom  strategies,  and  online  collaboration. 

Once  faculty  members  have  attended  the  workshop  sessions  they  are  recruited  to  create  technology 
intensive  courses.  Faculty  who  express  interest  for  integrating  technology  into  their  courses  are  offered  one-on-one 
assistance  to  facilitate  this  process.  Faculty  are  paired  graduate  students  from  the  Educational  Technology 
Department  who  work  for  the  project.  Typically,  the  pairs  will  meet  on  a weekly  basis  over  a semester.  These 
sessions  usually  last  about  an  hour  and  take  place  in  the  faculty  memb  er's  office.  The  ultimate  outcome  of  the 
sessions  is  a course  proposal  in  which  the  faculty  member  outlines  the  ways  in  which  they  have  redesigned  a course 
to  incorporate  technology  using  the  technology  intensive  standards.  The  proposal  also  describes  the  ways  in  which 
the  faculty  member  is  modeling  technology  use,  the  ways  in  which  the  students  are  using  technology,  the  specific 
hardware  and  software  being  used  and  any  future  technology  needs  of  the  faculty  member.  In  order  for  the  students 
to  be  able  to  effectively  mentor  the  faculty  toward  this  goal,  an  instructional  design  approach  is  used  to:  (a)  set  goals 
for  the  professional  development;  (b)  provide  expertise  in  creating  a revised  curriculum  especially  with  regard  to  the 
technology -intensive  standards;  and  (c)  assist  in  improving  technical  skills  to  help  faculty  members  reach  their 
technology  integration  goals. 

The  overseeing  project  faculty  member  and  experienced  graduate  students  pair  student  mentors  with  faculty 
members  based  upon  the  technology  areas  in  which  the  faculty  member  has  expressed  interest  and  the  specific  areas 
of  expertise  of  student  mentors.  Compatible  schedules  and  preferred  computer  platforms  such  as  Mac  or  PC  are  also 
important.  Once  the  pairs  have  been  established,  the  student  mentor  contacts  the  faculty  member  to  assess  their 
technology  needs  and  levels  of  confidence  and  experience.  A survey  developed  by  previous  graduate  students  is 
administered  to  determine  interest  and  experience  with  various  software  applications  and  digital  technologies  such 
as  video  and  still  imaging. 

Once  the  student  mentor  has  determined  the  technology  skill  and  confidence  levels  of  the  faculty  member, 
an  initial  session  is  scheduled.  At  this  session,  the  mentor  and  the  faculty  member  discuss  the  content  of  the  course 
to  be  redesigned  and  the  vision  and  goals  the  faculty  member  has  for  redesigning  the  course  to  integrate  technology. 
The  mentor  must  be  tactful  with  the  faculty  member  to  be  encouraging  and  yet  realistic  as  to  what  can  be 
accomplished  in  one  semester.  The  pair  discuss  their  respective  time  schedules  and  the  amount  of  time  the  professor 
has  to  dedicate  to  the  redesign  process.  A letter  of  agreement  provided  by  the  mentor  is  signed  by  the  faculty 
member  stating  that  a)  the  faculty  memberacknowledges  that  the  assistance  will  be  provided  throughout  the 
semester,  b)  his  or  her  involvement  will  include  his  or  her  participation  and  communication  about  the  progress  of  the 
course,  c)  he  or  she  will  cooperate  in  the  evaluation  of  the  course  by  the  grant  project,  and  d)  he  or  she  agrees  to  be 
listed  as  a project  participant  on  the  project  web  site. 
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Student  mentors  need  a number  of  skills  in  order  to  conduct  one-on-one  faculty  mentoring.  Not  only  do 
they  have  technology  skills  that  they  impart  to  the  faculty  but  they  also  use  instructional  design  knowledge  to  help 
the  faculty  member  plan  and  organize  the  course  to  be  redesigned.  The  goal  of  the  mentoring  is  to  assist  faculty 
members  to  be  independent,  self-sufficient  users  of  technology.  To  achieve  this  goal,  students  often  need  to  know 
how  to  handle  “sticky”  situations.  Examples  of  “sticky”  situations  include  faculty  members  expecting  the  mentor  to 
do  the  technology  work  for  them,  faculty  members  not  completing  work  they  have  agreed  to  do  in  order  for  the 
sessions  to  proceed,  chronic  appointment  cancellations,  and  faculty  who  are  continually  distracted  by  new  media  and 
stray  off  topic  every  week. 

In  order  to  conduct  one-on-ones  and  to  handle  these  situations,  incoming  graduate  students  who  are  hired  to 
work  one-on-one  with  faculty  must  be  properly  trained.  Training  students  to  be  mentors  is  organized  by  an 
overseeing  project  faculty  member  and  conducted  by  graduate  students  who  have  mentoring  experience.  These 
cross-training  sessions,  where  students  train  each  other,  are  held  early  in  the  semester. 

The  most  fundamental  of  the  cross-training  sessions  conducted  is  “how  to  conduct  one-on-ones.”  In  this 
session  trainees  are  introduced  to  a packet  of  materials  created  by  project  staff  and  students  to  assist  both  the  faculty 
member  and  the  student  mentor  with  the  one-on-one  sessions.  Included  in  the  materials  packet  is  the  technology 
needs  survey,  the  letter  of  agreement  to  be  signed  by  the  faculty  participant  and  the  student  mentor,  a copy  of  the 
technology  intensive  standards,  and  examples  of  the  types  of  materials  a mentor  may  want  to  use  to  keep  their 
sessions  organized  and  on  track,  such  as  lists  of  questions  for  faculty  members,  sample  agendas  and  suggestions  for 
how  to  keep  sessions  focused. 

Essential  features  of  this  training  session  are  role-playing  and  activities  in  which  the  new  mentors  are  asked 
to  come  up  with  solutions  to  “sticky”  situations.  The  first  activity  uses  a series  of  written  scenarios.  New  mentors 
are  paired  up  and  asked  to  read  the  scenarios,  come  up  with  solutions  and  provide  and  receive  feedback  on  the 
solutions  with  the  rest  of  the  group.  The  second  activity  involves  role-playing.  Experienced  graduate  students  act 
out  skits  of  problematic  one-on-one  sessions  and  ask  for  feedback  on  various  solutions.  Feedback  from  new 
mentors  has  been  positive  about  the  role-playing  and  scenario  problem  solving.  They  report  that  it  gives  them  a 
better  idea  of  what  to  expect  and  how  to  handle  a variety  of  situations.  Experienced  graduate  students  are  asked  to 
share  their  specific  experiences  conducting  one-on-ones  and  provide  helpful  hints  and  practical  tips  for  success. 

Another  cross  training  session  conducted  for  incoming  mentors  is  on  the  technology  intensive  standards. 
Experienced  graduate  students  report  that  one  of  the  most  complicated  parts  of  one-on-one  mentoring  is  helping  the 
faculty  understand  how  to  incorporate  the  standards  into  the  redesigned  course.  The  ETEC  faculty  member  who 
headed  the  standards  committee  is  usually  asked  to  conduct  this  workshop.  The  faculty  member  gives  a brief  history 
of  the  standards  project  including  how  and  why  they  were  developed.  She  also  goes  over  each  standards  category  in 
detail  and  provides  specific  examples  from  previous  proposals  on  how  a standard  was  incorporated  into  a 
technology  intensive  course. 

A third  cross  training  session  is  provided  to  incoming  student  mentors  on  customer  service.  As 
representatives  of  the  grant  project  and  the  Educational  Technology  Department  providing  a service  to  the  college, 
the  student  mentors  are  expected  to  project  professionalism,  helpfulness  and  a positive  attitude.  This  session  goes 
over  basics  of  positive  communication,  attentive  listening,  courtesy  with  fellow  workers  and  customers  as  well  as 
telephone  behavior. 

These  cross-training  sessions  compliment  the  technology-based  sessions  conducted  on  a more  informal 
basis  by  the  students  working  for  the  project.  The  faculty  member  overseeing  the  one-on-ones  and  a graduate 
student  organize  cross-training  sessions  to  be  conducted  by  ETEC  graduate  students  with  specific  knowledge  in 
software  or  hardware.  For  example,  ETEC  graduate  students  have  conducted  cross-training  on  web  design  using 
DreamWeaver  and  Claris  Home  Page,  advanced  features  of  MS  Word,  howto  use  equipment  such  as  data 
projectors,  digital  video  and  still  cameras,  effective  web  searching,  digital  video  editing,  WebCT  and  how  to 
construct  PowerPoint  presentations. 

The  one-on-one  sessions  have  provided  individualized  assistance  in  the  same  fashion  as  mentoring 
relationships.  According  to  Archer  (1999)  many  teachers  are  not  confident  in  integrating  technology  into 
instruction.  Based  on  project  evaluations,  mentoring  faculty  members  through  the  one-on-one  sessions  has  vastly 
improved  confidence  levels  of  the  COE  participants  who  have  already  created  technology  intensive  courses. 

NCATE  (1999)  suggests  that  mentoring  and  providing  feedback  is  an  effective  method  of  professional 
development  in  technology  integration.  A train -the-trainer  effort  was  implemented  so  those  faculty  members 
provided  with  professional  development  would  also  serve  as  mentors  to  K-12  faculty.  Thus  far,  17  general 
education  faculty  and  23  full-time  COE  faculty  and  3 instructors  have  been  involved  in  the  project. 

Response  and  Impact 
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Faculty  involved  in  the  project  have  been  positive  in  their  feedback  about  the  workshops  and  especially  the 
one-on-one  mentoring.  Surveys  and  interviews  reflect  that  the  most  popular  form  of  course  redesign  assistance  is 
the  one-on-one  session.  Faculty  report  that  while  the  workshops  increase  their  technology  skills,  the  one-on-ones 
meet  their  particular  needs  and  are  where  they  can  truly  concentrate  on  their  own  course  redesign.  The  value  of  one- 
on-one  assistance  is  perhaps  most  evident  in  the  in-service  teacher  training  program  launched  about  the  same  time 
the  technology  intensive  training  program  was  being  implemented  in  the  College  of  Education.  The  Professional 
Diploma  in  Education  (PDE)  program,  as  a distance  education  program  targeting  teachers  in  rural  areas  on  Oahu  and 
neighbor  islands,  relies  heavily  on  electronic  media,  particularly  web-  based  delivery  and  email  communication,  and 
to  a lesser  degree  interactive  TV.  It  was  largely  due  to  one-on-one  mentoring  that  the  PDE  faculty  was  able  to 
overcome  their  initial  difficulties  with  Web  CT  to  launch  the  distance  education  program. 

While  the  technology  intensive  mentoring  has  been  highly  successful,  there  are  instances  where  one-on-one 
sessions  have  not  been  successful.  Sometimes  these  cases  arise  due  to  mentor-faculty  incompatibility.  These 
instances  provide  valuable  insight  into  program  improvement.  For  instance,  a faculty  member  purchased  a 
complicated  statistical  software  program  and  wanted  to  incorporate  it  in  their  personal  research  and  statistics 
coursework.  None  of  the  student  mentors  were  familiar  with  the  program.  A student  mentor  attempted  to  learn  it 
but  was  also  not  familiar  with  statistics  and  felt  that  the  amount  of  time  it  was  taking  to  grasp  the  program  was  not 
well  spent  based  on  his  limited  progress.  The  faculty  member  was  encouraged  to  consult  with  another  professor 
versed  in  statistics  and  in  this  particular  software.  While  most  student  mentors  are  familiar  with  both  computer 
platforms,  Macintosh  and  PC,  a situation  arose  where  hardware  problems  a faculty  member  was  have  having  with  a 
Mac  could  not  be  solved  by  the  student  mentor  who  was  more  familiar  with  PCs.  Situations  like  this  are  referred  to 
other  mentors  or  the  Information  and  Technology  Services  Department  on-campus. 

Situations  have  also  arisen  where  a faculty  memberhas  signed  up  for  mentoring  and  had  intentions  of 
redesigning  a course  but  then  find  they  are  too  busy  to  add  additional  work  to  their  load.  Mentors  may  have  been 
told  this  up-front  or  may  have  discovered  it  gradually  through  chronic  postponing  or  cancellation  of  one-on-one 
sessions.  Although  faculty  may  be  enthusiastic  about  redesigning  their  courses  to  be  technology  intensive,  some 
may  not  be  ready  to  completely  redesign  their  courses.  The  expansion  of  the  technology  intensive  project  has 
addressed  this  challenge  through  a three-tiered  approach  to  course  redesign. 

Project  Expansion 

Although  the  project  has  been  highly  successful,  only  a third  of  the  College  of  Education  faculty  that  work 
with  pre -service  and  in-service  teachers  have  been  trained.  Two  challenges  facing  the  project  group  are  how  to 
recruit  faculty  members  still  reticent  about  using  technology  and  how  to  provide  a continuum  of  technology 
intensive  courses  throughout  a COE  student’s  schooling  from  the  Community  College  level  through  to  their  field- 
based  experiences.  Continued  federal  funding  from  the  US  Department  of  Education’s  “Preparing  Tomorrow’s 
Teachers  to  Use  Technology”  program  (PT3)  has  allowed  the  project  to  expand  to  meet  these  needs.  The  newest 
project  (PT3)  is  referred  to  as  “LEI  Aloha”  - Learning  Enhancement  through  Innovation. 

To  meet  the  challenge  of  reluctant  faculty,  the  LEI  Aloha  staff  has  developed  a three-tiered  approach  to 
support  the  needs  of  teacher  education  courses. 

Technology  Intensive  Courses: 

These  courses  follow  the  Technology  Intensive  Standards  and  Guidelines  to  improve  technology  literacy 
while  continuing  to  emphasize  course  content.  Students  have  a high  level  of  involvement  using  technology,  while 
faculty  meet  standards  as  exemplary  role  models  using  technology.  In  this  process  faculty  need  assistance  in 
rethinking  and  redesigning  their  courses  to  integrate  technology.  They  will  have  to  revise  course  objectives,  create 
new  strategies  and  activities,  locate  and  create  new  media,  and  develop  alternative  assessments. 

Technology  Applied  Courses: 

These  courses  add  the  use  of  technology  to  the  current  course  structure  encouraging  students  to  use 
technology  resources  in  their  research,  communication,  and  presentations.  The  faculty  member  demonstrates  and 
uses  technology  in  the  presentation  of  course  content  and  communicates  with  students  electronically.  Faculty  need 
help  to  become  familiar  with  and  use  the  multitude  of  alternatives  that  technology  provides. 

Technology  Enhanced  Courses: 
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In  these  courses,  the  faculty  member  demonstrates  and  uses  technology  in  the  presentation  of  course 
content  and  communicates  with  students  electronically.  Faculty  need  to  become  proficient  with  redesigning  their 
course  presentations  and  using  electronic  communications. 

To  meet  the  second  challenge  of  a continuum  of  technology  intensive  courses  throughout  a COE  student’s 
schooling,  the  project  has  expanded  its  mission  to  include  Community  College  faculty.  A majority  of  COE  students 
have  taken  courses  prerequisite  to  the  COE  at  the  Community  Colleges.  For  many  students.  Community  College 
faculty  members  provide  the  introduction  to  a college  career  and  have  a great  influence  on  how  students  begin  to 
shape  their  view  of  the  teaching  and  learning  environment.  LEI  Aloha  is  currently  working  with  each  of  the  seven 
community  college  campuses  of  the  UH  system  to  create  technology  intensive  courses  at  all  the  campuses. 

While  the  first  components  of  the  educational  continuum  are  the  community  college  and  COE  faculty,  the 
final  component  are  K-12  teachers  who  serve  as  mentors  to  students  in  their  field  experiences.  These  teachers  play  a 
vital  role  at  a critical  time  in  students’  careers.  Hinnant  (1997)  states  that  “the  elements  most  crucial  to  the 
successful  integration  of  technologies  into  teaching  are  teachers  and  their  ability  to  use  all  their  skills  to  inspire, 
motivate,  challenge,  and  enrich  their  students  (p.  1)”.  In  these  field  experiences,  students  see  first-hand  whether  and 
how  technology  can  be  integrated  into  the  K-12  school  environment.  Positive  role  models  are  crucial  at  this 
juncture. 

The  LEI  Aloha  project  is  developing  a series  of  web-based  technology  intensive  teacher  education  courses. 
These  courses  are  being  offered  in  conjunction  with  a technology  intensive  sabbatical  opportunity  for  K-12  teachers. 
The  program  will  train  in-service  teachers  to  be  technology  mentors  to  pre -service  teachers  completing  their  field- 
based,  student  teaching  requirements. 

This  project  has  been  designed  to  create  a broad  ranging  impact.  It  is  based  on  the  concept  of  mentoring 
and  training-the-trainer  to  integrate  technology.  COE  students  are  expected  to  become  teachers  who  will  provide 
their  own  students  with  the  benefits  of  their  knowledge.  This  will  create  a multiplier  effect  so  that  in  just  a few 
years,  including  the  students  of  students,  thousands  of  students  will  be  affected. 
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Abstract 

Trek  21  is  a 3 -year  PT3  implementation  grant  from  the  United  States  Department  of  Education  designed  to 
build  the  capacity  in  teacher  educators  (teacher  education  faculty,  Professional  Development  School 
faculty,  pre-service  interns)  to  integrate  technology  into  their  teaching.  The  goal  of  Trek  21  is  to  prepare 
educators  to  use  and  integrate  instructional  technologies  (ITs)  for  teaching  and  learning.  This  paper 
discusses  shifts  in  the  development  between  year  one  and  year  two  of  teachers*  web-based  instructional 
units,  with  a focus  on  the  degree  to  which  year  two  units  feature  greater  student  engagement  using 
networked  technologies.  Through  a detailed  discussion  of  project  events  and  participant  outcomes, 
strategies  of  the  Trek  21  Model  of  Professional  Development  process  are  illustrated. 


Introduction 

The  National  Commission  on  Teaching  and  America’s  Future  (1996)  asserts  that  pre -service  teacher 
education  has  the  potential  for  the  greatest  influence  in  enhancing  the  learning  opportunities  of  children.  The  report 
goes  on  to  state  that  excellent  pre -service  teacher  education  requires  bringing  together  the  contexts  of  schools  with 
the  preparation  of  teachers.  This  preparation  must  be  adequate  and  sustained  in  the  professional  practice  of  teachers. 

Current  instructional  technologies  offer  students  and  teachers  access  to  information  and  multiple  modes  of 
knowledge  construction.  By  design,  these  new  technologies  make  this  method  of  knowledge  construction  largely 
individualistic  and  demand  changes  in  teaching  and  learning  environments  so  as  to  effectively  integrate  them  into 
the  instructional  process.  It  is  essential  for  new  teachers  (pre -service)  to  develop  new  perspectives  on  instructional 
design  and  new  instructional  technology  skills  within  settings  and  environments  in  which  these  approaches  are 
modeled.  Students  exiting  teacher  preparation  programs  must  have  acquired  the  ability  to  integrate  new  and  future 
instructional  technologies,  and  must  have  gained  the  skills  necessary  to  adjust  to  teaching  environments  where 
limited  technology  infrastructure  and  capacity  exist. 

Preparing  Tomorrow’s  Teachers  to  Use  Technology  (PT3)  and  Trek  21 

PT3  is  an  initiative  of  the  United  States  Department  of  Education.  Grants  from  the  PT3  Initiative  provide 
funding  for  innovative  programs  to  develop  technologically  proficient  educators  who  are  well  prepared  to  meet  the 
needs  of  2I*‘  century  learners.  “Trek  21 : Educating  Teachers  as  Agents  of  Technological  Change”  is  a 3 -year  PT3 
implementation  grant  from  the  U.  S.  Department  of  Education  designed  to  prepare  educators  involved  in  West 
Virginia  University’s  five-year  teacher  preparation  program  to  integrate  instructional  technologies  (ITs)  into  their 
teaching.  The  grant  was  awarded  to  the  College  of  Human  Resources  and  Education  at  West  Virginia  University 
(WVU)  in  1999. 

The  Trek  21  model  of  teacher  professional  development  includes  host  teachers  in  West  Virginia 
University’s  Professional  Development  Schools  (PDS),  faculty  from  WVU’s  College  of  Human  Resources  and 
Education  (HR&E),  and  student  interns  in  their  fifth  year  of  the  teacher  preparation  program.  Trek  21  looks  to 
impart  lasting  change  in  the  culture  of  teacher  practice.  To  help  accomplish  this  change,  the  design  of  the  Trek  21 
model  includes  an  annual  cycle  of  professional  development  events:  Summer  Institutes  for  WVU  faculty  (seven 
days)  and  PDS  teachers  (three  weeks),  school  site  visits,  mini  conferences,  and  continuity  meetings  in  the  fall  and 
spring. 

Project  Design 
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The  Trek  21  model  is  an  annual  cycle  of  events  sequenced  in  such  a manner  so  as  to  ensure  long-term 
adoption  of  new  practice,  continuous  support  and  feedback,  and  sustainability  beyond  the  project. 

Summer  institutes 

A seven -day  technology  integration  summer  institute  for  university  faculty  and  a three-week  technology 
integration  summer  institute  for  PDS  host  teachers  begins  each  year  of  the  Trek  21  project  cycle.  These  Institutes 
address  genres  of  instructional  technology  applications  (Harris,  1998),  target  technical  training,  and  prepare 
instructional  technology  materials  and  resources  necessary  for  immediate  integration  into  classroom  instruction.  The 
final  outcome  is  a teacher-developed,  web-based  instructional  unit,  which  is  implemented  in  the  fall  by  the  teacher 
in  collaboration  with  a pre -service  student  intern. 

Continuity  meetings/site  visits 

Following  the  summer  institutes.  Trek  21  holds  continuity  meetings  with  PDS  faculty  once  each  semester 
(fall  and  spring)  to  address  issues  related  to  the  successful  integration  of  instructional  technologies  at  their  location. 
School  site  visits  occur  throughout  the  year  to  provide  continued  support  and  gather  data  on  unit  implementation  and 
local  concerns. 

Mini  conferences 

Scheduled  to  occur  twice  each  academic  year,  a mini-conference  is  held  in  partnership  with  West 
Virginia’s  ‘Technology,  Teacher  Education,  Tomorrow”  (T3)  non-profit  organization  whose  mission  is  to  share  best 
practices,  receive  technology  enhancement  training,  and  deliver  presentations  of  activities  related  to  the  integration 
of  instructional  technologies.  These  conferences  serve  as  our  opportunity  for  state-wide  dissemination  of  Trek  21 
research  results  and  presentation  by  participants  of  best  practice  where  the  integration  of  ITs  is  central. 

Summary  of  Year  One  Events  and  Outcomes 

Participants  for  the  first  year  included  47  PDS  teachers  who  supervised  pre -service  interns  during  the  fall 
semester.  Of  the  47  teachers,  17  were  elementary  school  teachers,  19  were  middle  school  teachers,  and  1 1 were 
high  school  teachers.  These  teachers  attended  one  of  three  summer  institutes,  each  lasting  three  weeks.  The  Trek  21 
Professional  Development  process  for  year  one  has  been  examined  along  multiple  foci  and  reported  (Adams, 
Dunham,  Wells,  & Shambaugh,  2001).  Initial  findings  profiled  K-12  participants  as  preferring  a teacher-centered 
approach,  providing  minimal  written  details  in  lesson  plans,  and  depicting  themselves  as  low-level  computer  users 
with  minimal  integration  levels  of  instructional  technology.  Analysis  of  lessons  submitted  by  teachers  at  the 
beginning  of  the  PDS  Summer  Institutes  revealed  30%  (14)  of  the  teachers  had  a complete  lesson(s),  32%  (15)  had 
some  lesson  plan  features,  and  38%  (18)  did  not  have  a lesson  plan. 

An  externally  developed  rubric  guided  the  evaluation  of  participants’  units,  which  resulted  in  an  initial  low 
percentage  of  posted  teacher  units.  Following  subsequent  revisions  during  the  fall  continuity  meeting,  all  teachers’ 
units  were  posted  to  the  Trek  21  web  site.  The  web-based  units  were  typically  structured  as  teacher  sites,  with  a 
limited  number  distinguishing  between  teacher  and  student  activity.  During  year  one,  not  enough  emphasis  was 
placed  on  the  development  of  student-centered  units  and,  as  a result,  learning  activities  were  frequently  depicted  in 
lessons  using  traditional  teacher-centered  approaches.  Although  some  ITs  were  integrated,  particularly  chat  rooms 
and  web  boards,  initially  the  units  lacked  the  procedural  details  needed  in  order  for  other  teachers  and  students  to 
utilize  them  as  the  participants  intended. 

The  most  frequent  learning  strategies  employed  by  teachers  in  their  units  involved  a)  problem  solving  using 
information  retrieval  and  desktop  publishing  technologies,  b)  concept  scaffolding  with  PowerPoint  presentations  and 
the  Internet,  and  c)  discovery  learning  through  information  retrieval  and  Internet  searches.  IT  applications  most 
utilized  by  teachers  were  presentations  (32  activities),  information  retrieval  (29),  and  Internet  searches  (17). 

Final  evaluations  of  teachers’  web-based  units  indicated  web  design  was  an  area  that  needed  to  be  more 
fully  addressed  early  in  the  development  process.  The  most  frequently  occurring  web  design  issues  were:  providing 
needed  structure  for  the  learning  activities  and  information  to  communicate  the  overall  intent,  giving  individual 
lessons  descriptive  titles,  reducing  scrollable  text,  documenting  individual  lessons  on  separate  pages,  reducing 
animated  GIFs,  improving  background/text  contrast,  addressing  web  browser  differences,  providing  appropriate 
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navigation  and  consistent  use  of  navigation  icons.  Addressing  these  issues  with  teachers  earlier  in  the  development 
of  their  web  units  would  provide  the  design  and  software  foundation  they  needed  to  design  well  from  the  start. 

The  first  year  of  professional  development  with  PDS  teachers  revealed  the  need  to  provide  teachers  with  a 
clearer  picture  of  the  overall  intent  of  their  web-based  unit.  For  some  participants,  translating  what  they  currently  do 
in  the  classroom  into  web -based  student-centered  activities  was  a major  shift.  Feedback  from  the  summer  2001 
institutes  indicated  that  providing  greater  clarity  in  our  expectations  of  teachers,  their  obligations  and 
responsibilities,  would  markedly  improve  the  instructional  outcomes  of  the  project.  Furthermore,  year  one  findings 
indicated  that  by  more  clearly  defining  terms  such  as  “lesson”  and  “unit”,  incorporating  strong  examples  of  lesson 
templates,  and  providing  consistency  between  the  evaluation  tools  and  instructional  strategies  employed,  the 
professional  development  process  would  improve. 

The  importance  of  sufficient  staff,  proper  facilities,  and  ample  time  is  well  understood.  During  the  Trek  21 
professional  development  process  teachers  expressed  the  need  for  as  much  time  as  possible  for  unit  development.  In 
addition  year  one  feedback  revealed  that  during  the  development  time  teachers  require  both  consistently  available 
technical  assistance  and  sufficient  pedagogical  expertise  as  it  relates  to  the  instructional  technologies  integrated. 

This  demand  for  both  technical  and  instructional  expertise  presented  a significant  staffing  challenge  during  year  one 
institutes. 

Year  Two  Design  Changes 

Based  on  year  two  experiences,  design  changes  were  implemented  to  address  specific  preparation  and 
training  areas  in  need  of  improvement.  Improvements  such  as  clearly  conveyed  expectations,  using  common 
terminology  understood  by  all  participants,  and  indicating  that  units  would  consist  of  a minimum  of  five  lessons 
enabled  teachers  to  plan  well  for  the  institute.  Their  planning  was  also  improved  by  asking  them  to  submit  units 
they  had  taught  many  times  before,  they  felt  very  comfortable  with,  and  that  they  would  be  teaching  in  the  fall. 

Re-designed  unit/lesson  templates  provided  participants  with  a consistent  structure  and  common 
terminology  upon  which  all  their  units  could  be  developed  and  evaluated.  The  significance  of  the  re-designed 
templates  was  the  way  they  guided  teachers  toward  separation  of  student  and  teacher  activities,  helping  to  emphasize 
the  goal  of  creating  student -centered  lessons.  As  a result,  prior  to  arriving  at  the  summer  institute  most  of  the 
participants’  units  were  pedagogically  complete  and  more  ready  for  instructional  technology  integration.  To  address 
some  of  the  web  design  issues  found  to  be  problematic  in  year  one,  participants  were  encouraged  to  begin  their  web 
page  development  using  Trek  21  pre  -designed  web  page  templates.  These  web  page  templates  included  invisible 
tables,  and  contained  consistent  design  and  navigation  features  that  enabled  participants  to  concentrate  on  the 
application  of  instructional  technologies  rather  than  web  page  design.  Participants  were  evaluated  using  a detailed, 
internally  developed  rubric  that  aligned  directly  with  requirements  designed  into  the  unit/lesson  and  web  page 
templates  they  all  used  when  submitting  application. 

The  evaluation  of  year  one  institutes  identified  the  need  for  classroom  expertise  during  training,  which  led 
to  the  incorporation  of  prior  teacher  participants  as  Instructional  Leaders  (ILs)  as  part  of  the  Trek  21  Professional 
Development  process.  The  Instructional  Leaders  served  as  master  teachers  to  help  ground  the  summer  institutes  by 
assisting  new  project  participants  with  pedagogical  issues,  and  by  providing  them  with  details  on  how  to  incorporate 
the  three  genres  of  instructional  technology  known  to  promote  student  engagement  (Harris,  1998).  The  presence  of 
Instructional  Leaders  allowed  other  Trek  21  personnel  to  focus  on  technical  support,  and  resulted  in  the  inclusion  of 
additional  (optional)  breakout  sessions  to  meet  the  diverse  needs  of  novice,  intermediate,  and  expert  participants. 

The  Trek  21  cycle  of  events  includes  fall  and  spring  continuity  meetings  and  informal  site  visits  in  an  effort 
to  provide  opportunities  for  further  skill  development  and  technical  support.  Each  continuity  meeting  offers  a full 
day  of  development  time  for  teachers  to  revise  their  units  with  the  convenience  of  a support  staff  readily  available  to 
provide  assistance.  The  arrangement  of  informal  site  visits  throughout  the  year  allows  Trek  21  staff  members  to 
visit  each  school  for  follow-up  sessions  with  participants,  assisting  participants  with  problems  and  becoming 
familiar  with  their  classroom  environment.  Additionally,  selected  project  participants  and  Trek  21  staff  members 
serve  as  part  of  a Trek  21  Leadership  and  Planning  Committee,  which  reflect  on  prior  institute  processes  and  offer 
suggestions  for  revision. 

Summary  of  Year  Two  Events  and  Outcomes 

Participants  in  year  two  consisted  of  27  teachers  selected  from  the  Professional  Development  Schools 
within  the  five  West  Virginia  counties  included  in  the  Trek  21  project.  This  group  was  comprised  of  two  preschool 
teachers,  17  elementary  school  teachers,  four  middle  school  teachers,  and  four  high  school  teachers.  These  teachers 
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attended  one  three-week  long  Summer  Institute.  Information  collected  from  a self-reporting  survey  indicated  that 
the  participants’  technological  skill  levels  varied  from  novice  computer  users  to  expert  computer  users. 

Random  sweeps  and  select  comprehensive  evaluations  of  the  pre-  and  post-institute  lessons  illustrated  the 
extent  of  student  engagement  in  units.  Although  teachers’  objectives  and  methods  of  assessment  were  not  changed 
by  technology  integration,  modifications  were  apparent  when  comparisons  were  made  between  pre  - and  post- 
institute lessons.  Active  student  engagement  in  instructional  procedures,  instructional  strategies,  and  integration  of 
instructional  technologies  increased  significantly  in  post-institute  units,  reflecting  an  increase  in  student -centered 
lesson  design.  For  example,  active  student  engagement  included  overt  responses  to  instructional  prompts  such  as 
sequencing  cards,  responding  verbally  in  writing,  retrieving  information  from  a web  site,  and  participating  in 
discussion.  Units  developed  during  the  summer  2001  institute  clearly  indicate  the  development  of  learner-centered 
units  where  student  engagement  was  encouraged  via  the  application  of  instructional  technologies.  Evidence  of 
changes  to  instructional  procedures  included  the  existence  of  a motivating  introduction,  review  information,  new 
content,  guided  practice,  independent  practice,  closure,  and  extensions.  Changes  to  instructional  procedures  also 
included  active  student  participation  in  procedures  for  student-centered  activities  located  within  teachers’  units. 
Indicators  of  change  in  instmctional  strategies  took  the  form  of  advanced  organizers,  whole  group  instruction,  peer- 
mediated  instruction,  group  discussion,  active  responding,  problem-solving,  research,  inquiry,  hands-on  instruction, 
manipulatives,  dramatic  representation.  Journaling  or  writing,  student  presentations,  or  teacher  demonstrations. 

Most  participants  included  detailed  information  on  the  unit  web  pages  on  how  to  apply  the  instructional  strategies  in 
order  to  promote  student  engagement  in  the  unit.  Similarly,  improvements  in  IT  integrations  involved  extensions  to 
activities  such  as  Computer  Aided  Instruction  drill-and-practice,  simulation,  educational  games,  word  processing, 
information  retrieval,  internet  access,  e-mail,  bulletin  boards,  listservs,  authoring,  multimedia  development,  desktop 
publishing,  electronic  presentations,  video  development,  open  lab  access,  or  web  page  development.  Overall, 
instructional  technologies  integrated  by  teachers  provided  a variety  of  avenues  that  promoted  student  engagement 
and  participation  in  unit  lessons. 

In  addition  to  meeting  the  need  for  greater  assistance  with  project  participants  during  the  institute. 
Instructional  Leaders  guided  participants  on  approaches  to  effective  student  engagement  and  the  enhancement  of 
teaching  and  learning  via  instructional  technology.  Because  sufficient  staff  and  facilities  were  available,  participants 
were  afforded  flexible  technical  training  whereby  they  could  select  individual  IT  training  sessions  based  on 
integration  potential  within  their  unit  lessons.  By  and  large.  Trek  21  participation  resulted  in  unit  modifications  by 
teachers  that  unmistakably  reflected  strong  student-centered  lessons. 

Implications  for  Year  Three 

Based  on  the  overall  results  from  years  one  and  two,  minor  refinements  will  be  incorporated  into  the  design 
of  year  three  to  improve  the  Trek  21  Professional  Development  process.  These  refinements  include  clearer 
benchmarks  pinpointed  throughout  the  institute,  perhaps  in  the  form  of  a checklist,  so  participants  can  manage  the 
amount  of  time  spent  on  each  section  appropriately.  Minor  revisions  to  project  documents  such  as  the  evaluation 
rubric  and  the  unit/lesson  templates  will  provide  more  precise  guidelines  for  development.  By  offering  concise 
directions  and  consistent  expectations  for  all  personnel  involved  in  the  project,  participants  will  be  more  prepared  to 
create  web-based  instructional  units  that  integrate  instructional  technologies  appropriately  for  student  engagement. 

Implications  for  year  three  focus  on  expanding  the  number  of  ITs  by  offering  additional  (optional)  breakout 
sessions.  During  the  first  two  days  of  the  institute,  exemplars  will  be  identified  that  better  illustrate  the  educational 
purpose  of  instructional  technologies  that  will  be  offered  throughout  the  institute.  Teachers  will  be  encouraged  to 
choose  from  a pedagogical  standpoint  the  instructional  technologies  they  want  to  learn  and  integrate  into  their 
lessons.  This  will  allow  the  participants  to  set  their  own  limits  on  the  new  knowledge  gained  throughout  the 
institute,  so  they  are  not  overwhelmed  with  new  possibilities. 

Conclusions 

The  design  of  professional  development  with  instructional  technology  as  a focus  is  complex  and  requires 
clear  guidelines,  continual  communication,  and  flexibility  to  meet  individual  teacher  needs.  The  existence  of  clear 
instructions,  consistent  objectives,  and  defined  expectations  contribute  to  the  overall  success,  particularly  in  the  area 
of  active  student  engagement.  Factors  such  as  collaboration  with  instructional  leaders,  the  development  of  a 
Leadership  and  Planning  Committee,  and  the  use  of  an  evaluation  rubric  by  external  reviewers  have  all  led  to  the 
specification  of  clear  and  distinct  roles  and  appropriate  support  mechanisms  for  project  participants.  Requiring 
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teachers  to  submit  a unit  with  a minimum  of  five  related  lessons  prior  to  the  institute  allowed  us  to  begin  the  institute 
with  illustrations  and  demonstrations  of  appropriate  student  engagement  on  unit  web  pages. 

Results  from  the  Trek  21  model  of  Professional  Development  clearly  indicate  an  effective  process  for  the 
development  of  learner-centered  units  facilitated  through  the  integration  and  application  of  instructional 
technologies.  Preliminary  findings  suggest  that  the  implementation  of  Trek  21  participants’  units  will  enhance 
student  engagement  in  the  instructional  strategies,  procedures,  and  integration  of  instructional  technologies. 
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Abstract 

To  prepare  diverse  learners  for  various  roles  in  life,  teachers  should  develop  contextualized  learning 
environments  that  enable  students  to  develop  individual  skills  while  contributing  to  the  knowledge  of  the 
classroom  community.  This  is  accomplished  by  integrating  various  subject  matter  domains  with 
interdisciplinary  themes  that  incorporate  real-life  roles  and  contexts.  This  paper  provides  a framework  for 
designing  and  implementing  integrated  units  that  address  the  needs  of  all  students. 

Introduction 

As  K-12  classrooms  continue  to  diversify,  it  becomes  increasingly  important  that  teachers  structure 
learning  experiences  to  meet  the  individual  needs  of  all  students.  Many  students  cope  with  learning  and  emotional 
disabilities,  economic  challenges,  and  limited  English  proficiency.  Instruction  must  be  designed  to  engage  such 
students  while  appealing  to  multiple  learning  styles  and  those  who  are  gifted.  Instruction  is  more  meaningful  for 
students  as  they  progress  through  material  that  is  presented  in  an  integrated,  individualized  manner.  The  challenge  is 
to  promote  learning  and  inclusiveness  by  engaging  and  scaffolding  students  with  varying  abilities.  This  is 
accomplished  through  thematic,  integrated  lessons  that  utilize  various  media  and  technologies.  Literature  on 
learning  theories  informs  the  development  of  such  student-centered  environments.  Experts  in  the  fields  of  learning, 
cognition,  and  instructional-design  suggest  approaches  that  serve  as  guidelines  for  tailoring  instruction  to  meet  the 
needs  of  all  students  so  that  they  may  participate  to  the  fullest  extent  possible  and  progress  to  increasingly  higher 
levels  of  expertise.  Drawing  upon  theory,  such  instruction  may  be  designed  by  utilizing  the  steps  listed  below  and 
displayed  graphically  in  Figure  1. 

Determine  broad  goals 

Establish  an  enriched  context 

Develop  domain -specific  goals  related  to  the  context 

Use  multiple,  authentic  assessments  as  a base  line  for  developing  individual  learning  experiences 

Use  a variety  of  strategies  and  activities  that  address  diverse  learning  styles 

Provide  a variety  of  media  and  technology  to  individualize  learning 

Scaffold  through  constant  monitoring  and  guidance 

Allow  time  for  student  work  to  be  self-paced 

Encourage  a learning  community  that  values  the  input  of  individuals 

Extend  learning  and  transfer  in  subsequent  themes 
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Figure  L Model  for  Designing  and  Implementing  Integrative  Thematic  Unit  for  Individualization 
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Determine  Broad  Goals 


The  first  step  in  designing  an  individualized,  integrated  learning  environment  is  to  identify  the  broad 
picture  and  goals  of  society  (Harless,  1998).  If  the  ultimate  outcome  of  education  is  to  produce  accomplished 
citizens,  then  instructional  goals  must  move  beyond  a subject  matter  orientation.  Rather  than  developing  lessons  to 
teach  in  response  to  segregated  math,  science,  and  language  arts  goals,  planning  should  first  address  broader  goals 
that  equip  students  with  the  skills,  knowledge,  and  attitudes  that  will  prepare  them  for  various  roles  in  society 
(Harless,  1998,  p.48).  Once  those  broader  goals  are  developed,  the  teacher  may  translate  them  into  smaller 
competencies  (Gagne,  Briggs,  & Wager,  1992  pp.  145-184).  Reigeluth’s  (1999)  Elaboration  Theory  advocates 
beginning  with  broader  topics  within  a domain  and  then  progressing  to  more  detailed  principles,  which  provides  a 
sequence  that  “enables  learners  to  understand  the  tasks  holistically”  (p.433).  Reigeluth  further  explains  that  this 
type  of  instruction  allows  students  to  develop  a schema,  which  can  be  expanded  through  subsequent  lessons. 
Addressing  subject  matter  goals  through  a rich,  integrated,  contextualized  environment  facilitates  making  meaning 
and  connections  of  new  knowledge. 

Establish  a Context 

Providing  an  enriched  environment  that  includes  adequate  resources  and  is  based  upon  relevant  topics  that 
enphasize  real  people  and  places  engages  and  sustains  interest  (Kovalik  1997;  1999).  For  example,  a fifth  grade 
classroom  begins  a two -week  unit  with  an  around-the-world  theme.  When  the  students  enter  the  room  on  the  first 
Monday  of  the  unit,  they  find  maps,  newspapers,  pictures,  and  other  artifacts  representing  various  regions  from 
around  the  world  lining  the  walls  and  tables  of  the  classroom.  Students  instantly  become  immersed  in  a 
contextualized  environment,  which  will  engage  them  in  activities  that  promote  deeper  understanding  of  people  and 
cultures  of  the  world.  These  learning  experiences  will  prepare  students  to  become  global  citizens. 

The  teacher  begins  by  Letters  From  Felix  (Langen  & Droop,  1994)  aloud  to  the  students.  This 

story  is  about  a stuffed  rabbit  named  Felix  who  gets  separated  from  his  owner,  Sophie  at  an  airport.  Felix  boards 
numerous  planes  and  travels  around  the  world  to  several  countries  as  he  tries  to  find  his  way  back  home  to  her  in 
Ohio.  As  Felix  stops  in  each  country,  he  writes  a letter  to  Sophie  and  includes  descriptions,  drawings,  and  pictures 
of  what  he  sees.  As  Sophie  receives  each  letter,  her  family  discusses  characteristics  of  the  described  country.  The 
book  has  envelopes  with  pullout  copies  of  these  letters.  This  story  sets  the  context  for  learning  activities  and 
motivates  children  to  learn  more  about  other  cultures  and  regions. 

Develop  Domain-Specific  Goals 

As  students  take  on  the  role  of  global  citizen  during  the  unit,  they  develop  a greater  understanding  of  their 
world  and  develop  skills  in  specific  subject  areas  in  meaningful  ways.  Math,  science,  social  studies,  art,  language 
arts,  and  technology  objectives  easily  emerge  and  skills  are  applied  in  realistic  ways.  Math  goals  are  addressed  as 
students  learn  to  measure  and  calculate  distances  traveled;  compare  and  contrast  data;  use  calculators  to  compute 
travel  budgets;  and  add,  subtract,  multiply,  and  divide  numbers  with  decimals.  Science  goals  are  addressed  as 
students  discover  global  weather  patterns  and  landforms.  Learners  apply  the  scientific  process  as  they  conduct  local 
weather  experiments  and  compare  data  with  other  cities.  Social  studies  goals  are  attended  to  as  students  develop  an 
understanding  of  diverse  cultures,  places,  and  environments.  Art  goals  further  such  understanding  as  students  study 
art  and  artifacts  of  cultures.  For  example,  students  may  discover  that  artists  Vincent  Van  Gogh  of  the  Netherlands, 
Diego  Rivera  of  Mexico,  and  Faith  Ringgold  of  the  United  States  all  spent  time  in  Europe  and  created  paintings  of 
sunflowers.  These  kinds  of  patterns  promote  dialogue  about  similarities  and  differences  among  countries. 

As  students  use  the  Internet,  software,  and  word  processing  to  gather  and  express  information,  they  become 
proficient  in  the  use  of  technology,  which  is  in  alignment  with  national  technology  goals.  Language  arts  goals  thread 
these  content  area  goals  as  students  read,  research,  speak,  and  write  about  global  information.  Students  read  a broad 
range  of  fictional  and  non-fictional  texts  to  promote  understanding.  Regional  poetry,  music,  and  folk  literature 
enrich  data  gathered  from  encyclopedias,  newspapers,  and  maps.  Refer  to  Figure  2 for  an  abundant  list  of  goals 
addressed  in  this  particular  unit. 


Figure  2.  National  Standards  Addressed  in  this  Scenario 
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National  Standards  Addressed  in  this  Scenario 
SCIENCE 

National  Science  Education  Standards 

http://www.nap.edu/readingroom/books/nses/ 

Students  develop  understanding  of  populations,  resources,  and  environments. 

Students  develop  understandings  about  scientific  inquiry. 

Students  develop  abilities  necessary  to  do  scientific  inquiry. 

SOCIAL  STUDIES 

National  Council  for  the  Social  Studies 
http://www.socialstudies.org 

Students  develop  understandings  of  cultural  diversity. 

Students  develop  understandings  of  people,  places,  and  environments. 

Students  develop  understandings  of  global  connections  and  interdependence. 

MATH 

National  Council  of  Teachers  of  Mathematics 
http://www.nctm.org 

Students  understand  various  meanings  of  multiplication  and  division. 

Students^understand  the  effects  of  multiplying  and  dividing  whole  numbers. 

Students  develop  fluency  in  adding,  subtracting,  multiplying,  and  dividing  whole  numbers. 

Students  understand  the  need  for  measuring  with  standard  units  and  become  familiar  with  standard 
units  in  the  customary  and  metric  systems. 

Select  and  apply  appropriate  standard  units  and  tools  to  measure  length,  temperature,  etc. 

Students  collect  data  using  observations  and  experiments. 

Students  solve  problems  that  arise  in  mathematics  and  other  contexts. 

Students  organize  and  consolidate  their  mathematical  thinking  through  communication. 

LANGUAGE  ARTS 

National  Council  of  Teachers  of  English 

http://ncte.org 

Students  read  a wide  range  of  print  and  nonprint  texts  to  build  an  understanding  of  texts, 
themselves,  and  of  the  cultures  of  the  United  States  and  the  world  and  to  acquire  new  information.  Among 
these  texts  are  fiction  and  nonfiction. 

Students  read  a wide  range  of  literature  in  many  genres  to  build  an  understanding  of  many 
dimensions  of  human  experiences. 

Students  adjust  their  use  of  spoken,  written,  and  visual  language  to  communicate  effectively  with 
a variety  of  audiences  and  for  different  purposes. 

Students  conduct  research  by  gathering,  evaluating,  and  synthesizing  data  from  a variety  of 
sources  to  communicate  their  discoveries  in  ways  that  suit  their  purpose  and  audience. 

Students  use  a variety  of  technological  and  information  resources  to  gather  and  synthesize 
information  and  to  create  and  communicate  knowledge. 

Students  use  spoken,  written,  and  visual  language  to  accomplish  their  own  purposes  (e.g.,  for 
learning,  persuasion,  and  the  exchange  of  information). 
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ARTS 

National  Standards  for  Arts  Education 

http://www.didaxinc.com/standards/artstandards.html 

Students  relate  various  types  of  arts  knowledge  and  skills  within  and  across  the  arts 
disciplines.  This  includes  mixing  and  matching  competencies  and  understandings  in  art-making,  history 
and  culture,  and  analysis  in  any  arts -related  project. 

Students  develop  and  present  basic  analyses  of  works  of  artfrom  structural,  historical,  and 
cultural  perspectives,  and  from  combinations  of  those  perspectives.  This  includes  the  ability  to  understand 
and  evaluate  work  in  the  various  arts  disciplines. 

Students  communicate  at  a basic  level  in  the  four  arts  disciplines — dance,  music,  theatre,  and 
the  visual  arts.  This  includes  knowledge  and  skills  in  the  use  of  the  basic  vocabularies,  materials,  tools, 
techniques,  and  intellectual  methods  of  each  arts  discipline. 

TECHNOLOGY 

International  Society  for  Technology  in  Education 

http://cnets.iste.org 

A.  Students  are  proficient  in  the  use  of  technology. 

Students  use  technology  tools  to  enhance  learning,  increase  productivity,  and  promote  creativity. 

Students  use  productivity  tools  to  collaborate  in  constructing  technology -enhanced  models, 
prepare  publications,  and  produce  other  creative  works. 

Students  use  technology  to  locate,  evaluate,  and  collect  information  from  a variety  of  sources. 

Students  use  technology  tools  to  process  data  and  report  results. 


Use  Multiple  Authentic  Assessments 

Individualizing  instruction  is  necessary  so  that  a//  students  can  participate  in  this  unit.  If,  for  example,  the 
reading  levels  of  this  fifth  grade  class  range  from  first  to  seventh  grade,  one  textbook  would  not  be  at  an 
appropriately  challenging  level  for  all.  Teachers  must  choose  appropriate  media  and  technology  to  scaffold  learners 
with  varied  abilities.  According  to  Vygotsky  (1978,  p.86),  each  student  has  a zone  of  proximal  development,  which 
is  “the  distance  between  actual  development  level  as  determined  by  independent  problem  solving  and  the  level  of 
potential  development  as  determined  through  problem  solving  under  adult  guidance  or  in  collaboration  with  more 
capable  peers.”  In  order  to  determine  the  student’s  beginning  point  of  his/her  zone  of  proximal  development, 
teachers  should  use  multiple  forms  of  authentic  assessments.  The  unit’s  contextualized  activities  serve  as  tasks  for 
assessment.  Frederikson  & Collins  (1989,  p.31)  advocate  that  tasks  “included  within  an  assessment  system  would 
vary  from  structured  tasks  that  measure  students’  understanding  of  critical  concepts  or  skills  to  open-ended  tasks 
that  allow  students  to  demonstrate  special  knowledge  and  creativity”.  The  many  open-ended  products  throughout 
this  thematic  unit  provide  information  about  the  reading  levels,  learning  styles,  strengths,  and  weaknesses  of 
students  that  further  inform  media,  technology,  and  strategy  selection  during  the  unit  and  in  subsequent  units. 

Use  a Variety  of  Strategies  and  Activities  that  Address  Diverse  Learning  Styles 

Humans  in  all  cultures  use  multiple  intelligences  to  solve  problems  and  to  create  products  (Gardner  1983). 
Gardner’s  (1999)  intelligences  include  verbal-linguistic,  math-logic,  spatial,  bodily-kinesthetic,  musical, 
interpersonal,  intrapersonal,  and  naturalist.  Students  may  be  more  developed  in  certain  areas,  but  should  have  the 
opportunity  to  strengthen  all  intelligences.  The  eight  intelligences  should  be  considered  when  developing  lessons.  A 
variety  of  teaching  and  learning  strategies  can  be  used  to  facilitate  the  development  of  the  intelligences  throughout 
any  unit.  Refer  to  Figure  3,  which  details  the  ways  the  multiple  intelligences  are  addressed  in  this  context. 
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Figure  3.  Activities  Within  Each  of  the  Multiple  Intelligences 
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Provide  a Variety  of  Media  and  Technology 

Media  and  technology  are  powerful  tools  in  aiding  a teacher  in  individualizing  instruction.  Media  may  be 
selected  so  that  each  student  is  working  within  her  zone  of  proximal  development.  Multimedia  programs  increase 
acquisition  and  retention  of  information  by  enlisting  a variety  of  senses  as  students  gather  data  visually  and  aurally. 
Technology  has  a dichotomous  role  in  individualizing  instruction  because  it  gives  students  control  over  their 
learning  and  adapts  to  their  responses  accordingly. 

After  Letters  From  Felix  (Langen  & Droop,  1994)  is  read  and  discussed,  students  label  maps  and  chart 
Felix’s  path  of  travel.  Then,  students  form  heterogeneous  groups  to  further  research  one  of  the  countries  he  visited. 
Knowing  the  reading  levels  is  critical  for  this  activity.  Books  about  countries  that  are  written  with  larger  print, 
provide  more  pictures,  and  use  simpler  sentence  structures  scaffold  lower-level  readers.  The  series  that  includes 
Postcards  From  Russia  (Arnold,  1996)  and  Postcards  From  Mexico  (Arnold,  1996)  serve  this  purpose.  Books  that 
have  more  sophisticated  vocabulary  and  sentence  structure,  such  as  The  Portable  World  Factbook{Lyt,  1995),  are 
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more  appropriate  for  higher*level  readers.  These  books,  in  conjunction  with  software,  scaffold  group  members  as 
they  refine  their  research  skills  and  seek  details  about  their  assigned  country.  The  groups  use  IBM’s  World  Book 
Millennium  2000  CD-ROM  to  quickly  locate  pictures,  music  clips,  timelines,  and  maps.  This  software  is 
motivating,  appeals  to  diverse  learning  styles,  and  enables  students  to  quickly  access  detailed  information  about 
related  topics  by  linking  to  other  articles  on  the  disk’s  encyclopedia.  The  plethora  of  visual  information  allows 
children  with  limited  English  proficiency  or  learning  disabilities  to  gather  data  without  having  to  rely  on  text. 

Scaffold  Through  Monitoring  and  Guidance 

The  teacher  serves  as  a guide  through  this  research  process  and  as  Jonassen  (1999,  p. 230-6)  describes,  may 
provide  modeling  and  coaching  to  further  scaffold  learners.  The  teacher  may  demonstrate  note  taking,  highlighting, 
concept  mapping,  and  software  usage.  As  students  work,  the  teacher  monitors  and  informally  assesses  progress. 
Students  apply  their  new  knowledge  about  their  country  by  writing  a detailed  letter  to  Sophie  from  the  perspective 
of  Felix.  Writing  on  transparencies  facilitates  sharing,  as  the  letters  can  be  projected  onto  a larger  screen.  Groups 
may  use  a combination  of  text  and  drawings,  as  Felix  did,  to  convey  interesting  details  about  their  assigned  country. 

Groups  may  work  at  their  own  pace  to  complete  letters  and  share  accordingly.  If  others  finish  earlier,  they 
may  go  on  to  individually  research  a country  that  was  not  mentioned  in  the  story.  Software,  leveled  books,  maps, 
and  other  resources  scaffold  students  as  they  apply  their  research  skills  in  a new  assignment.  Text  and  drawings 
depicting  the  country’s  characteristics  are  added  to  index  cards  that  are  placed  into  envelopes.  Separating  and 
labeling  data  on  cards  by  categories  such  as  climate,  economy,  government,  h istory,  flag,  location,  and  population 
promotes  classification  skills.  Finished  envelopes  are  displayed  on  an  interactive  bulletin  board,  so  that  classmates 
may  explore  envelopes  and  read  about  other  countries.  For  students  with  writing,  spelling,  and  drawing  challenges, 
additional  software  such  2ls  Storybook  Weaver  Deluxe,  Start  Write,  Creative  Writer,  and  Word  may  be  used  so  that 
all  students  can  participate.  Printed  documents  can  be  inserted  into  the  envelopes  instead  of  index  cards. 

Some  students  may  finish  before  others.  As  they  do,  they  are  given  paper  cutouts  of  rabbits.  Students 
synthesize  information  from  their  envelope  research  projects  and  use  text  and  drawings  to  depict  their  country’s 
characteristics  on  one  side  of  the  rabbit.  They  use  resources  to  locate  and  express  the  same  kind  of  information 
about  their  home  state  on  the  opposite  side  of  the  rabbit.  The  rabbits  are  hung  with  string  from  the  ceiling  and 
provide  continuity  for  the  context  of  Felix,  the  world  traveler. 

Allow  Time  for  Student  Work  to  be  Self-Paced 

Imagine  this  fifth  grade  classroom:  as  students  complete  their  envelopes,  others  are  comparing  and 
contrasting  state  and  country  facts  on  paper  rabbits,  while  others  are  moving  on  to  the  next  task  at  their  own  pace. 
Carroll  (1963)  posits  that  students  learn  at  varying  rates  and  should  be  given  adequate  time  to  learn  new  material. 

He  suggests  that  as  students  master  objectives  before  others,  they  go  on  to  participate  in  enrichment  activities. 

Ample  time  to  “thoroughly  explore,  understand,  and  use  information  and  skills”  (Kovalik  1997;  1999)  leads  to 
comprehensive  understanding.  Opportunities  to  repeat  skills  within  the  same  theme  and  in  subsequent  themes 
extends  schema  and  reinforces  learning  (Reigeluth  1999;  Kovalik  1997;  1999). 

Encourage  a Learning  Community 

Next,  learners  are  grouped  into  pairs.  The  teacher  discusses  persuasion  and  how  it  is  used.  The  pairs 
choose  a country  to  research  and  use  the  facts  to  persuade  people  to  visit  this  country  via  a travel  commercial. 
Students  write  a persuasive  script,  include  five  to  ten  details  about  the  country,  construct  a backdrop  and  props,  and 
present  a travel  commercial  in  front  of  the  class  as  the  teacher  video  tapes  it  for  later  review.  This  experience 
provides  yet  another  form  of  assessment  for  the  teacher  and  allows  students  to  capitalize  on  and  increase  their 
intelligences  as  Gardner  (1983,  1999)  suggests.  In  this  case,  the  travel  commercial  engages  students  in  role-playing 
and  collaboration  to  increase  understanding. 

All  students  may  not  complete  a travel  commercial.  When  most  students  complete  most  tasks,  the  class 
explores  and  discusses  the  format  of  local,  national,  and  international  newspapers.  The  unit’s  culminating  project  is 
the  creation  of  a global  newspaper.  This  project  promotes  community  and  collaboration  through  problem  solving, 
planning,  and  coordination.  Since  knowledge  is  distributed  among  the  students  in  the  classroom  community  (Hewitt 
& Scardamalia  1998  ),  knowledge  is  shared  (Scardamalia  & Bereiter,  1994)  as  students  rely  on  each  other  to  build 
knowledge  and  complete  the  task  (Bell  & Winn,  2000,  p.  128-9).  This  project  promotes  individualized  instruction 
within  a community  setting.  Students,  who  are  most  interested  in  sports,  may  report  on  current  sporting  events  from 
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around  the  world.  Students  motivated  by  scientific  topics  may  report  on  global  weather  conditions  or  current 
innovations  in  medicine  and  technology.  Students  intrigued  by  numbers  and  patterns  (Gardner  1999)  may  report  on 
business  trends  or  compute  average  temperatures  around  the  world.  Others  may  use  narratives  (Gardner  1 999)  to 
express  insights  into  diverse  cultures.  Creating  a class  newspaper  is  a large  undertaking  and  requires  all  students  to 
contribute  to  ensure  its  completion.  This  implies  that  participants  have  a shared  interest  (Wenger,  1998)  and  are 
motivated  to  work  together  to  make  a collective  product  (Scardmalia  & Bereiter,  1994).  Software,  such  as  Creative 
Writer,  provides  a template  to  create  a newspaper  where  students  make  choices  about  the  number  of  columns  and 
the  type  of  font,  headlines,  borders,  and  graphics  to  be  used.  The  Internet  serves  as  another  scaffolding  tool  to  help 
learners  acquire  current  information  for  their  newspaper  articles.  The  final  product  serves  as  another  assessment 
piece  and  can  be  shared  with  younger  grades  or  community  members. 

Extend  Learning  and  Transfer  in  Subsequent  Themes 

One  measure  of  learning  is  the  ability  to  transfer  new  knowledge  to  other  contexts.  Though  not  proponents 
of  situated  learning  environments,  Anderson,  Reder,  and  Simon  (1996)  studied  the  work  of  others  to  determine  three 
components  related  to  transfer.  Their  findings  indicate:  (1)  there  are  varying  degrees  of  transfer,  including  negative 
transfer;  (2)  representation  and  degree  of  practice  are  major  determinants  of  transfer  between  tasks,  and  transfer 
between  different  domains  varies  directly  with  the  number  of  symbolic  components  shared  between  them;  and  (3) 
the  amount  of  transfer  depends  on  where  attention  is  drawn  during  instruction  (p.  6-8). 

To  increase  the  degree  of  transfer,  subsequent  themes  may  be  developed  that  review  and  expand  on  the 
skills  learned  in  the  first  unit.  Using  similar  characters  and  tasks  in  later  units  facilitates  the  transfer  of  knowledge 
and  provides  students  a framework  to  build  knowledge  upon  in  new  contexts.  For  example,  Felix’s  travels  may  lead 
to  further  inquiry  as  he  visits  the  moon  in  Felix  Explores  Planet  Earth  (Langen  & Droop,  1 996)  or  as  he  travels  back 
through  history  in  Felix  Travels  Back  in  Time  (Langen  & Droop,  1995).  Felix  may  even  be  introduced  to  another 
traveler  such  as  Flat  Stanley,  who  slips  through  the  mail.  Flat  Stanley  is  a young  boy  who  fits  into  envelopes 
because  he  was  flattened  by  a bulletin  board  temporarily  'mFlat  Stanley  (Brown,  1964;  1992).  Units  developed 
around  these  stories  and  characters  serve  to  extend  prior  knowledge  in  new  ways  so  that  students  may  progress 
toward  increasingly  difficult  objectives.  This  kind  of  connected,  threaded  curriculum  helps  learners’  brains  seek 
patterns  and  develop  a mental  organizer  to  extract  from  over  and  over  again  (Koval ik  1 997;  1 999). 

Applying  Theory:  Why  This  Works 

This  sample  unit  exemplifies  components  of  learning  models  espoused  by  experts.  This  unit  promotes 
knowledge  construction  through  hands-on  activities,  collaboration,  problem  solving,  and  active  engagement  (Abdal- 
Haqq,  1998).  The  varied  activities  address  diverse  learning  styles  by  appealing  to  visual,  oral,  auditory,  and  tactual 
senses  and  employ  multiple  intelligences  (Gardner  1999)  by  engaging  students  in  collaboration,  mathematical 
computation,  hands-on  activities,  and  role-playing.  Retention  of  material  is  greatly  increased  as  Sousa  (2001,  p. 95) 
explains  the  average  retention  rate  24  hours  after  lecture  is  5%;  after  reading  is  10%;  after  audiovisual  exposure  is 
20%;  is  30%  after  demonstration;  is  50%  after  discussion;  is  75%  after  practice  by  doing;  and  after  immediate  use  of 
learning  is  90%. 

Understanding  is  further  increased  through  the  pursuit  of  broad  goals  in  thematic,  integrated  contexts  where 
media  and  technology  scaffold  learners  as  they  complete  individual  and  group  projects.  Since  students  collaborate 
and  share  knowledge  at  different  points  in  the  unit,  they  develop  cooperation  skills  that  can  be  applied  later  in  real 
life  situations.  Students  simultaneously  learn  that  everyone  has  an  area  of  expertise  and  something  to  contribute. 
The  continual  application  of  skills  in  different  contexts  during  the  unit  leads  to  mastery  of  both  knowledge  and  skills 
in  a threaded  manner  that  can  be  revisited  in  subsequent  lessons.  Through  this  holistic  approach  to  instruction, 
teachers  facilitate  the  development  of  multifaceted  citizens  who  are  better  prepared  for  a global  society  and  its 
diversity,  which  is  the  ultimate  goal  of  education. 
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Abstract 

Asian  students  arriving  at  American  universities  are  subject  to  massive  cultural  change,  especially  if  it  is 
their  first  time  being  in  the  United  States,  Taking  concepts  from  Change  Theory  (Rogers,  1983  and 
Havelock  & Zlotolow,  1995)  and  combining  it  with  those  of  the  Affective  Domain  (Martin  & Briggs,  1986), 
the  authors  present  a cultural  adaptation  model  that  may  prove  useful  in  working  with  Asian  students  in 
American  Universities, 

Introduction 

Every  day  that  passes  presents  massive  change  to  students  and  instructors  at  all  levels  of  education. 
Nowhere  are  these  changes  more  evident  than  in  the  area  of  instructional  technology,  especially  at  the  postsecondary 
level.  This  scenario  is  doubly  complex  when  the  participants  are  from  countries  other  than  where  they  were  bom 
and  raised.  Instmctors  and  administrators  in  universities  can  take  steps  to  insure  a smooth  transition  from  one 
culture  to  the  next  by  addressing  a multiplicity  of  levels  that  impact  students  and  instmctors  from  abroad. 

This  paper  addresses  the  affective  domain,  and  how  concerns  related  to  the  initial  well  being  of 
international  participants  first  has  its  focus  on  basic  survival  skills.  We  readily  acknowledge  that  subsequent  focus 
will  ultimately  center  on  cognitive  skills  but  will  reserve  most  of  that  discussion  for  another  time  and  place. 

Background 

During  the  2000  AECT-Denver,  a group  of  Asian  students  shared  their  concerns  about  coming  to  the 
United  States  to  study  in  instmctional  technology  program  areas.  Using  Maslow’s  Hierarchy  as  a metaphor,  they 
traced  a number  of  areas  of  concern,  most  of  which  revolved  around  affective  domain  issues.  These  included  being 
accepted  by  their  American  peers,  working  in  harmony  with  teams,  knowing  what  to  wear,  shopping  for  ethnic 
foods,  developing  an  understanding  relationship  with  their  instmctors,  dating,  establishing  friendships,  and  more. 

Harvard’s  Dr.  Robert  Doyle  hosted  the  session  and  in  his  summary  remarks,  indicated  that  this  was  one  of 
the  most  valuable  and  insightful  sessions  that  he  has  attended  in  many  years.  This  paper  reviews  some  of  the  issues 
discussed  during  the  Denver  conference  and  connects  them  more  directly  to  the  affective  domain  and  change  theory. 
The  work  of  Bloom  (1956),  Cotton  (1995),  Martin  and  Briggs  (1986),  Rogers  (1983);  Havelock  & Zlotolow  (1995) 
contain  significant  implications  as  to  development  of  viable  strategies  to  ease  the  transition  of  international  students 
and  instructors  into  the  academic  and  personal  mainstream  of  the  countries  in  which  they  choose  to  study. 
Multicultural  theories  cited  by  a number  of  other  writers  connect  the  dots  related  to  affective  and  change  and  thus 
create  a better  understanding  (Lewis,  2000;  Mead,  1998;  Morrison,  Conaway  & Douress,  2000).  While  these  latter 
authors  mainly  address  multicultural  elements  related  to  business,  it  is  our  belief  that  many  graduates  of  instmctional 
technology  programs  will  find  themselves  facing  dilemmas  within  the  context  of  multinational  organizations. 

Differing  Issues  Identified 
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In  the  following  pages  the  reader  will  experience  something  very  remarkable  - articulate  concerns 
expressed  by  a number  of  students,  few  of  whom  were  able  to  speak  English  with  fluency  less  than  two  years  ago. 
They  are  now  not  only  able  to  write  with  considerable  precision  but,  when  asked  to  deliver  their  material  verbally, 
do  so  with  confidence  and  enthusiasm.  Those  same  students,  twenty-four  months  previous,  would  have  remained 
mostly  mute. 

The  reasons  for  such  changes  are  in  part,  due  to  caring  faculty  members  who  very  quickly  identify 
communication  problems  with  their  international  students.  Rogers  (1983,  p.  321)  describes  part  of  the  problem: 

Change  agent  [professors]  empathy  with  clients  [students]  is  especially  difficult  when  the  clients  are  very 
different  from  the  changer  agent;  we  expect  change  agents  to  be  more  successful  if  they  can  empathize  with 
their  client.  Although  there  is  very  little  empirical  support  for  this  expectation,  we  tentatively  suggest 
Generalization  9-4:  Change  agent  success  is  positively  related  to  empathy  with  clients. 

Rogers  adds: 

If  empathy  is  important  in  change  agent  effectiveness,  how  can  it  be  increased?  One  method  lies 
in  the  selection  of  change  agents;  those  who  have  been  in  the  client’s  role  are  better  able  to 
empathize  with  it.  For  example,  agricultural  agencies  often  seek  to  employ  change  agents  who 
have  come  from  farm  backgrounds. 

While  we  do  not  advocate  that  all  professors  who  teach  international  students  mirror  the  ethnicity 
of  their  learners,  we  feel  that  a faculty  member  who  has  prior  international  experience  is  more  inclined  to  be 
empathetic  than  one  who  has  not.  If  a professor  has  several  international  students  in  his  or  her  class,  there  is  a 
strong  likelihood  that  they  reflect  an  educational  experience  that  has  been  formal,  hierarchal,  and  pedagogically 
behaviorist.  Martin  and  Briggs  (1986,  p.  x)  describe  what  the  contemporary  educator  emphasizes  when  they  state: 
‘The  stronghold  of  behaviorism  has  lessened,  and  cognitive  inquiry,  cognitive-information  processing,  and 
humanistic  and  developmental  ideologies  and  theories  have  been  revised,  modified,  or  developed  to  meet  new 
needs.  Indeed,  what  is  an  international  student  to  think  when  confronted  with  a professor  who  actually  encourages 
their  students  to  think  and  do  for  themselves,  to  manage  much  of  their  own  learning? 

Martin  and  Briggs  (p.  x)  articulate  the  potential  for  future  problems  among  international  students  of  rigid 
academic  backgrounds  when  they  discuss  evolving  curricular  changes  in  that  “...the  focus  of  curricular  and 
instructional  content  has  broadened.  Affective  aspects  of  the  curriculum  are  now  being  incorporated  into  lessons 
and  units.  Attention  to  attitudes,  values,  ethics,  morals  and  the  self-esteem  of  learners  is  demanding  time,  energy, 
and  effort,  alongside  the  important  cognitive  dimensions  of  curricula.”  Given  these  changes  in  pedagogy,  how  do 
our  international  students  view  such  changes?  How  do  they  adapt  and  subsequently  for  some,  adopt  the  change  in 
how  they  view  both  studies  and  life  in  an  American  university? 

Indeed,  the  following  short  essays  written  by  several  Asian  students,  all  of  whom  brought  with 
them  pedagogical  rigidity,  provides  a near-alarming  look  at  how  they  perceive  their  professors  and  how  they  have 
adapted  to  such  change.  Appended  to  some  of  the  essays  are  recommendations;  others  declined  to  offer  such. 

There  is  some  degree  of  overlapping  ideas  presented,  primarily  due  to  the  reality  that  many  of  the  affective 
issues  they  raise  are  woven  into  the  larger  fabric  of  work  and  life  outside  the  classroom. 

Interaction  with  Peer  Students 
By  Corey  Lee 


Extensive  research  in  effective  teaching  and  learning  emphasizes  the  significance  of  social  interaction. 
According  to  Vygotsky  (1978)  social  development  theory,  social  interaction  is  vital  to  cognitive  development;  all 
higher-order  functions  originate  because  of  interaction  among  individuals.  In  light  of  this,  the  amount  and  type  of 
classroom  interaction  cannot  be  ignored  when  examining  Chinese  students’  learning  in  the  United  States. 

It  has  been  recognized  that  Chinese  students  pursuing  a higher  education  in  Western  countries  demonstrate 
limited  interaction  in  American  classrooms  (Tu,  2001).  Several  factors  might  contribute  to  Chinese  students’ 
inadequate  participation. 
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First,  the  foundation  of  Chinese  education  is  grounded  in  Confucianism,  in  which  values  and  morals  are 
much  different  from  Western  philosophy  (Brooks,  1997).  While  individualism,  assertiveness,  and  sometimes, 
aggressiveness  are  often  promoted  in  Western  society,  silence  and  temperance  are  valued  in  Chinese  society. 
Consequently  in  communication,  Chinese  places  the  “emphasis...  on  the  receiver  and  listening  rather  than  the 
sender  or  speech  making’’  (Yum,  1994,  p.83).  Therefore,  in  a classroom  environment,  Chinese  students  tend  to  be 
more  conservative  and  quiet  than  are  their  American  counterparts. 

Second,  the  presence  of  a rigid  hierarchical  system,  valued  in  Confucianism,  leads  Chinese  students  to 
regard  their  teacher  as  an  “all-knowing”  savant  on  stage  with  absolute  authority  over  the  subject  matter.  One’s 
teacher  is  the  sole  source  of  knowledge  in  Chinese  classrooms.  With  this  mentality,  discussion  and  interaction 
among  students  are  considered  trivial  and,  sometimes,  useless. 

Third,  even  when  Chinese  students  are  willing  to  participate  in  classroom  discussions,  they  fear  expressing 
what  may  be  erroneous  opinions  to  fellow  students,  which  leads  to  embarrassment  or  creation  of  a negative 
impression.  Therefore,  Chinese  students  are  often  considered  a silent  group  in  the  American  classroom.  Despite  the 
instructor’s  encouragement,  they  are  reluctant  to  express  their  feelings  or  opinions  and  to  participate  in  classroom 
discussions  (Tu,  2001). 

Finally,  reduced  or  no  interaction  by  Chinese  students  also  originates  from  their  limited  ability  to  speak 
English,  which  restricts  their  participation  in  the  classroom.  In  Taiwan  and  China,  English  language  education 
predominately  emphasized  reading  and  writing;  little  attention  was  paid  to  listening  and  speaking.  As  many  Chinese 
students  come  to  the  United  States,  they  lack  sufficient  communication  skills,  which  hinders  their  classroom 
participation.  Moreover,  the  swift  change  of  discussion  topics  in  the  classroom  poses  an  even  greater  problem  in 
this  situation  (Tu,  2001).  As  Chinese  students  struggle  to  conceptualize  the  subject  Just  discussed,  the  focus  changes 
to  the  next  topic. 

Having  a cultural  background  rooted  in  Confucianism  and  deficiency  in  English  capability,  Chinese 
students  exhibit  far  less  interaction  and  participation  in  American  classrooms.  By  being  sensitive  to  these 
difficulties  that  face  Chinese  students,  the  Western  teacher  can  play  an  active  role  in  facilitating  far  more  productive 
interactions  between  and  among  Chinese  and  American  students. 

Some  suggested  techniques  are: 

1.  Rather  than  call  on  a Chinese  student  in  front  of  the  class,  form  students  into  small  groups  and  let  the 
group  consensus  be  reported  by  one  among  them,  ensuring  that  the  person  providing  the  answer  is  not 
always  Western. 

2.  Have  students  write  an  answer  to  a question  and  randomly  call  on  all  students  in  the  class,  including 
those  who  are  Chinese.  Having  sufficient  time  to  frame  an  adequate  response  will  generate  more 
appropriate  replies,  thus  lessen  the  loss  of  face  feared  by  the  Chinese  student. 

3.  Have  the  students  create  a one-slide  Power  Point  response  to  a question,  assigned  in  advance.  Having 
prepared  before  class,  (and  the  Chinese  students  will  invariably  have  done  so),  reduces  anxiety,  and 
allows  all  students  to  contribute  equally. 

4.  Involve  Chinese  students  in  panel  discussions  wherein  they  are  members  of  a team  and,  as  such,  will 
feel  increased  responsibility  to  participate. 

5.  As  the  Chinese  students  become  more  familiar  with  other  members  of  the  class,  and  with  the 
techniques  listed,  they  eventually  should  be  called  upon  individually,  to  reply  in  class.  When  a teacher 
does  not  call  upon  the  Chinese  students  or  lets  them  off  with  low-order  responses,  no  one  is  served. 

Cultural  and  pedagogical  assimilation  for  Chinese  students  can  be  extremely  uncomfortable,  especially 
during  the  beginning  semesters  in  a Western  university  environment.  The  sensitive  teacher  will  accommodate  this 
discomfort  by  gradually  moving  their  Chinese  students  into  the  mainstream  of  classroom  activities.  By  showing 
concern  for  such  students  from  the  beginning,  the  good  teacher  can  make  the  educational  experiences  of  their 
Chinese  students  productive,  fulfilling,  and  yes,  even  enjoyable. 

On  Chinese  Self-Discipline 
By  Cheng-Chang  Pan 

When  it  comes  to  self-discipline  and  its  role  and  impact  on  multicultural  learning  environments,  Chinese 
students  tend  to  acquire  and  value  self-discipline.  According  to  Ritts  (2000),  self-discipline  as  well  as  other  qualities 
are  passed  on  to  new  generations  by  “training”  or  Chao-shun,  acquired  from  parents,  and  rooted  in  Confucianism. 

Confucius’  teachings  have  been  conveyed  to  his  descendants  culturally  and  individually,  and  have  had  a 
major  impact  on  the  ethnic  group.  Confucius  was  temperate  and  self-disciplined,  and  he  was  a man  of  wisdom  and 
good  manners  (Beck,  2001). 
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A perspective  of  social  learning  theory  holds  that  individuals  learn  by  interacting  and  observing  others  as  to 
how  new  behaviors  are  performed.  Kearsley  (2001)  supports  this  concept  when  he  states  that  “...individuals  are 
more  likely  to  adopt  a modeled  behavior  if  it  results  in  outcomes  they  value.” 

Chinese  students,  given  their  heritage,  are  inclined  to  place  authority  figures  in  high  esteem.  Teachers,  for 
instance,  are  highly  valued  by  students,  which  may  not  be  the  same  elsewhere. 

Given  the  author’s  experiences  in  Taiwan,  teachers  are  also  highly  valued  in  society.  Teaching  positions 
are  gaining  in  popularity  as  the  economy  shrinks  and  slows.  Affectively  speaking,  students  honor  themselves  when 
listening  to  the  authority’s  lectures,  so  they  model  the  authority  and  follow  the  authority’s  teaching.  Therefore,  they 
pride  themselves  in  being  a follower  of  the  master. 

Individually,  Chinese  students  are  apt  to  accomplish  tasks  designated  by  the  authority  to  demonstrate  their 
capabilities  and  thus  reflect  their  potential.  They  seldom  ask  questions  in  class,  in  part  because  they  feel  ashamed  to 
do  so.  Those  individuals  are  ones  who  wish  they  had  been  better  prepared  before  coming  to  class. 

When  Chinese  students  come  to  the  Western  world,  they  encounter  a great  cultural  impact.  While 
accommodating  themselves  to  the  foreign  country,  they  are  assimilating  themselves  into  this  melting  pot,  in  hopes 
they  will  adjust  to  the  new  environment  sooner  and  stand  out  among  their  fellow  classmates.  If  they  take  a hybrid 
stand  and  merge  selfdiscipline  from  the  East  with  assertiveness  from  the  West,  Chinese  students  will  have  a 
promising  future  in  the  foreign  environment. 

Professor-Student  Interaction 
By  Zhengzi  Wang 

As  an  Asian  student  intending  to  study  in  the  United  States,  a main  concern  was  how  to  establish 
contact  with  professors  in  the  department.  It  is  well  known  that  professors  are  key  sources  of  information  about 
assistantships,  or  even  scholarships.  E-mail  became  this  author’s  best  method  for  communication  with  the 
professors.  First,  it  was  possible  to  proofread  letters  to  professors  and  others  at  the  chosen  university  many  times. 
Secondly,  it  was  fast  and  there  was  no  need  to  make  an  appointment  with  the  professor.  Other  Asian  students  had 
the  same  experience.  Many  Asian  students  begin  to  communicate  with  their  professors  long  before  they  get 
admitted  to  the  universities  in  the  U.S.  The  author  corresponded  over  100  times  with  her  professor  before  coming  to 
the  United  States. 

The  professor  is  generally  the  first  and  the  only  person  that  an  international  student  can  comfortably 
contact.  Students  will  probably  throw  a wide  range  of  questions  at  their  future  professors,  from  admission 
procedures,  to  cultural  issues,  to  advice  about  the  apartment  rentals.  In  this  case,  a professor,  who  is  already  busy  in 
daily  life,  must  meet  the  challenge  of  efficiently  dealing  with  these  inquires.  Students  should  reserve  their  difficult 
questions — i.e.,  how  can  I get  an  assistantship?  — for  professors,  while  answering  simpler  questions  using  the 
variety  of  resources  available,  such  as  the  Internet,  friends,  books,  their  chosen  university’s  website,  etc. 

China's  cultural  system  is  based  not  on  the  strength  of  the  individual,  but  on  the  pattern  of  relationships 
maintained  by  all  people.  In  communication,  the  Chinese  put  emphasis  on  the  receiver  of  messages  and  on  listening 
rather  than  on  the  sender  (Brooks,  1 997).  As  Bond  & Hwang  (1986)  have  pointed  out,  "the  western  starting  point  of 
the  anomic  individual  is  alien  to  Chinese  considerations  of  man's  social  behavior,  which  see  man  as  a relational 
being,  socially  situated  and  defined  within  an  interactive  context." 

Due  to  the  utterly  different  cultural  factors,  a lot  of  Asian  students  experience  difficulty  when  adjusting 
themselves  to  the  American  classroom  culture.  Silence  is  not  golden  anymore.  American  peers  are  eloquent, 
talkative,  and  humorous  in  class.  Sometimes,  we  feel  much  pressure  competing  with  them  for  the  professor's 
attention.  A lot  of  negative  self-talk  generally  results,"  I don't  understand  my  professor's  words,  what  shall  I do?”  “ 

I don't  agree  with  the  professor,  so  should  I confront  him  right  away?”  “I  want  to  go  to  the  restroom,  but  can  I leave 
when  the  professor  is  talking?”  “Oh,  she  speaks  so  fast!  I couldn't  even  understand  a word!”  So,  many  Asian 
students,  like  the  author,  don’t  participate  in  class  at  the  beginning,  even  if  they  can  speak  English  well. 
Consequently,  some  concerns  arise.  “What  will  the  professor  think  about  me?  Dumb?  Slow-leamer?”  “Does  he 
know  I am  smart  enough  to  understand  what  he  said,  but  just  didn't  speak  out?" 

There  is  solid  social  background  behind  this  thinking.  Confucian  philosophy  permeates  the  whole  of 
Chinese  society.  Confucius  believed  that  a hierarchical  system  was  essential  to  the  harmonious  well  being  of 
society.  This,  in  turn,  is  reflected  in  the  Chinese  classroom.  Students  regard  their  teacher  as  all  knowing,  and  the 
absolute  authority  on  the  subject  matter.  Due  to  the  rigid  teacher-student  relationship,  Chinese  teachers  are  under 
severe  pressure  not  to  make  mistakes,  not  to  misguide  students,  and  not  to  be  criticized.  Such  behavior  on  their  part 
allows  the  teacher  to  maintain  their  "all- knowing"  and  "ever-correct"  status.  It  is  the  duty  of  students  to  give  the 
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utmost  respect  to  their  teacher.  Thus,  to  ask  questions  of  the  teacher  is  tantamount  to  questioning  the  position  of  the 
teacher,  and  therefore  is  not  a feature  of  Chinese  classrooms.  Rigid  order  and  formality  are  the  main  features  of  the 
Chinese  learning  environment.  Chinese  teachers  and  students  know  that  the  classroom  is  a place  where  serious 
knowledge  is  taught. 

The  major  concern  of  Asian  students  is  that  their  American  professors  think  less  of  them  because  they  don’t 
speak  a lot.  Professors  need  to  be  aware  that  classroom  participation  does  not  always  mean  “shouting  and  yelling 
and  hands  up”.  A lot  of  times,  Asian  students  participate  in  class  in  a different  way.  They  do  more  critical  thinking 
in  their  minds.  Sometimes,  it’s  better  to  give  the  floor  to  Asian  students  rather  than  waiting  for  a general  response. 
On  the  other  hand,  students  shouldn’t  get  too  self-conscious  about  themselves  speaking  English  with  an  accent. 

They  should  practice  English  more  to  overcome  the  initial  fear  of  speaking  English  in  public. 

A rewarding  college  experience  is  typically  viewed  as  students  in  a classroom  busily  engaged  in  the  pursuit 
of  learning  with  their  professors.  Missing  from  this  picture  are  faculty  and  students  interacting  outside  of  the 
classroom  environment.  Yet  this  less  common  image  is  equally  important  to  a successful  college  experience 
(Maestas,  2000). 

In  the  author’s  college  days  back  in  China,  we  seldom  had  social  contacts  with  the  professors.  Mostly,  we 
met  our  professors  in  class,  discussing  serious  academic  issues.  Teachers  are  considered  as  an  authority  figure,  and 
it  was  hard  to  look  at  them  as  a person. 

In  the  United  States  Chinese  students  can  access  their  professors  through  e-mail  24  hours  a day,  7 days  a 
week,  discussing  academic  issues.  Also,  some  professors  like  to  entertain  international  students  in  their  homes,  to 
celebrate  special  occasions.  This  is  a good  time  and  place  to  talk  to  the  professors,  letting  him/her  know  you  as  a 
person.  In  our  program,  all  the  international  students  are  regularly  invited  as  guests  to  our  professors'  house  to 
celebrate  Thanksgiving,  Christmas,  Chinese  New  Year,  Autumn  Festival,  etc. 

Asian  students  should  establish  an  academic  contact  as  well  as  a social  contact  with  the  professors.  In  this 
way,  they  can  get  more  attention  and  demand  more  interest  from  professors  about  their  studies. 

The  fact  that  all  the  international  students  pay  far  more  tuition  more  than  local  residents  is  a reality. 
Consequently,  rightly  or  wrongly,  we  expect  quality  education  as  well  as  cultural  enrichment.  Lack  of 
communication  and  a restrictive  cultural  notion  of  propriety  seem  to  be  the  major  obstacles  to  professor-student 
interactions,  as  each  attempts  to  find  ways  to  balance  tradition  and  modernization.  It  takes  time  for  both  sides  to 
realize  the  gap  and  to  improve  mutual  understanding. 

The  Purpose  of  Seeking  a Graduate  Degree  in  America 
By  Alan  Ku 

Every  year  hundreds  of  thousands  of  Chinese  students  come  to  America  for  advanced  studies.  Why  do 
they  choose  to  fly  thousands  of  miles  from  home,  leaving  their  families  and  friends  behind  to  go  to  an  unfamiliar 
country  where  language,  culture,  life  style,  and  language  are  so  different?  This  essay  has  a focus  on  their  objectives 
in  engaging  in  study  abroad  and  examines  different  motivations  between  those  students  studying  abroad  and  their 
American  counterparts. 

First,  American  and  Chinese  students  place  different  utilitarian  value  to  a graduate  diploma.  To  American 
students,  obtaining  a graduate  degree  means  they  have  more  professional  knowledge  and  skills  in  their  field, 
therefore  they  have  a better  opportunity  to  upgrade  their  existing  position  and  earn  a higher  salary.  But  to  Chinese 
students,  a graduate  degree,  especially  from  the  United  States,  not  only  increases  the  likelihood  of  employment;  but 
also  promotes  social  status.  This  is  because,  in  Chinese  society,  the  diploma  he  or  she  holds  largely  determines  a 
person’s  dignity  and  value.  Therefore,  although  obtaining  a Masters  degree  takes  much  time  and  money,  they  still 
choose  this  path. 

The  second  factor  that  drives  Chinese  students  is  pressure  from  the  family.  In  the  traditional  Chinese 
culture,  children  receive  continuing  support  from  their  family  throughout  their  schooling.  Chinese  parents  consider 
it  is  their  responsibility  to  do  so.  In  addition,  Chinese  parents  encourage  their  children  to  pursue  higher  degrees  and 
sometimes  even  make  that  decision  for  their  children.  Most  Chinese  students  follow  their  parents’  wishes  when 
attending  graduate  school.  This  is  both  a natural  and  inseparable  condition  in  Chinese  society. 

On  the  contrary,  American  students  do  not  expect  their  parents  to  pay  for  their  tuition  for  higher  education. 
American  parents,  unlike  their  Chinese  counterparts,  do  not  expect  their  children  to  look  after  them  in  the  sunset  of 
their  life.  The  result  is  that  most  American  students  make  their  own  decisions  when  choosing  their  major  and  are 
not  in  a hurry  to  obtain  a graduate  degree! 


The  Classroom  Environment 


By  Terry  Tao 


According  to  MacAulay  (1990),  “A  classroom  environment  consists  of  the  intellectual,  social,  and  physical 
conditions  within  or  exogenous  to  a classroom  that  influence  the  learning  situation.” 

It  is  not  difficult  to  identify  a variety  of  physical  factors  that  affect  how  students  feel  about  their  classroom 
environment.  Following  is  a sampling  of  available  information  that  supports  the  need  to  consider  many  aspects  of 
the  classroom  climate  in  order  to  create  an  environment  for  maximum  learning. 

Environments  that  are  psychologically  or  emotionally  negative  inhibit  learning  (Midjaas,  1984).  Studies 
have  found  direct  correlations  between  students'  cognitive  styles  and  responsiveness  to  environmental 
characteristics.  In  responsive  classroom  environments,  students’  achievement  increased  (Dunn  and  Dunn  1992). 
Many  classrooms  in  American  universities  are  better  equipped  than  in  other  countries.  Normally  in  every  American 
university’s  classroom,  there  are  projectors,  computers,  and  a central  air  conditioning  system. 

Fewer  students  in  a larger  instructional  space  enhance  the  use  of  available  resources.  Spaces  where  high 
density  and  few  resources  exist  add  to  increased  student/teacher  conflict  and  poor  student-to-student  behavior 
(Midjaas,  1984).  When  comparing  the  population  density.  United  States’  class  sizes  are  much  smaller  than  in  Asian 
countries.  All  these  advanced  physical  classroom  environments  enrich  the  quality  for  studying  by  international 
students  in  the  United  States. 

The  amount  and  quality  of  classroom  light  is  conducive  to  1)  greater  comfort  and  contentment;  2)  a more 
cheerful  environment;  3)  more  concentration  and  a greater  desire  to  work;  4)  less  fatigue  and  therefore  fewer  related 
side-effects,  such  as  laziness,  bad  posture,  nervousness,  and  lack  of  interest;  and  5)  greater  accuracy  and  neatness 
(Hathaway,  1983).  Use  of  blues  and  greens  fosters  feelings  of  relaxation.  Use  of  red  and  orange  colors  in 
instructional  areas  induces  anxiety  behaviors  (Weinstein,  1981).  Color  affects  changes  in  mood,  emotional  states, 
psychomotor  performance,  muscular  activity,  rate  of  breathing,  pulse  rate,  and  blood  pressure  (Hathaway,  1983). 

When  referring  to  a cheerful,  cozy  and  comfortable  classroom  design,  there  are  two  minor  differences 
between  United  States  university  classrooms  and  Asian  university  classrooms. 

In  American  universities,  many  classrooms  have  no  windows.  This  amazes  me  Chinese  students  when 
first  entering  the  classroom.  There  are  two  primary  reasons  for  this  difference:  projected  visuals  will  be  viewed 
better  without  windows  and  the  scene  outside  of  the  window  will  not  distract  students.  Actually,  there  is  no  clear- 
cut  relationship  between  the  noises  associated  with  school  and  student  performance  on  task  (Arends,  1994).  In  Asia, 
it  is  very  typical  construction  design  to  put  some  windows  in  the  classroom,  and  plant  some  trees  outside  of  the 
window  to  block  the  students’  view.  In  most  Asians  minds,  the  best  place  for  studying  (sometime  they  call  it 
training)  is  a place  with  nice  natural  environment  and  fresh  cool  air.  The  film  “Crouching  Tiger  Hidden  Dragon”, 
depicted  a good  Asian  sense  as  to  the  ideal  training  environment. 

Another  difference  is  the  design  of  both  seats  and  desks.  In  American  universities,  many  classrooms  do  not 
have  desks  for  students.  Frequently  there  is  only  a piece  of  board  combined  with  the  chair.  However,  in  Asian 
Universities,  every  classroom  has  desks  for  students.  Asian  students  received  extensive  training  on  how  to  write 
when  they  are  growing  up.  For  example,  when  Chinese  children  are  doing  their  homework,  they  are  required  to  sit 
up  straight  and  put  two  hands  on  the  paper.  In  order  to  adjust  the  children’s  study  habits  and  develop  good  attitudes, 
correcting  their  sitting  and  writing  style  is  a good  way  in  the  Asian  parents  and  teachers  minds  to  maintain  a positive 
study  attitude.  Gradually  the  connection  between  desks  and  writing  is  solidified.  Therefore,  some  professors  will 
feel  curious  that  many  Asian  students  seldom  take  notes  during  the  class,  when  there  are  no  real  desks  available. 

Personally,  this  author  feels  that  the  physical  side  of  the  classroom  environment  is  not  as  big  a problem  for 
international  students  as  compared  to  the  “soft”  side  of  the  classroom  environment. 

What  is  the  intellectual  and  social  environment  within  the  classroom?  What  takes  place  during  the  time 
before  the  class  begins,  during  the  class  break,  and  after  class?  We  can  call  it  social  time.  Normally  in  every 
country,  students  will  cherish  it  and  make  friends  with  their  classmates.  They  will  naturally  divided  into  several 
little  groups  and  chat  with  each  other. 

Rogers  (p.  71)  felt  that  ...’’interpersonal  networks  among  peers  energizes  the  [diffusion}  process.  He  goes 
on  to  state  that  one  of  the  important  roles  of  [diffusion]  leaders  (professors?)  was  to  interconnect  the  spatially  related 
cliques  of  the  village”  and  that”  a general  conclusion  from  who-to-whom  studies  is  that  space  and  social  distance  are 
the  main  determinants  of  who  talks  to  whom  in  diffusion  networks  [classes].”  Herein  lies  a sensitivity  point  for 
professors  to  ponder. 

For  international  students,  these  social  times  are  frequently  the  most  uncomfortable.  There  are  several 
reasons  that  lead  to  them  felt  embarrassed  or  uncomfortable.  First,  the  language  barrier  is  the  main  reason.  For 
international  students,  English  is  their  second  language.  There  are  so  many  different  accents,  slang,  and  terms. 
Second,  unfamiliar  topics  are  another  reason.  Most  of  the  time,  international  students  feel  they  have  nothing  in 
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common  to  talk  about  related  to  these  topics.  Therefore,  it  is  not  strange  that  most  international  students  just  remain 
quiet. 

This  kind  of  classroom  environment  will  definitely  influence  the  learning  of  international  students.  As 
international  students,  we  need  do  lots  of  work  from  our  side.  We  cannot  complain  that  it  is  “discrimination”. 
Learning  the  culture  and  communication  skills  is  as  important  as  learning  academic  content.  Of  course,  some  times 
we  also  need  the  professors’  help  to  open  up  some  possibilities  for  students  to  know  each  other.  It  is  also  very 
important  to  establish  collaborative  learning.  Different  professors  some  times  really  can  make  a big  difference. 

This  author  took  some  courses  in  which  he  made  several  very  good  friends;  and  he  also  took  some  courses  in  which 
he  did  not  know  any  of  his  classmate’s  names. 

There  are  several  other  things  about  classroom  environment.  For  example,  an  instructor’s  personality  often 
determines  the  type  of  classroom  environment.  Tonelson  (1981)  had  no  doubt  about  the  interconnectedness  of 
teacher  personality  and  the  learning  atmosphere  in  a classroom.  He  suggested  a mechanism  whereby  the  teacher’s 
personality  can  affect  student-learning  outcomes  through  the  psychological  environment  of  the  classroom.  He 
believed  that  this  environment  was  essentially  the  product  of  the  kinds  of  interactions  teachers  have  with  students. 

He  argued  that  the  character  of  the  teacher  is  translated  into  the  working  social  atmosphere  of  the  classroom,  which 
in  turn,  influences  students.  That  type  of  atmosphere  sets  the  stage  for  learning.  For  international  students,  an 
instructor’s  personality  can  significantly  interfere  with  their  learning.  A good  instructor  must  demonstrate  a 
repertoire  of  appropriate  interpersonal  and  pedagogical  skills.  In  the  classroom,  students  learn  knowledge,  adapt  to 
the  instructor’s  personality,  absorb  the  culture,  and  master  communication  skills  simultaneously. 

Conclusions 

Martin  and  Briggs  (p.  449),  in  summarizing  their  seminal  work,  include  the  statement  that,  “we  believe  that 
the  cognitive  evaluations  that  individuals  attach  to  feelings,  the  category  emotions,  underlie  the  individual’s 
developrrent  of  attitudes,  interests,  social  competence,  and  other  related  affective  categories.  Feelings,  we  propose, 
are  generally  attached  to  emotions,  at  least  as  the  way  we  have  defined  them.”  It  is  this  development  within  our 
international  students  that  each  of  us  is  charged  to  cultivate.  The  essays  we  include  in  this  article  address  each  of 
these  concerns  and  more. 

The  process  of  interweaving  multicultural  elements  with  change  and  affect  theories,  while  complex  in  its 
execution,  will  hopefully  give  insight  as  to  the  nature  of  innumerable  problems  faced  by  students  and  instructors 
when  they  go  abroad  for  study  and  teaching.  What  we  have  indicated  as  being  problems  faced  by  Asian  students 
are,  in  reality,  those  faced  by  all  students,  no  matter  from  where  they  may  have  come. 
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Introduction 

Like  most  aspects  of  life,  educational  technology  is  multi-dimensional  in  its  relations  to  people  and  the 
environment;  educational  technology  can  be  good  and  bad  and  everything  in-between.  This  group  of  experienced 
educational  technologists  has  taken  a balanced,  honest,  and  sobering  look  at  people's  (including  our)  uses  of 
technologies  for  education,  and  we  realize  somewhat  uncomfortably  that  often  some  of  our  connections  to 
educational  technology  have  questionable  consequences  for  ourselves  and/or  the  people  we  intend  to  help.  Our 
connections  are  ethically  questionable.  As  representatives  of  the  AECT  Ethics  committee,  we  have  chosen  to  present 
(and  publish)  a discussion  regarding  these  occasions  when  we  have  found  ourselves  involved  in  ethically 
challenging  situations.  Each  author’s  primary  aim  is  to  relate  an  instance  when  he  or  she  was  associated  with  an 
ethically  questionable  or  hurtful  use  of  educational  technology  and,  then,  to  encouraged  discussion  among  others 
regarding  that  instance  or  a similar  one  from  their  own  experience. 

Further,  each  presenter  will  discuss  her  or  his  instance  in  terms  of  ethical  principles  and  how  the  principles 
should  guide  educational  technologists'  work.  The  ethical  principles  come  primarily  from  AECT's  and  AERA's 
(American  Educational  Research  Association)  professional  codes  of  ethics.  The  idea  of  iatrogenics  (where  that 
which  people  create  to  overcome  a "problem"  makes  the  problem  worse)  and  the  paradox  of  human  consciousness 
(it  hurts  as  much  as  it  helps)  also  are  used  to  explain  our  ethically  questionable  uses  of  educational  technology. 

In  terms  of  the  primary  aim  of  this  paper,  the  first  author  tells  the  story  of  the  ethical  dilemmas  encountered 
as  an  instructor  in  intellectual  property,  when  she  finds  herself  dealing  with  students  who  violate  copyright  law  in 
her  classes.  The  next  author  tells  of  instances  in  one  school  district  where  she  has  seen  too  little  being  done  to  offset 
the  fact  that  technology  is  promoted  without  enough  thought  and  conversation  among  all  concerned  parties.  The 
final  author  describes  the  dilemmas  he  faces  as  an  educational  technology  teacher-educator  between  encouraging 
pre -service  to  learn  technology  skills  and  technology  integration  and  the  technical,  pedagogical,  and  curricular 
realities  that  the  teachers  will  face  once  they  are  in  classroom. 

The  formal  part  of  our  AECT  presentation  ended  with  these  questions: 

• Once  you  discover  that  you  or  someone  else  is  doing  harm  with  educational  technology,  what  should 
you  do? 

• If  you  continue  with  the  status  quo,  do  ethical  breaches  move  from  the  realm  of  ethical  mistakes  that 
might  be  forgiven  to  intentional  behavior  that  requires  punishment  of  some  sort? 

In  our  AECT  session  we  each  spoke  briefly  about  the  examples  that  follow.  We  then  opened  the 
presentation  for  discussion,  which  was  lively.  Each  author  remarks  on  the  discussion  in  the  section  following  his  or 
her  example. 

First  Example:  The  Ethical  Dilemma  Created  by  Copyright  Violations  in  the  College  Classroom 

My  ethical  dilemma  stems  from  the  muddy  world  of  copyright  law.  As  a faculty  member  who  teaches 
classes  in  intellectual  property,  I am  very  aware  of  the  many  instances  of  copyright  violation  that  can  occur  in  a 
college  or  university  setting  - especially  those  among  students  in  my  own  college  classrooms.  This  comes  to  a head 
during  assignments,  oral  and  written  presentations,  many  times  when  1 ask  for  informational  materials  or  media  to 
support  a particular  belief,  argument,  or  point  of  view.  Invariably,  one  or  two  students  feel  that  the  best  way  to 
support  their  argument(s)  is  to  take  large  amounts  of  factual  material  directly  from  one  or  more  sources,  copy  it,  and 
give  each  class  member  a copy.  This  is  often  in  violation  of  the  Fair  Use  principles  of  copyright  law  and/or  without 
obtaining  permission  from  the  owner  or  creator  of  the  work  from  which  they  are  borrowing  or  copying.  In  most 
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instances,  other  than  if  they  are  taking  or  have  taken  my  intellectual  property  classes,  these  students  have  no  concept 
that  they  are  violating  copyright  law.  Additionally,  because  copyright  law  is  so  complex,  the  other  classes  I teach  do 
not  usually  have  any  aspect  of  copyright  law  as  part  of  the  curriculum. 

Thus,  when  as  an  instructor  I am  suddenly  confronted  with  these  violations,  I am  not  sure  what  I should  do. 
Certainly,  this  might  be  a "teachable  moment,"  whereby  I go  into  the  whole  of  copyright  and  what  that  particular 
law  means  to  us  as  students  and  educators.  However,  in  most  instances,  I am  not  happy  with  the  time  that  this  will 
take  away  from  the  subject  of  the  class  in  progress.  In  addition,  I also  am  reluctant  to  embarrass  the  student(s)  who 
may  be  in  the  process  of  violating  the  law  as  they  hand  out  whole  articles,  book  chapters,  etc.,  sans  the  permission 
needed  to  copy  these  legally  for  class.  Thus,  I find  myself  in  the  unique  position  of  realizing  that  there  might  be  an 
infringement  in  progress,  but  not  wanting  to  put  the  student  on  the  spot  by  questioning  him/her  in  the  midst  of 
his/her  presentation.  Also  in  many  instances,  such  a subject  "leap"  could  redefine  the  whole  atmosphere  of  the  class 
in  progress.  Thus,  my  ethical  dilemma  is:  how  much  "policing"  should  a faculty  member  do  in  a situation  that 
appears  to  be  within  the  realm  of  copyright  violation,  and  yet  probably  will  not  go  outside  the  classroom?  What 
about  the  time  needed  to  change  the  foci  of  classes?  Should  I pursue  a subject  area  that  will  change  the  focus  of  a 
class,  especially  if  it  interrupts  student  presentations? 

Solution? 

Given  the  current  foci  on  intellectual  property  issues  in  technology,  especially  the  electronic  environment, 
perhaps  the  answer  to  my  dilemma  lies  within  my  syllabus.  By  placing  a copyright  statement  in  the  syllabi  of  each 
class,  and  going  over  the  syllabus  at  the  first  class  meeting,  a short  discussion  on  copyright  and  violations  will 
automatically  be  covered.  Thus  at  later  dates,  should  a copyright  violation  occur,  it  may  be  possible  to  make  a 
statement  at  the  end  of  the  class  about  how  we  as  students  and  educators  need  to  observe  and  follow  intellectual 
property  laws.  In  this  way,  student  presentations  and  discussion  would  not  be  interrupted  and  a short  teachable 
moment  could  still  occur.  What  do  you  think  about  this  dilemma? 

Second  Example:  Consulting  with  a School  District 

My  ethical  dilemma  involves  the  role  many  of  us  play  in  our  communities:  educational  technology 
consultant  to  the  school  district.  I have  often  been  asked  to  provide  consultation,  training  or  staff  development,  serve 
on  technology  committees,  and  so  on,  in  the  communities  around  our  university.  What  this  has  often  meant  has  not 
been  a “pure”  consultation  opportunity;  it  has  been,  instead,  an  invitation  to  help  promote  or  encourage  the  use  of 
technology  in  the  dis  trict,  and  to  help  teachers  use  more  technology  in  their  classrooms. 

Why  is  this  a dilemma?  Isn’t  helping  to  promote  educational  technology  a big  part  of  our  role?  The 
problem  comes  for  me  because  1 am  often  not  asked  to  offer  suggestions  on  how  best  to  integrate  technology,  or 
which  technology  to  purchase  and  use.  My  expertise  is  requested  after  the  moneys  have  been  spent  and  the 
equipment  is  in  the  buildings,  and  teachers  are  “resisting”  using  it  enough  to  suit  the  administration  who  offered  its 
purchase.  1 know  1 am  being  asked  to  help  promote  technology  use,  to  provide  staff  development  on  the  best  ideas 
for  technology  integration,  and  to  support  the  purchase  and  use  of  technology  already  in  place.  I am  not,  really, 
being  asked  to  consult  on  purchases,  or  question  expenditures,  or  facilitate  discussions  with  teachers  about  whether 
or  not  they  should  use  technology  in  their  classrooms.  That  much  is  clear  from  the  requests  I receive.  And  whether 
or  not  these  are  consultancies  for  which  I am  being  paid  (they  usually  are  not),  I feel  some  responsibility  to  “give  the 
client  what  he  wants”,  to  provide  the  service  requested. 

The  dilemma  comes  from  my  reluctance  to  continually  promote  technology  without  questioning  its 
appropriate  use  and  the  problems  that  can  occur.  School  district  administration  professionals  are  not  asking  me  to 
help  them  explore  the  inappropriateness  of  a technology  purchase,  nor  are  they  expecting  me  to  help  teachers 
examine  the  problems  they  will  probably  encounter  in  attempting  to  integrate  technology.  These  administrators  want 
to  hear  the  joys  and  positive  effects,  not  the  problems  or  tribulations  of  innovation  or  the  possible  negative  effects  on 
the  learning  environment  of  adding  too  much  technology  without  being  prepared.  What  should  I do?  How  can  I take 
this  opportunity  to  help  teachers  and  administrators,  and  still  have  my  voice  heard,  and  included  the  next  time?  How 
far  should  a consultant  go  in  suggesting  the  reasons  that  the  requested  work  is  answering  the  wrong  questions,  or 
that  the  solution  already  identified  may  create  more  problems  than  it  solves? 

Solution? 
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The  audience  at  our  presentation  enthusiastically  identified  with  this  problem.  Many  of  them  encouraged  a 
similar  solution:  be  prepared  to  offer  your  expertise,  both  sides  of  the  issues,  regardless  of  the  original  request.  Some 
pointed  out  that  it  was  impractical  not  to,  since  the  consultant  could  be  blamed  later  if  an  innovation  did  not  work 
out.  “You  told  us  that  if  we  did  it  this  way,  learning  would  improve”,  that  sort  of  attack  could  follow.  However,  the 
case  example  also  reminded  members  of  the  presentation  audience  that  we  have  an  awesome  responsibility  as  well 
as  the  technology  skills.  We  are  ethically  bound  to  help  people  examine  their  choices  and  to  question  the  efficacy  of 
the  solutions  being  posed.  If  we  try  to  “please”  by  merely  teaching  teachers,  or  supporting  technology  professionals, 
we  are  only  doing  half  our  jobs.  The  discussion  encouraged  us  all  to  consider  the  long-range  use  of  technology,  the 
problems  teachers  and  students  face  with  technology  integration,  and  to  be  honest  and  open  about  our  doubts  or 
reluctance  when  we  are  being  asked  to  consult  with  schools. 

Third  Example:  Technology  Training  and  Teacher  Education 

My  dilemma  arises  from  my  role  as  a teacher-educator  of  educational  technology  responsible  for  teaching 
the  only  required  technology  course  at  my  institution.  In  this  five  week  summer  course  I introduce  students  to  a 
myriad  of  technical  and  pedagogical  skills  including  web  development,  constructing  computer-based  presentations, 
creating  spreadsheets,  evaluating  educational  software  and  web  sites,  using  electronic  communications  like 
discussion  boards,  learning  Fair  Use  guidelines  and  creating  technology -infused  lesson  plans.  The  course  involves  a 
tremendous  amount  of  content  and  skills  building  in  a short  time.  In  the  context  of  taking  this  course  students  are 
taking  a full  schedule  of  other  courses  as  they  begin  their  year-long  teacher  education  program.  In  short,  my  students 
are  quite  overwhelmed. 

I find  myself  in  an  ethical  conflict  as  I consider  how  much  information  that  I should  provide  my  students 
about  the  realities  of  what  they  may  face  in  the  schools  in  which  they  will  teach.  These  realities  could  include: 

• computers  that  are  unreliable,  out-of  -date  and  that  frequently  crash 

• Internet  connections  that  are  slow  or  inconsistent 

• filtering  software  that  blocks  sites  that  they  wish  their  students  to  visit 

• computers  that  don’t  have  the  software  that  teachers  wish  to  use  or  are  locked  down  so  that 
teachers  can’t  add  their  own  legally  purchased  software  without  going  through  a long,  often 
unsuccessful  bureaucratic  process 

• curricula  that  is  driven  by  standardized  tests  and  that  leave  little  room  for  creative  uses  of 
technology 

• lack  of  professional-development  opportunities  related  to  technology 

These  are  just  a few  of  the  roadblocks  to  using  technology  that  teacher’s  may  face.  In  addition  to  reflecting 
on  how  much  of  this  information  to  provide  students,  there  is  also  a body  of  literature  about  educational  technology 
that  is  critical  of  the  uses  of  computers  in  the  classroom.  This  literature  addresses  a variety  of  topics  including  how 
technology  takes  resources  away  from  other  aspects  of  schooling  including  the  upkeep  of  buildings,  how  a heavy 
reliance  on  technology  in  education  can  detract  from  other  ways  of  knowing  and  learning,  and  as  the  previous 
example  notes  how  it  is  not  always  appropriate  to  use  technology  because  of  various  developmental  and  pedagogical 
reasons. 

What  I have  encountered  in  grappling  with  this  dilemma  over  the  years  is  that  delivering  too  much  of  the 
“bad  news”  and  critiques  discourages  students  who  are  overwhelmed  and  need  to  be  motivated  as  they  make  their 
way  through  the  course  content.  Part  of  how  I address  this  is  by  informing  students  of  these  challenges  in  a situated 
manner  throughout  the  course. 

In  addition  to  this  example,  during  my  presentation  I mentioned  a similar  dilemma  that  I have  in  my  role  as 
a technology  integration  trainer  for  faculty  in  my  college.  Because  of  tremendous  administrative  and  accreditation 
pressures  faculty  are  being  compelled  to  demonstrate  how  they  integrate  technology  into  their  teaching.  As  a trainer 
I must  always  measure  how  much  information  that  I provide  faculty  about  the  amount  of  technical  information  that 
they  must  learn  to  implement  their  ideas.  I want  to  be  realistic  without  being  discouraging. 

Solution? 

In  discussing  this  issue  with  audience,  the  consensuses  emerged  that  one  is  obligated  to  present  students 
and  faculty  with  the  full  picture.  Audience  members  observed  that  by  providing  all  the  information  about 
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educational  technologies  that  students  and  faculty  can  make  fully  informed  choices  about  whether  and  how  to 
integrate  technology.  In  general,  1 agreed  with  these  observations,  however  this  solution  did  not  address  for  me  the 
issue  of  motivation  in  the  face  of  ISTE  (International  Society  for  Technology  in  Education)  standards  and  NCATE 
(National  Council  for  Accreditation  of  Teacher  Education)  accreditation.  Technology  integration  in  some 
institutions  is  no  longer  a choice  for  teachers  and  faculty,  but  a requirement  for  tenure  and  promotion.  The  dilemma 
still  remains  for  me  of  how  to  maintain  a realistic  and  critical  representation  of  educational  technologies  in  the  face 
of  institutional  and  societal  pressures  that  discourage  us  from  taking  steps  away  from  the  forward  momentum  of 
technological  innovation  and  diffusion. 

Conclusions 

Professionals  in  educational  technology,  professors  of  our  field,  future  teachers,  and  students  entering  our 
field,  all  need  to  continue  to  discuss  and  debate  the  ethical  dilemmas  in  which  we  find  ourselves.  The  AECT  Ethics 
Committee  encourages  all  of  us  to  read,  understand,  consider,  and  debate  the  Ethics  Statement  and  to  help  explore 
the  ethical  choices  we  all  make.  The  AECT  website  (http://www.aect.org)  can  provide  further  information  on  our 
Ethics  and  the  TechTrends  Q^o\\xmn  on  ethics  and  other  AECT  publications  can  provide  further  examples  of  cases  for 
discussion  by  all  of  us,  especially  those  of  us  who  are  teaching  future  professional  in  our  field. 


Systems  Planning  for  Faculty  Development:  Integrating  Instruction  with  Technology 


Henryk  R.  Marcinkiewicz 
Pennsylvania  College  of  Technology 


A process  for  implementing  a faculty  development  program  is  described.  The  goals  are  integrating 
instruction  with  technology  and  professional  development.  The  model  is  based  on  systems  planning  and  draws 
together  learning  factors  and  implementation  strategies.  Outcomes  include  understanding  the  feasibility  and  critical 
concerns  of  this  initiative. 

The  key  word  in  this  discussion  is  “planning.”  The  field  of  instructional  technology  is  rich  with  models  and 
systems  for  planning  or  designing  that  are  particularly  useful  for  planning  a program  for  faculty  development 
because  the  intent  of  such  a program  is  to  be  instructional.  The  organization  of  this  discussion  includes  five  parts:  1) 
Developing  a Vision;  2)  Looking  at  Successful  Practice;  3)  Implementing  the  Vision;  4)  Realities;  and  5)  The 
Future.  Pertinent  considerations  are  discussed  and  relevant  models  are  referenced  to  help  guide  the  planning  for  a 
faculty  development  program. 

Part  1:  Developing  a Vision 

Imagine  your  vision  of  the  institution  in  a few  years.  See  yourself  being  there.  See  how  faculty 
development  is  done.  Have  a vision.  Appreciate  the  substantive  nature  of  the  change  in  the  organization  from  the 
current.  See  real  changes  that  result  from  effort  (Marcinkiewicz,  2001). 

Because  the  implementation  of  a successful  faculty  development  program  requires  a change  in  the 
organization  of  an  institution,  it  can  be  daunting.  Knowing  that  you  can  accomplish  this  is  the  most  gratifying  part. 
And,  it  is  a good  idea.  Perform  a short  test  on  yourself,  “What  would  be  the  consequences  if  you  did  not  proceed?” 
See  if  you  would  be  satisfied  with  the  consequences. 

Goals 


There  are  two  goals  of  this  discussion,  your  appreciation  for  and  understanding  of  processes  that  will  lead 
to  the  institutionalization  of  faculty  development,  and  the  integration  of  technology  with  instruction. 

Short  History  of  Faculty  Development 

Faculty  Development  programs  in  higher  education  are  a recent  phenomenon.  One  of  the  earliest  was 
formed  about  twenty  years  ago  at  the  University  of  Michigan.  But  there  is  a reason  that  development  offices  have 
not  been  commonplace;  simply,  the  absence  of  development  is  a reflection  of  the  expectations  of  higher  education 
on  the  professorate.  Most  often  professors  have  been  hired  because  of  their  expertise  in  a subject  area  or  because  of 
their  researcher.  The  contradiction  is  obvious,  professors  were  expected  to  know  something,  teaching  was  either 
simple  enough  for  an  intelligent  person  to  figure  out  or  it  was  less  important  than  subject  expertise  or  research. 

A change  in  thinking  in  higher  education  about  teaching,  the  roles  of  professors,  their  preparation  for 
teaching,  and  their  continuing  learning  about  teaching  is  reflected  in  the  establishment  of  faculty  development 
programs.  One  of  the  motivations  for  faculty  development  may  very  well  have  been  the  changed  role  of  students. 
Students  are  expressing  their  expectations  of  professors  one  of  which  is  for  the  professor  to  be  an  effective  teacher. 

There  are  a variety  of  names  for  faculty  development  programs.  They  are  usually  organized  as  centers  and 
their  titles  often  include  one  or  more  (sometimes  all)  of  these  terms:  faculty,  development,  teaching,  learning,  and 
excellence.  In  summary,  we  have  come  to  expect  teaching  from  teachers  and  we  are  providing  the  means  for  faculty 
members  to  learn  about  teaching  until  they  are  done  learning,  which  is  to  say  that  they  will  learn  continuously. 

Part  2:  Looking  at  Successful  Practice 

The  first  planning  tool  discussed  is  a gap  analysis  that  identified  the  areas  to  be  addressed  at  my  former 
institution,  Ferris  State  University  in  Michigan.  The  program  there  was  the  Center  for  Teaching,  Learning,  & 
Faculty  Development  (CTL&FD).  The  discussion  of  the  CTL  & FD  is  organized  by  the  structures  of  a gap  analysis: 
the  optimal  state,  the  current  state,  followed  by  the  response  and  the  outcomes. 
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Optimal  State — Wanted... 


The  optimal  state  included  several  areas.  First,  there  \vas  a desire  for  widespread  interaction  among  faculty 
on  a decentralized  campus.  Second,  there  was  a felt  need  for  increased  communication.  Third,  there  was  a desire  for 
reflection  about  teaching.  Fourth,  there  was  a need  for  training  for  faculty.  And,  finally  there  was  a need  for 
professional  development. 

In  general,  we  wanted  to  communicate  and  to  learn.  The  gaps  reflected  the  view  taken  by  Peter  Senge  in  the 
“Fifth  Discipline,”  that  the  goal  of  communication  is  learning.  We  wanted  to  learn  and  our  communication  channels 
were  not  functioning  well. 

Current  State — Had... 

The  current  state  comprised  several  areas  throughout  the  university. 

The  faculty  were  frustrated  with  the  absence  of  a means  for  them  to  improve  or  even  to  establish  skills  in 
teaching.  Many  faculty  members  were  directly  hired  from  business  and  industry  and  so  unlike  their  colleagues  who 
were  hired  from  other  academic  institutions,  they  were  not  even  familiar  with  the  setting.  Both  sets  of  colleagues  did 
share  a lack  of  preparation  for  teaching. 

There  was  a sense  of  fragmentation  at  the  university  that  was  evidenced  by  activities  that  were  occurring  at 
one  college  that  had  relevance  for  the  entire  institution  but  were  not  known  by  members  outside  of  the  college. 

There  had  been  proposal  for  a faculty  development  program  which  was  a reflection  of  pent-up  demand  for 
learning.  These  are  the  artifacts  of  a functioning  system  that  was  not  acting  cohesively. 

Response — Did. . .Structure... 

The  Office  of  the  Vice-President  for  Academic  Affairs  approved  the  establishment  of  the  CTL  & FD. 

These  are  some  of  the  structural  elements  that  were  introduced. 

Varied  schedules  of  activities 

Commonly  available  times  (“holy  hours”).  Thursdays  from  1 1 AM  to  1 PM  had  been  a part  of  the  existing 
schedule  reserved  for  university-wide  activities  such  as  faculty  development.  The  CTL  & FD  encouraged  adherence 
to  the  schedule. 

Training  sessions  were  scheduled  for  short  and  long  periods.  Some  sessions  lasted  three  to  five  days.  We 
favored  very  short  sessions  lasting  fifty  minutes  that  focused  on  a single  concept  and  which  were  repeated  several 
times.  The  aim  in  scheduling  was  to  offer  as  many  opportunities  for  attendance  as  possible,  in  other  words,  to  be 
accommodating  to  the  widely  varying  schedules  of  faculty  members. 

Special  interest  groups 

We  actively  made  the  acquaintance  of  faculty  members  sometimes  directly  visiting  offices  to  introduce 
ourselves  in  an  effort  to  build  awareness  for  the  activities  of  the  CTL  & FD  and  to  learn  about  the  interests  of  faculty 
members.  These  activities  led  to  the  formation  of  special  interest  groups  that  were  supported  by  the  CTL  & FD  with 
coordination  of  meeting  times  and  places  and  the  provision  of  refreshments.  Some  groups  that  formed  included 
interests  such  as  public  speaking,  web  development,  grant  writing,  and  scholarly  writing. 

Individual  consultations 

The  interactions  between  the  CTL  & FD  staff  and  the  faculty  were  not  only  in  workshops  or  grovap 
meetings,  we  also  scheduled  individual  consultations  with  professors  as  well  as  meetings  with  entire  departments. 

Faculty  as  facilitators 

There  was  variety  among  our  presenters  as  well.  The  richest  pool  was  from  the  faculty  itself.  In  fact,  faculty 
appreciated  very  much  the  opportunity  to  learn  from  each  other. 

Response — Did. ..Philosophy 
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We  listened  carefully  and  often  to  faculty  members  and  our  practices  were  true  to  learner-centered 
philosophy  in  an  iteration  referred  to  as  “Progressive  Education.”  We  ensure  that  the  teaching  is  tailored  to  the  needs 
of  the  learner  and  we  provide  opportunities  for  the  faculty-learner  to  practice  during  the  instruction.  The  philosophy 
of  our  practice  is  faculty-as -learner-centered  development. 

The  CTL  & FD  enjoyed  an  independent  status  and  was  not  affiliated  with  any  of  the  colleges,  but  rather 
reported  directly  to  the  office  of  the  vice-president  for  academic  affairs.  As  with  any  effective  instruction,  we  tried 
to  follow  up  on  any  activities  that  we  offered.  For  example,  if  a person  participated  in  a workshop  on  syllabus 
preparation,  he  or  she  was  asked  to  be  available  to  advise  colleagues.  Or,  sometimes  we  would  simply  send  an  e- 
mail  message  asking  whether  the  workshop  helped  or  continued  communication  with  the  individual. 

Response — Did. . . Incentives... 

The  well-established  rule  about  incentives  is  that  they  must  be  the  appreciated  by  the  receiver  in  order  to  be 
valued  and  motivating.  With  faculty  members,  money  was  not  the  primary  incentive.  We  tried  a variety  of  financial 
incentive  plans,  and  large  amounts  of  money  did  not  seem  more  desirable  than  a nominal  sum  that  would  be 
considered  a professional  honorarium.  When  asked,  faculty  members  first  sought  the  learning;  any  tokens  of 
recognition  were  appreciated  if  they  expressed  a value  for  the  faculty  as  professionals.  One  requisite  that  was 
universally  effective  was  the  provision  of  food.  Another  incentive  for  faculty  especially  at  institutions  that  are 
decentralized  is  the  opportunity  to  meet  their  colleagues  which  helps  to  build  esprit  de  corps.  Perhaps  the  primary 
incentive  for  faculty  to  continue  learning  is  their  innate  love  of  learning.  Faculty  members  are,  after  all,  educators.  A 
final  practical  point  is  that  participation  in  development  activities  counted  towards  promotion  and  tenure  and  post- 
tenure review  consideration. 

Response — Did. . .Activities . . . 

The  CTL  & FD  organized  its  activities  in  three  main  areas  mirroring  those  conducted  by  Centers  around  the 
country  as  suggested  by  the  Professional  and  Organizational  Development  network. 

Professional  development 

This  category  of  activities  focuses  on  the  career  of  a professor  such  as  mentoring,  grant  writing,  scholarly 
writing,  promotion  and  tenure. 

Instructional  development 

This  category  of  activities  focuses  on  teaching  and  learning  including  instruction,  assessment,  learner 
characteristics,  and  instructional  media. 

Support  of  the  institution 

This  category  of  activities  focuses  on  the  professor  participating  in  and  understanding  the  system  of  the 
institution,  including  its  mission,  history,  town  and  gown  relationships,  the  community,  and  the  personnel  makeup  of 
the  institution. 

Achieved... 

The  faculty  and  the  CTL  & FD  together  achieved  very  active  participation  and  growth  in 

programming. 

• 70%  participation  rate 

There  were  over  450  full-time  tenure  track  faculty  members  who  were  organized  in  a collective 
bargaining  unit  for  which  there  was  no  mandate  to  participate  in  faculty  development  activities. 
Reporting  of  statistics  like  these  were  facilitated  by  the  database  created  for  recording  our 
activities.  It  served  us  well  when  we  reported  to  three  accreditation  agencies. 

• 1 1%  in  1 5- week  courses 

One  example  of  participation  was  the  over-capacity  sign  up  for  10  and  15  week  CTL  & FD  led 
courses. 
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• 97%-ile  of  worldwide  users  of  WebCT 

The  demand  for  a web  presence  was  explosive  and  faculty  driven.  Once  a course  management 
system  was  introduced,  faculty  members  sought  training  and  were  productive  in  delegating  some 
of  their  instruction  to  the  system. 

• Hesburgh  certificate  of  excellence 

The  CTL  & FD  was  awarded  TIAA-CREF’ s award  for  a program  aimed  at  developing 
undergraduate  faculty  to  improve  student  learning. 

• Distributing  our  services 

The  CTL  & FD  began  to  train  faculty  at  area  institutions. 

• Exemplars  for  other  institutions 

We  communicated  with  several  institutions  to  advise  about  setting  up  programs. 

• 80-90%  faculty  satisfaction  rating 

The  CTL  & FD  enjoyed  high  approval  ratings  on  university-wide  surveys. 

• Goal  of  continuing  learning 

The  strong  support  by  the  faculty  was  an  indicator  of  progress  toward  the  goal  of  continuing 
learning. 

• Yearlong  transition  program  for  new  faculty 

An  accomplishment  was  the  initiation  of  a highly  successful  program  for  new  faculty. 

• No  indifference 

It  was  very  gratifying  that  the  mood  on  campus  lacked  indifference.  There  was  strong  and 
expressed  interest  among  the  faculty. 

Part  3:  Implementing  the  Vision 

The  discussion  about  establishing  a program  at  your  institution  is  organized  by  the  structures  of  a gap 
analysis  similar  to  Part  2,  the  optimal  and  current  states,  but  implementation  steps  are  described  at  length.  The 
conditions  are  all  assumed  but  considered  to  be  very  likely. 

Optimal  State — Wants... 

Because  of  the  universal  interest  in  these  two  areas  among  institutions  of  higher  education,  it  can  be 
assumed  that  your  institution  would  also  be  want  to  operate  a faculty  program  for  professional  development  and 
instructional  development. 

Current  State — Has... 

The  appraisal  of  the  actual  state  at  your  institution  reveals  that  the  following  conditions  exist.  There  is 
motivation,  limited  funding,  and  an  estimated  retirement  of  50%  of  your  faculty  within  the  next  10  years.  These  are 
fairly  safe  assumptions  and  reflect  the  reality  of  most  institutions  of  higher  education. 

Implementation — Can  Do... 

Use  instructional  system  design  strategy  to  organize  your  planning  especially  because  your  focus  will 
include  instruction.  There  are  a variety  of  models  available  and  in  general  it  is  probably  less  important  which  model 
you  use  than  that  you  do  use  a model  to  keep  track  of  your  thinking.  These  well  known  planning  activities  organize 
this  discussion  about  implementation:  assess,  design,  develop,  implement,  and  evaluate.  Bear  in  mind  that  while, 
they  are  presented  in  sequential  order,  untoward  conditions  may  affect  the  order  in  which  you  may  actually  be  able 
to  conduct  planning.  Try,  however,  to  establish  your  evaluation  methods  and  techniques  to  be  congruent  with  the 
objectives  you  establish. 

It  is  assumed  that  you  would  have  conducted  a process  to  determine  your  needs  and  that  was  described  in 
the  previous  section  detailing  the  results  of  a gap  analysis.  The  next  processes  in  which  you  would  engage  are  design 
and  evaluation. 

Design  Steps... Objectives 
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The  following  is  an  example  of  suitable  objectives  for  a faculty  development  program. 

• Learn  about  your  field 

• Learn  about  teaching  & learning 

• Serve  the  institution 

• Reflect  upon  teaching 

• Understand  students 

• Engage  in  scholarship 

The  following  is  an  example  of  suitable  objectives  for  a development  program  that  also  serves  staff  and 
administration.  Recall  that  traditionally  higher  education  administrators  come  from  faculty  ranks  and  do  not  have 
formal  training  in  management  or  leadership. 

• Learn  about  the  field 

• Learn  communication  technology 

• Serve  the  institution 

It  is  important  to  focus  on  the  ultimate  objective  of  any  faculty  development  program,  that  is,  the 
development  of  students.  Example  objectives  for  students  include... 

• Improved  learning 

• Retention 

It  is  also  important  possible  and  desirable  to  identify  objectives  for  the  institution.  Example  objectives 
include... 

• Competitiveness 

• Retention  of  faculty  & students 

• High  Esteem 

• Esprit  de  Corps 

Design  Steps... Evaluation 

Some  purposes  of  evaluation  are  to  gauge  whether  your  program  is  meeting  its  objectives  and  what  factors 
or  conditions  are  contributing  to  its  operation.  To  facilitate  your  assessment  and  evaluation  processes,  it  is 
recommended  that  you  establish  a database  that  includes  your  activities  as  well  as  the  various  units  of  your 
institution  and  the  members  of  the  faculty.  Gather  data  counts  of  participants  but  also  count  repeat  participants.  For 
example,  there  may  be  two  activities  with  10  participants,  but  they  may  be  the  same  10  individuals.  It  is  desirable  to 
know  whether  the  participants  are  the  same  or  not.  Consider  various  quantitative  and  qualitative  measures. 

As  part  of  your  evaluation  process,  count  your  failed  activities  as  well  as  your  successful  ones.  Make  plans 
for  what  you  will  do  as  your  activities  succeed  or  fail  to  meet  your  objectives. 

A valuable  practice  to  introduce  for  your  faculty  is  the  Small  Group  Instructional  Diagnosis  (SGID) 
procedure  (Clark  & Redmond,  1982).  This  simple  procedure  is  conducted  early  in  the  semester  in  the  absence  of  the 
professor.  It  is  a kind  of  focus  group  activity  intended  to  get  early  and  candid  feedback  from  the  students.  It  can  be 
effective  and  is  inexpensive.  We  assess  using  the  SGID  format  after  each  of  our  activities,  then  annually  we  do  a 
more  detailed  survey.  Assessment  is  a burden  and  intrusion  and  so  we  limit  our  use  of  them.  Try  to  use  varied  other 
formats  to  gather  information.  Use  phone  calls  or  face-to-face  conversations  to  get  input  from  faculty.  Note  how 
often  and  for  what  reason  a faculty  calls  the  senior  administration;  it  must  be  pretty  urgent  for  a call  to  be  made  to 
senior  administration 

Design  Steps... Planning  Considerations 

As  part  of  the  design  process  it  is  recommended  to  consider  four  factors  affecting  learning.  These 
were  suggested  by  Jenkins  (1979)  and  Bransford  (1979):  media,  assessment,  the  learners’  characteristics,  and 
instruction. 

Media 

These  are  pertinent  questions  to  consider. 

• What  media  will  you  use? 
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• Who  or  what  will  deliver  instruction? 

• What  media  are  available  to  you? 

• What  do  you  currently  use? 

• What  can  you  not  use? 

• What  is  your  institution’s  plan  for  providing  equipment  to  faculty,  for  classrooms,  for  web- 
enhanced  instruction,  for  complete  online  instruction? 

Learners*  Characteristics 

You  need  to  consider  you’re  the  faculty  and  staff  administrators  and  their  needs  as  a group,  as  subgroups, 
and  as  individuals.  Some  guiding  questions  are... 

What  are  their  particular  needs? 

• Who  are  faculty? 

• How  are  people  in  the  academy  unique? 

• Are  there  characteristics  particular  to  members  of  the  institution? 

• Are  they  similar  to  academics  in  CA  (diverse)  or  NV  (from  many  areas  and  newly  arrived)? 

• Who  are  managers  and  administrators? 

• To  what  kind  of  information  do  they  respond? 

• Emphasize  practical  over  theoretical  pedagogy. 

• People  outside  of  a field  tend  to  lack  the  interest  and  the  preparation  to  participate  in  a theoretical 
discussion  at  length. 

• Low  ceilings  in  faculty  career 

• Build  steps  into  the  career  path.  Consider  starting  a senior  learning  community.  See  the  work  of 
Milton  Cox  at  Miami  University,  Ohio. 

• Administrators  often  untrained  in  mgmt. 

Characteristics  of  Faculty  Members  and  Other  Academics 

• Highly  intelligent 

• Love  of  learning 

• Strong  sense  of  collegiality 

Ask... 

• How  would  you  characterize  academics  as  compared  to  airline  personnel,  or  the  military? 

• What  are  their  career  expectations? 

• Why  would  they  want  to  learn  more? 

Instruction 

Methods  of  instruction  need  to  be  considered.  After  all,  the  learners  must  do  something  in  order  to  learn. 
What  will  they  do?  How  will  you  orchestrate  the  activity  so  that  it  is  effective?  These  are  guiding  questions... 

• In  what  activities  will  they  engage? 

• What  scheduling  tactics?  Formats?  Instructors? 

• What  methods? 

• How  will  you  pace  the  activities. 

Instruction — Best  Practices 

Some  practices  that  worked  well  at  the  CTL  & FD  included  the  following. . . 

• Learning  communities.  Focus  on  activities  that  demand  communication  and  collegiality. 
Remember  the  strong  sense  of  collegiality  among  academics. 

• Learner-centered  instruction:  customize  and  demonstrate  leaming(Progressive  ed.) 

• Collegial  activities... panels.  Allow  the  faculty  to  practice,  often.  Customize  the  training  to  meet 
their  needs. 
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Patience  & non-threatening  practice.  Allow  faculty,  adult  learners,  time  to  learn  and  many 
opportunities  to  practice. 


Content 


The  questions  that  will  guide  planning  for  content  are... 

• What  do  faculty  need  to  know  about? 

• What  do  administrators  need  to  know  about? 


Faculty 


Just  as  you,  the  planner  of  instruction  for  faculty  need  to  consider  four  factors  of  learning,  so  do  your 
faculty  need  to  learn  about,  understand  and  address  four  factors  of  learning  in  the  instruction  that  they  plan  and 
implement.  Once  again,  four  factors  of  learning  are  media,  assessment,  instruction,  learners’  characteristics, 
administration,  & advising 

Administrators 

For  administrators  focus  on  three  elements  of  leadership:  management,  team-building,  and  creating  the 
future.  These  are  suggested  by  Sullivan  and  Harper  in  their  book,  “Hope  is  Not  a Method,”  in  which  they  discuss 
how  the  armed  forces  were  reorganized  from  dealing  with  predictable  conditions  to  unpredictable  conditions. 

Part  4:  Realities 

You  may  still  be  wondering,  “Is  this  all  possible?”  Gilbert  (1978)  has  suggested  three  areas  that  must  be 
accounted  for  if  competence  is  to  occur.  In  this  case,  these  conditions  need  to  be  addressed  in  order  for  faculty 
development  to  occur.  The  areas  are  contributed  to  externally  by  the  institution  and  internally  by  the  individuals 
aspiring  to  competent  behavior. 

The  Model  of  Competence  suggests  the  external  and  Internal  provisions  of  the  following  conditions: 

• Motivation  & Incentives 

• Equipment  & Ability 

• Information  & Understanding.. .and  training 

In  application,  the  model  can  be  completed  as  follows.  The  institution  provides  incentives,  equipment,  and 
expresses  expectations.  The  individual  becomes  motivated,  adapts  for  competency,  and  learns. 

• Costs 

• There  are  a variety  of  costs  involved. 

• Effort 

• Personnel 

• Time 

• Restating  the  mission 
Not  all  costs  are  directly  financial. 

One  cost  in  restating  the  mission  may  be  the  most  motivating:  giving  value  to  the  learning  that 
professionals  do. 

Part  5:  The  Future 

Your  endeavor  may  result  in  the  following... 

• Unique  program 

• Change  in  way  of  life 

• Achieved  goals 
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Several  models  relevant  to  planning  a faculty  development  program  were  discussed  in  the  context  of  an 
instructional  design  model.  The  discussion  followed  a sequence  from  developing  a vision  to  looking  at  past  practice 
to  implementation  to  reality  checks  to  the  future.  Some  points  that  deserve  restating:  a faculty  development  program 
requires  an  institutional  change,  the  implementation  requires  a vision  as  well  as  effort  as  would  be  expected  of  any 
successful  and  rewarding  effort.  To  paraphrase  a chief  of  the  Oglala  Lakota  Native  Americans,  ..vision  with  work 
is  only  a dream;  work  without  a vision  is  a drudgery,  but  together  they  can  change  the  world, , , 
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Introduction 

‘'Culture  is  a complex,  abstract,  and  pervasive  matrix  of  social  elements  that  functions  as  an  all -encompassing  form 
of pattern  for  living  by  laying  out  a predictable  world  in  which  an  individual  is  firmly  oriented.  " 

Richard  Porter  & Larry  Samover 

Offering  a predictable  world  and  orienting  students  in  that  controlled  environment  offer  the  faculty  a new 
level  of  challenge.  Technology  makes  this  task  possible.  The  ultimate  goal  of  this  project  challenges  us  to  create 
and  provide  a communicative  environment  in  which  is  most  natural  to  L2  and  C2*"*.  We  seek  to  teach  in  the  world 
where  the  instructors  and  students  coexist  in  non-artificial  simulation. 

The  computer-assisted  classroom  is  no  longer  a simulation,  but  a real  life.  The  idea  of  “Learning  can  be 
undertaken  in  state-of-the-art  classrooms,  but  can  also  be  integrated  into  the  living  arrangements”  (Gilman)  is 
obsolete.  In  21*‘  century  life,  our  lives  globally  connect  to  each  other  with  technological  network.  The  networked 
life  is  the  living  life.  In  that  environment,  one  of  the  most  important  issues  is  the  understanding  of  L2  and  C2  to 
make  successful  communications. 

Since  the  fall  semester  of  1995,  students  at  Baylor  University  taking  Japanese  and  students  at  Tokyo 
Institute  of  Polytechnics  have  had  opportunities  to  communicate  each  other  and  learn  together  at  in  a reakworld 
situation.  Until  the  fall  2000,  the  format  limited  them  to  e-mail  communication.  Today,  we  have  begun  a new 
adventure  in  learning  culture  and  communicating  appropriately  with  the  colleagues  from  the  other  side  of  this  planet 
using  other  Internet  tools. 

Software  and  Hardware  Used 

Communication  software,  iVisit,  utilizing  web-based  camera  provides  an  instant  visual,  oral,  and  aural 
transoceanic  communication  along  with  text  chat.  This  method  brings  a classroom  halfway  across  the  globe  next 
door.  We  use  this  web-based  free-of-charge  software  in  the  introductory  and  concluding  discussions.  Due  to  our 
hardware  limitation,  each  school  used  one  web-based  camera. 

FirstClass  offers  students  to  communicate  in  the  manner  of  a bulletin  board  service.  It  allows  students  to 
leave  messages,  possibly  enhanced  with  graphical  and  audio  files,  anytime  of  the  day  from  anywhere  in  the  world. 

It  allows  students  to  read  the  postings  and  share  ideas.  Contrary  to  iVisit,  FirstClass  gives  time  to  students  to  think 
and  examine  what  to  post.  The  client  software  is  free-of-charge  and  supports  many  different  platforms  and 
languages. 

Criteria 


For  this  study,  Baylor  University  selected  the  parameter  range  of  students  to  be  3*^^  semester,  150  contact 
hours,  or  higher.  Tokyo  Institute  of  Polytechnics  has  selected  students  from  Intercultural  Communication  Studies 
who  have  been  studying  English  for  at  least  6 years.  Students  from  both  universities  interacted  in  LI  and  L2  to 
complete  the  task. 

The  Project  Time  Table 

1.  Introductory  iVisit  session 

At  the  end  of  September  2000,  the  first  iVisit  session  took  place.  Baylor  class  met  at  7p.m.  to 
accommodate  the  time  difference  of  14  hours.  Students  answered  Pre -Project  questionnaires  to  examine  what 


LI  is  the  first  language  and  L2  is  the  second  language.  Cl  is  the  culture  of  the  students  own,  and  C2  is  the  culture 
which  student  is  learning  as  the  second  culture. 
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technological  and  cultural  experiences  they  have  had  and  how  those  experiences  will  change  them  at  the  completion 
of  the  project. 

During  this  one-hour  introductory  session,  instructors  shared  the  intention  and  schedule  of  the  project  first. 
Then  many  students  took  the  opportunity  to  introduce  thems  elves  and  asked  some  questions  such  as  weather, 
fashion  fads,  popular  music,  politics,  and  other  current  events.  Some  questions  were  prepared  and  others  were 
spontaneous. 

2.  Research  in  Cl 

The  myth  exists  among  Japanese  that  folks  in  Texas  still  use  horses  as  the  main  transportation  and  chases 
and  feeds  livestock  all  day.  Then  there  is  a myth  among  people  in  the  U.S.  that  Japanese  still  walk  around  with 
katana  swords  in  their  sash  and  drink  sake  with  geisha  girls.  Although  such  misunderstanding  exists,  students  have 
very  shallow  understanding  of  their  own  culture  to  help  others  understand  the  truth.  Therefore  we  assigned  students 
to  research  in  Cl  to  deepen  their  knowledge  before  learning  C2  any  further.  Japanese  students  took  the  subject  of 
bushi/samurai  while  U.S.  students  researched  cowboys. 

Due  to  the  great  number  of  students  involved  in  this  project,  the  students  were  divided  into  six  groups; 
history,  jobs,  appearances,  life  style,  culture,  and  beliefs  and  spirits.  By  mid  October  2000,  students  posted  their 
research  results  in  assigned  rooms  on  FirstClass  server  in  Tokyo.  We  chose  students  to  post  their  messages  of  Cl  in 
LI. 


3.  Read  in  L2  and  learn  C2,  then  post 

Students  read  the  posted  messages  in  L2  and  learn  C2.  Then  they  are  required  to  post  a response,  including 
a question,  in  the  mixture  of  LI  and  L2  by  the  end  of  October  2000.  The  interface  of  FirstClass  is  very  similar  to 
that  of  many  popular  email  applications.  Therefore,  it  is  user  intuitive  and  user  friendly. 

4.  Read  and  post  in  the  mixture  of  LI  and  L2 

By  middle  of  November  2000,  students  have  read  the  comments  and  questions  toward  their  research. 

These  comments  and  questions  required  further  research  in  Cl . By  this  time  the  communications  are  getting  less 
formal.  Students  are  developing  healthy  friendship  within  their  small  groups.  We  encourage  students  to  correspond 
as  much  as  they  prefer. 

5.  Concluding  iVisit  session 

At  the  beginning  of  December  2000,  we  conducted  a farewell  i Visit  session.  Students  from  U.S.  met  at  6 
p.m.  to  accommodate  the  time  difference.  (A  cultural  lesson,  meeting  time  in  U.S.  changes  by  one  hour  since  there 
is  no  Daylight  Saving  Time  in  Japan.)  The  excitements  fill  the  both  rooms  to  meet  their  cyber  friends  from  their 
small  groups  and  finally  put  the  name  and  face  together.  Students  filled  the  Post-Project  questionnaires  consisting 
44  questions,  to  complete  this  project. 

Five  C’s 

** Culture  is  not  the  people  but  the  communication  that  links  them  together.  ” 

Mary  Jane  Collier 

Our  goal  is  to  see  how  students  can  learn  C2  through  their  communication  in  more  realistic  way  through 
technology.  In  order  to  complete  this  approach,  we  must  consider  and  examine  the  Five  C’s  and  apply  this  idea  to 
enhance  learning. 

In  the  first  C,  communication,  we  use  iVisit  to  encourage  and  enhance  students’  speaking  and  listening 
skills.  In  contrast,  during  FirstClass  sessions  students  mainly  applied  their  reading  and  writing  skills.  Many 
students  enhanced  their  FirstClass  presentation  with  visual  and  aural  communication  tools,  such  as  sound  and 
graphic  files  to  deepen  the  communication. 

As  mentioned  earlier,  this  project  is  not  only  learning  C2,  but  begins  from  examining  Cl . This  enables  the 
complete  reflection  and  learning  of  cultures.  Examining  and  understanding  oneself  only  enhances  understanding 
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others.  FirstClass  sessions  allowed  students  to  examine  the  myth  and  find  the  truth.  iVisit  sessions  filled  students’ 
appetite  with  better  understanding  of  contemporary  popular  culture. 

As  students  learn  another  culture  in  a language  class,  there  is  an  opportunity  to  make  connections  to  other 
academic  fields.  The  following  chart  indicates  some  of  what  students  had  shared  and  learned  throughout  the 
project. 


iVisit 

FirstClass 

Politics 

Discussion  of  contemporary  agenda 

Political  system  from  the  appropriate  era. 

History 

N/A 

Cowboys  and  Samurai 

Music 

Discussion  on  contemporary  music 

Music  that  cowboys  and  samurai  listened. 

Art 

Contemporary  fashion  fads. 

Paintings  depicting  the  appropriate  era 
were  shared. 

Literature 

N/A 

Appropriate  literatures  were  shared. 

Sociology 

Experiencing  such  traits  as  tardiness  and 
shyness  of  students  make  a reflection  to  its 
culture. 

Fashion,  jobs  and  other  related  issues  were 
discussed. 

From  entire  project  students  received  opportunities  to  make  comparisons.  One  way  is  to  compare  and 
contrast  now  and  then.  It  is  done  in  different  subject  fields,  which  we  discussed  previously  as  a part  of  connection. 
Comparison  of  Cl  and  C2  within  the  particular  time  occurs  concurrently.  These  comparisons  take  place  during  both 
iVisit  and  FirstClass  sessions. 


Then 

Cl 

< > 

C2 

A 

A 

\/ 

V 

Now 

Cl 

< > 

C2 

To  provide  Communities,  we  used  iVisit  for  its  visual,  oral,  and  aural  communications.  This  community 
was  a large  community  where  students  shared  one  camera,  one  microphone  and  one  screen.  FirstClass  clients  were 
more  intimate.  It  used  small  groups  for  the  reading  and  writing  communities.  Individual  involvement  became  more 
crucial  in  this  exercise.  Yet  we  realize  that  a true  global  community  evolves  during  the  project.  Both  software  can 
be  used  from  anywhere  in  the  world. 

These  five  c’s  take  significant  part  of  this  project.  Each  c depends  on  each  other  to  make  this  collaboration 
effective.  Only  when  these  five  c’s  become  interactive,  this  project  claims  success. 

Data 

Since  the  first  semester,  all  of  the  targeted  Baylor  students  have  already  been  assisted  on  their  language  acquisition 
by  computers  from  their  “E-mail  Penpal”  programs.  However,  their  change  in  opinion  and  attitude  toward 
computer-assisted -learning  has  not  yet  been  measured  until  now.  Students  have  and  will  complete  two 
questionnaires  for  this  collaboration.  One  must  be  taken  before  the  project  and  the  other  immediately  after 
completion  of  the  project.*^ 


Questionnaires  are  available  for  the  viewing  purpose  only  at  http://www.baylor.edu/^Japanese/interculture.html. 
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Although  current  result  limits  in  accuracy  to  determine  its  outcome  from  the  data  of  only  one  project, 
preliminary  data  result  from  Baylor  students  show  the  effectiveness  of  computer-assisted  acquisition.  In  asking, 


□ Pre-Project 
■ Post-Project 


“Do  you  think  computer  is  a useful  learning  tool  in  academic  setting?”  clearly  there  is  an  increase  in  recognizing  the 
usefulness  and  effectiveness  in  application  of  technology  in  education  during  one  semester. 

Another  interesting  question  asked,  “Are  you  interested  in  foreign  cultures?”  The  interest  in  foreign 
cultures  increased  after  experiencing  this  collaborative  project.  Therefore,  as  predicted,  the  technologically  enhance 
culture -learning  collaboration  place  learners  in  positive  and  predictable  communicative  world  in  which  learners 
must  experience  to  acquire  another  culture. 


On  the  Japanese  side,  the  questionnaire  results  show  no  statistically  significant  change  of  attitudes  among 
students.  However,  quite  a few  post  project  comments  from  these  Japanese  students  tell  that  they  became  able  to 
perceive  their  virtual  partners  as  real  persons  and  build  a kind  of  relationship  that  promotes  learning.  The 
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importance  o/personal  relationship  for  network -based  learning  programs  has  been  suggested  by  several  Japanese 
researchers,  and  this  study  meets  such  a claim. 

Problems  to  be  solved 

As  with  any  project,  we  can  expect  some  problems.  The  problem,  which  we  cannot  resolve,  is  the  time 
difference  during  i Visit  sessions.  14  or  15  hours  in  time  difference,  depends  on  the  Daylight  Savings  Time, 
becomes  a menace.  Another  problem  in  iVisit  session  is  the  poor  connection  speed.  The  poor  connection  speed  will 
result  in  jerky  and  dropped  video  and  audio  feed.  Another  result  of  the  poor  connection  speed  is  the  extreme 
reverberation  of  audio  feed  which  disables  the  recognition  of  the  language. 

Limitation  in  the  number  of  video  windows  forces  us  to  use  only  one  camera.  It  would  be  ideal  to  have  all 
30+  students  to  appear  in  individual  windows.  But  software  issue  and  connection  speed  limits  us  to  perform  at  that 
level  yet. 

In  FirstClass  neither  the  time  nor  connection  speed  raise  serious  concern.  The  limitation  in  number  of 
students  who  can  login  simultaneously  brings  a serious  concern.  This  problem  disables  to  login  to  FirstClass  server 
as  an  individual  during  a class  and  has  discussion  on  particular  postings. 

Conclusion 

After  all,  ''when  we  are  merely  being  ourselves,  acting  according  to  our  deepest  instincts,  human  being 
reveal  fundamental  differences  in  what  we  all  tend  to  think  of  as  normal  behavior""  (Storti)  Communication  errors 
easily  and  rapidly  occur  in  our  networked  life.  Strangers  from  all  over  the  world  gather  in  the  online  community. 
Without  proper  understanding  of  cultures,  the  behaviors  progress  inappropriately. 

Five  c’s  take  an  important  role  in  teaching  culture  in  foreign  language  courses.  In  the  acquisition  of 
language  and  culture,  the  technology  assists  teachers  to  offer  innovative  and  effective  method  of  educating. 
Technology  aids  communication,  assists  to  learn  culture  of  L2,  make  connections  with  speakers  of  the  target 
language,  provides  comparisonshtiwttn  LI  -L2  and  C1-C2,  and  offers  to  participate  in  communities  using 
languages  other  than  LI . 
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Abstract 


To  attain  highly  efficient  instructional  conditions,  it  is  important  to  adapt  instruction  to  the  individual 
trainee.  This,  so-called,  personalisation  of  training  by  dynamic/automatic  task  selection  is  the  focus  of  the 
present  paper.  Recently,  cognitive  load  measures  have  been  proposed  as  a useful  addition  to  conventional 
performance  measures  like  speed  and  accuracy.  The  combination  of  conventional  performance  measures 
and  cognitive  load  measures  can  be  used  to  obtain  information  about  the  mental  efficiency  of  instructional 
conditions.  We  argue  that  the  dynamic/automatic  selection  of  learning  tasks  on  basis  of  mental  efficiency 
will  have  a significant  influence  on  the  optimization  of  the  learning  process. 

1.  Introduction 

People  are  faced  with  increasingly  demanding  working  environments  in  modem  society.  The  time 
constraints  are  increasing  and  task  environments  are  getting  more  complex.  People  have  to  master  complex  working 
skills  quickly  and  efficiently,  because  training  time  is  often  limited  and  mistakes  can  lead  to  dangerous  situations 
and  high  costs,  especially  in  technical  domains  such  as  aviation  and  industry.  One  way  to  meet  these  requirements  is 
to  create  more  efficient  training  by  personalizing  instruction 

The  primary  goal  of  this  article  is  to  argue  that  task  selection  based  on  mental  efficiency  optimizes  the 
learning  process.  Mental  efficiency  is  a combined  measure  that  uses  information  on  performance  measures  and 
cognitive  load  measures  (Paas  & Van  Merrienboer,  1993).  Existing  Intelligent  Tutoring  Systems  (ITS)  only  use 
performance  measures  as  a determinant  for  task  selection.  Although  cognitive  load  sometimes  is  measured,  it  is  not 
used  as  a determinant  for  task  selection 

In  this  article,  we  will  first  give  a review  of  various  task  selection  approaches.  We  make  a distinction 
between  static  and  dynamic  task  selection  approaches.  In  both  procedures  the  training  is  based  on  the  trainee’s  prior 
knowledge.  However,  the  selection  of  a set  of  learning  tasks  can  either  be  chosen  by  the  teacher/trainer  prior  to  the 
start  of  the  training  (static  procedures)  or  can  be  adjusted  during  the  training  (dynamic  procedures). 

It  has  been  proposed  that  when  teaching  student  complex  cognitive  skills,  part-task  training  can  have  a 
higher  learning  efficiency  and  reduced  training  costs  than  whole-task  training  (Wightman  & Lintem,  1985).  We  will 
use  the  whole -task  and  part -task  approaches  as  subcategories  for  static  and  dynamic  selection,  yielding  four 
approaches.  These  are  static  whole -task  approaches,  static  part -task  approaches,  dynamic  whole -task  approaches, 
and  dynamic  part -task  approaches.  Then  a new  approach  using  mental  efficiency  in  a dynamic  whole  task  procedure 
will  be  presented.  Finally,  we  will  discuss  this  approach. 

2.1  Static  whole-task  selection  approaches 

The  elaboration  theory  (Reigeluth  & Stein,  1983)  states  that  one  should  start  with  the  simplest  kind  of 
typical  task  that  an  expert  would  perform  and  to  gradually  progress  to  more  complex  tasks.  All  the  tasks  in  the 
training  are  whole -tasks.  Furthermore,  the  sequence  of  the  tasks  in  the  training  is  preset  before  the  training. 

The  elaboration  theory  makes  a distinction  between  task  expertise  and  domain  expertise.  Recently,  a new 
approach,  the  Simplifying  Conditions  Method  (SCM),  for  building  task  expertise  has  been  developed.  It  offers 
guidance  for  analyzing,  selecting,  and  sequencing  the  learning  tasks.  Given  that  any  complex  task  has  some 
conditions  under  which  it  is  easier  to  perform  than  under  others,  a SCM  sequence  begins  with  the  simplest  version 
of  the  task  that  is  still  fairly  representative  of  the  task  as  a whole.  Then  it  gradually  progresses  to  more  complex 
versions  of  the  task  until  the  desired  level  of  complexity  is  reached,  making  sure  that  the  learner  is  aware  of  the 
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relationship  of  each  version  to  the  other  versions.  Each  version  of  the  task  is  a class  or  group  of  complete,  real-world 
performances  of  the  task  (Reigeluth,  in  press). 

Another  new  approach,  called  the  familiarity  approach,  uses  the  prior  knowledge  (familiarity)  of  the 
trainees  and  the  difficulty  of  the  tasks  to  base  the  training  sequence  on  (Scheiter,  Gerjets,  & Tack,  2001).  The  first 
lessons  or  parts  of  a training  contain  high  familiarity  and  are  of  low  difficulty.  As  a learner  progresses  through  the 
lessons  or  training,  familiarity  decreases  and  difficulty  increases. 

The  SCM  and  the  familiarity  approach  share  many  aspects.  Both  approaches,  rightfully,  claim  that  the 
training  should  be  adjusted  to  the  prior  knowledge  of  the  trainee.  After  having  correctly  performed  a task  of  a low 
complexity,  the  complexity  is  gradually  increased  during  the  training.  Note  however,  that  the  training  has  been 
preset  and  the  sequence  of  the  tasks  in  the  training  is  not  subject  to  any  change.  These  approaches  do  not  adapt  to  the 
needs  of  the  individual  trainee.  They  only  adapt  to  the  prior  knowledge  of  a target  group  and  not  specifically  to  each 
individual  learner  before  the  training  starts. 

2.2  Static  part-task  selection  approaches 

Wightman  and  Lintern  (1985)  have  proposed  that  when  teaching  students  complex  cognitive  skills,  part- 
task  training  can  have  a higher  learning  efficiency  and  reduced  training  costs  than  whole -task  training.  Segmentation 
is  one  of  the  three  of  methods  of  task  decomposition  that  were  discussed  and  evaluated  by  Wightman  and  Lintern 
(1985).  Segmentation  involves  partitioning  a whole  task  into  components  along  spatial  or  temporal  dimensions. 

Prior  to  the  training  the  size  of  these  components  is  adapted  to  the  prior  knowledge  of  the  trainee. 

A well-known  method  of  segmentation-based  training  is  backward  chaining,  in  which  the  last  component 
of  a task  is  practiced  first  and  earlier  components  are  introduced  later  in  the  training  (Proctor  & Dutta,  1995). 

Benefits  of  backward  chaining  in  comparison  to  whole -task  training  may  originate  from  the  role  of  knowledge  of 
results  (KR)  in  learning  (Wightman  & Lintern,  1985).  Wightman  and  Lintern  (1985)  have  suggested  that,  as  a result 
of  training,  the  perceived  competency  in  performing  the  final  task  component  can  act  as  KR  for  the  preceding 
components,  thus  facilitating  learning  of  earlier  components  of  the  task. 

Ash  and  Holding  (1990)  found  that  forward  chaining,  in  which  the  order  for  adding  task  components  is  first 
to  last,  was  more  effective  for  learning  selective  keyboarding  skills.  Forward  chaining  may  sometimes  be  superior  to 
backward  chaining  because  in  this  method  task  component-completion  feedback  is  always  proximal  to  the 
component  task  being  introduced  (Proctor  & Dutta,  1995). 

An  approach  that  is  linked  to  segmentation  is  the  blocked  versus  random  scheduling  approach.  These  part- 
task  sequencing  methods  have  been  contrasted  with  each  other  in  numerous  studies  (e.g.,  Carlson  & Yaure,  1990). 
The  random  schedule  contained  four  rules  intermixed  from  trial  to  trial  within  practice  blocks,  compared  to  the 
blocked  schedule  that  contained  only  a single  rule  appearing  repeatedly  with  each  practice  block.  It  was  found  that 
acquisition  performance  for  a set  of  individually  presented  component  skills  was  better  when  the  components  were 
practiced  in  a blocked  schedule  than  in  a random  schedule.  However,  retention  and  transfer  performance  was  better 
following  a random  practice  schedule.  These  results  have  been  replicated  by  Lundy,  Carlson,  and  Paquiot 
(1995).Their  explanation  for  this  finding  was  that  random  practice  provides  a richer  set  of  cues  for  distinguishing 
among  the  items  in  a set.  This  results  in  more  accurate  retrieval  of  a particular  item  from  long-term  memory.  The 
richer  set  of  cues  is  another  way  of  saying  that  the  variation  was  higher.  They  furthermore  reasoned  that  rule- 
specific  processing  occurred  in  the  blocked  schedule  and  that  relational  processing  occurred  in  the  random  schedule. 
Rule -specific  reasoning  emphasizes  the  need  to  reconstruct  rule -like  procedures  in  working  memory.  Relational 
reasoning  emphasizes  the  opportunity  to  compare  multiple  representations,  thus  making  relations,  in  working 
memory.  These  two  processes  resemble  the  rule  automation  and  schema  acquisition  processes  that  were  identified 
by  van  Merrienboer  and  Paas  (1990). 

The  hierarchical  approach  that  was  developed  by  Gagne  (1968)  also  is  a static  part -task  approach.  It  is 
based  on  the  observation  that  a skill  is  made  up  of  simpler  “component  skills”  that  you  must  learn  before  you  can 
learn  the  larger,  more  complex  skills  of  which  they  are  a part.  Gagne  distinguished  five  intellectual  skills  that  are 
increasingly  detailed  and  difficult.  At  the  bottom  level  there  are  discriminations,  followed  by  concrete  concepts  and 
defined  concepts,  at  the  next  level  there  are  rules  and  at  the  top  level  there  are  high-order  rules.  The  hierarchical 
arrangement  of  these  five  skills  helps  one  to  figure  out  what  the  prerequisites  a given  skill  might  have.  To  make  sure 
the  learner  is  not  confronted  with  learning  tasks  of  skills  that  (s)he  already  has  mastered,  the  training  needs  to  be 
started  at  the  level  of  “entering  knowledge”  of  the  learner.  A hierarchical  sequence  is  one  which  never  teaches  a skill 
before  its  prerequisites  (Gagne,  1968). 

All  three  approaches,  rightfully,  claim  that  one  should  adapt  the  training  to  the  trainee’s  prior  knowledge. 
And  like  the  hierarchical  approach  states,  some  skills  should  be  learned  before  a trainee  can  start  to  learn  a more 
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complex  skill.  However,  none  of  the  approaches  allows  adaptations  to  the  individual  trainee  to  be  made  during  the 
training. 

2.3  Dynamic  whole-task  selection  approaches 

The  last  decade,  dynamic  approaches  have  been  widely  used  to  adapt  more  efficiently  to  the  needs  of  the 
individual  trainee.  It  is  possible  to  respond  to  the  learner’s  problems  during  the  training,  with  decisions  being  made 
that  are  typically  based  on  the  performance  of  the  trainee. 

In  order  to  keep  track  with  the  trainee’s  history  of  the  tasks  and  the  correlating  performance,  many  ITS 
(Intelligent  Tutoring  Systems)  use  a student  model.  A student  model  builds  a knowledge  base  of  the  trainee,  and 
updates  that  knowledge  as  the  trainee  progresses  through  the  tasks  of  the  training.  Certain  learning  objectives  have 
been  specified  prior  to  the  training,  which  are  used  to  check  the  progress  of  the  trainees.  Performance  measures  are 
collected  and  are  compared  to  the  learning  objectives.  After  this  comparison,  the  selection  rules  indicate  the  next 
learning  task  to  present  to  the  learner. 

However,  many  approaches  focus  on  elaborating  the  operationalization  of  student  modelling  while  not 
being  clear  on  the  selection  rules  that  are  being  used.  These  approaches  include  psychometric  approaches  (for  a 
discussion,  see  Everson,  1995),  agents  (e.g.,  Capuano,  Mersella,  & Salerno,  2000;  Giroux,  Leman,  & Marcenac, 
1995),  and  fiizzy  logic  (e.g.,  Virvou,  Maras,  & Tsiriga,  2000).  Mostly,  the  primary  function  of  student  models  is  to 
give  specific  feedback  to  the  learners  about  their  performance.  An  ITS  that  explicitly  describes  the  selection  rules 
that  are  being  used  is  CASCO  (Completion  Assignment  Constructor).  CASCO  is  an  ITS  for  the  dynamic 
construction  of  assignments  to  practice  introductory  programming  (Van  Merrienboer,  Luursema,  Kingma,  Houweling, 
& De  Vries,  1996).  Based  on  different  actions  of  the  trainee,  CASCO  can  decide  whether  the  trainee  has  learned  the 
programming  skill  or  not,  can  adjust  the  amount  of  presented  information  by  increasing  or  decreasing  information  in 
the  completions  tasks,  and  decide  to  either  delete  or  add  the  program  code,  explanations,  questions,  and  instructional 
tasks  (Van  Merrienboer,  & Luursema,  1995). 

CASCO  uses  several  straightforward  selection  rules.  The  most  important  rule  states  that  a good  problem  is 
suitable  to  present  new  learning  elements  and  to  practice  known  learning  elements.  CASCO  could  therefore  be 
classified  as  a Progressive  Mental  Model  (PMM).  The  other  rules  state  that  a good  problem  is  not  too  difficult,  has 
not  been  presented  to  the  learner  before,  and  is  suitable  to  remediate  learning  elements  the  learner  makes  mistakes 
with.While  the  learner  is  working  on  an  assignment,  student  diagnosis  takes  place  in  order  to  update  the  student 
model.  The  learner’s  results  on  questions  and  instructional  tasks  form  the  input  of  student  diagnosis.  For  all  learning 
elements  that  have  already  been  presented  to  the  learner,  the  so-called  Expertise  and  Incompetence  are  computed 
(Van  Merrienboer,  Krammer,  & Maaswinkel,  1994).  Expertise  indicates  the  learner’s  proficiency  in  correctly  using  a 
particular  learning  element,  while  Incompetence  indicates  the  learner’s  tendency  to  make  errors  with  a particular 
learning  element.  For  each  learning  element,  the  Expertise  and  Incompetence  are  further  modeled  as  fiizzy  sets.  The 
truth  value  of  the  membership  of  those  sets  may  range  between  0 and  1 . To  keep  track  of  the  students’  progress  two 
sets  have  been  developed.  The  Learning  Set,  which  contains  the  learning  elements  that  the  learner  is  already 
practicing  but  has  not  mastered  yet.  And  the  Incompetence  Set,  which  contains  the  learning  element  that  the  learner 
is  already  advancing  in  but  still  makes  mistakes  with  (Schuurman,  1999). 

It  is  common  for  ITS  to  base  their  student  model  and  task  selection  on  performance  measures  like  speed 
and  accuracy.  We  argue  that  it  is  important  to  use  mental  effort  as  an  additional  determining  factor  because  different 
students  can  attain  the  same  performance  level  with  different  amounts  of  invested  mental  effort. 

2.4  Dynamic  part-task  selection  approaches 

The  first  dynamic  task  selection  approach,  in  a very  raw  version,  was  branching.  This  part -task  approach 
attempts  to  diagnose  the  learner’s  response,  usually  on  the  basis  of  a multiple -choice  question.  The  training  starts 
with  offering  pieces  of  information  and  during  the  training  more  information  is  added  with  each  next  step  until  the 
trainee  reaches  the  state  where  the  whole  task  can  be  performed.  This  process  is  also  known  as  snowballing  (van 
Merrienboer,  1997).  After  the  learners  have  been  presented  a certain  amount  of  information,  they  are  given  a 
multiple -choice  question.  If  they  answer  correctly,  they  branch  to  the  next  body  of  information.  However,  if  they  are 
incorrect,  they  are  directed  to  additional  information,  depending  on  the  mistake  they  made  (Clark,  1997).  The 
amount  of  branching  may  vary  considerably,  from  occasional  branch  points  to  branching  after  every  student’s 
response.  Branching  can  be  based  on  individual  performances,  cumulative  performance,  or  student  choice.  The 
direction  of  branching  can  either  be  forward,  sideways,  or  backward.  Forward  means  that  the  learner  skips 


221 


710 


information  that  most  students  see.  Backwards  means  that  the  learner  is  returned  to  repeat  instructions.  And 
sideways  means  that  the  learner  is  exposed  to  extra  information  that  most  learners  skip  (Allesi  & Trollip,  1991). 

Most  of  the  research  on  branching  has  been  done  in  the  1960s  and  1970s.  Although  the  theoretical 
foundation  is  good  and  makes  sense,  the  practical  part  of  proving  the  superiority  of  branching  over  normal  fixed 
sequencing  appeared  to  be  difficult.  There  are  several  studies  that  showed  evidence  in  favor  of  the  branching 
sequence  over  a fixed  sequence  (Coulson,  Estavan,  Melaragno,  & Silberman,  1961;  Hurlock,  1972;  Slough,  Ellis,  & 
Lahey,  1972),  but  most  studies  fail  to  show  a superior  effect  of  branching  (Campbell,  1962;  Glaser,  Reynolds,  & 
Harakas,  1962;  Holland  & Porter,  1961;  Lahey,  1973).  It  was  stated  that  the  Computer  Assisted  Instruction 
programs  usually  consisted  of  a simple  algorithm  for  branching  among  a few  fixed  alternative  questions.  Such  rigid 
plans  do  not  provide  a model  of  how  a tutor  can  adapt  the  generation  of  tasks  to  suit  the  particular  needs  of  each 
student  (McArthur,  Stasz,  Hotta,  Peter,  & Burdorf,  1988).  Furthermore,  the  costs  and  time  needed  to  complete  a 
part -task  training  are  mostly  very  high. 

3.  The  use  of  mental  efficiency  in  a dynamic  whole-task  selection  approach 

Our  approach  incorporates  various  elements  of  the  discussed  task-selection  approaches.  Like  in  the  whole 
task  approaches  we  start  with  the  simplest  version  of  a whole  task.  The  complexity  of  the  training  should  be  adjusted 
to  the  trainee’s  prior  knowledge.  In  other  words,  the  familiarity  should  be  at  an  appropriate  level.  All  the  versions  of 
a whole  task  are  categorized  in  learning  tasks  which  gradually  increase  in  complexity  in  respect  to  each  other.  The 
trainee  starts  with  the  lowest  complexity  and  proceeds  to  the  highest  complexity,  while  the  pace  of  the  progression 
depends  on  performance  and  cognitive  load  measures.  Like  in  CASCO  (e.g.,  Van  Merrienboer,  Luursema,  Kingma, 
Houweling,  & De  Vries,  1996)  we  can  adjust  the  amount  of  information  that  is  presented  to  a learner.  Dependent  on 
the  learner’s  performance  and  load,  the  information  can  be  decreased,  kept  constant,  or  increased.  Furthermore,  as  in 
the  dynamic  approaches  we  can  select  learning  tasks  during  the  training  and  adjust  to  the  individual  learner’s  needs 
during  training. 

Another  characteristic  of  the  approach  is  that  instead  of  presenting  more  whole  tasks,  it  can  be  more 
efficient  to  present  part -task  training  for  the  part  of  the  task  that  the  trainee  has  not  mastered  yet.  Consider  for 
example  learning  how  to  drive  a car.  If  a trainee  has  mastered  steering  skills,  gas -and  brake  skills  but  still  has  not 
mastered  the  shifting  gear  skill,  then  this  skill  should  be  practiced  in  isolation  until  the  required  performance  level 
has  been  reached.  With  part -task  training  of  this  component  the  costs  in  terms  of  cognitive  capacity  will  decrease 
and  the  learner  will  be  able  to  acquire  the  whole  task  more  efficiently. 

Cognitive  load  has  been  acknowledged  as  an  important  factor  in  the  training  of  complex  cognitive  skills. 
When  learners  acquire  simple  skills,  cognitive  load  only  plays  a minor  role.  A badly  designed  training  does  not 
necessarily  have  negative  effects  because  learners  can  invest  mental  effort  to  compensate  for  the  bad  design. 
However,  when  learners  are  presented  with  a training  of  complex  skills,  they  are  not  able  to  so  anymore.  They  do 
not  have  mental  effort  for  this  purpose  because  the  training  generates  a high  cognitive  load  upon  the  learner’s 
capacities.  Students  can  attain  a high  performance  but  with  varying  amounts  of  load.  By  taking  cognitive  load  into 
account  the  decision  in  task  selection  can  lead  to  optimal  individual  learning.  Learners  attain  the  highest 
performance  when  they  have  to  invest  an  optimal  amo  unt  of  load.  Therefore,  we  propose  to  use  performance 
measures  in  combination  with  mental  effort  measures  for  dynamic  task  selection.  A procedure  for  combining  mental 
effort  and  performance  measures  in  a measure  of  mental  efficiency  was  described  by  Paas  and  van  Merrienboer 
(1993).  The  efficiency  measures  take  differences  in  cognitive  capacity,  expertise,  and  motivation  into  account.  The 
use  of  mental  efficiency  is  expected  to  make  the  individual  training  more  efficient,  and  to  lead  to  better  transfer 
results. 

A first  confirmation  for  this  claim  was  found  in  a study  conducted  by  Camp,  Paas,  Rikers  and  van 
Merrienboer  (2001).  They  compared  four  methods  of  task  selection  in  the  domain  of  air-traffic  control.  Learning 
tasks  could  be  presented  in  a fixed  order  from  simple  to  complex,  or  the  selection  could  be  based  on  performance, 
mental  effort,  or  the  combined  measure  of  both,  mental  efficiency.  Although,  they  did  not  find  differences  in  transfer 
performance  as  a function  of  learning  task  selection  method,  the  results  showed  that  participants  in  the  mental 
efficiency  condition  were  confronted  with  high  variability  of  learning  tasks,  and  that  participants  in  the  performance 
condition  were  confronted  with  low  variable  learning  tasks. 

We  believe  that  our  approach  is  a promising  one.  It  takes  more  information  about  the  learner’s  learning 
progress  into  account  than  the  conventional  approaches.  Mental  efficiency  combines  performance  measures  and 
cognitive  load  measures  on  which  the  task  selection  decisions  are  based.  Furthermore,  tasks  can  be  presented  in 
different  formats  to  the  trainee.  Dependent  on  the  mental  efficiency  of  a previous  task,  the  decision  can  be  made 
whether  the  learner  support  can  be  decreased,  kept  constant  or  increased.  The  variability  is  high,  which  means  that 
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trainees  will  be  exposed  to  various  analogies  that  induce  schema  acquisition.  Because  of  the  higher  training 
efficiency,  trainees  participating  in  our  approach  should  also  be  able  to  reach  better  results  on  transfer  tasks.  We  are 
currently  developing  more  experiments  to  obtain  more  evidence  for  the  expectation  that  our  approach  indeed  leads 
to  more  efficient  individualized  training. 

4.  Discussion 

The  goal  of  this  article  was  to  argue  that  task  selection  based  on  mental  efficiency  is  a promising  approach 
for  optimizing  the  learning  process.  In  order  to  reach  this  goal,  we  first  gave  a review  of  various  task  selection 
approaches.  We  made  a distinction  between  static  task  selection  approaches  and  dynamic  task  selection  approaches. 
Within  these  main  approaches,  a distinction  was  made  between  whole -task  and  part -task  approaches . The  resulting 
four  approaches  were  static  whole-task  approaches,  static  part -task  approaches,  dynamic  whole  -task  approaches,  and 
dynamic  part -task  approaches.  The  static  whole -task  approaches  including  the  familiarity  approach  (Scheiter, 
Gerjets,  & Tack,  2001)  and  the  elaboration  theory  (Reigeluth  & Stein,  1983)  claim  that  the  training  should  be 
adjusted  to  the  prior  knowledge  of  the  trainee.  After  having  correctly  performed  a task  of  a low  complexity,  the 
complexity  is  gradually  increased  during  the  training.  Our  main  critic  on  the  static  aspect  was  that  the  training  has 
been  preset  and  the  sequence  of  the  tasks  in  the  training  is  not  subject  to  any  change.  In  this  situation  it  is  impossible 
to  optimally  adapt  to  the  needs  of  the  individual  trainee,  especially  during  training. 

The  static  part -task  approaches,  including  segmentation  (Wightman  & Lintem,  1985),  blocked  vs.  random 
schedules  (e.g.,  Lundy,  Carlson,  & Paquiot,  1995)  and  the  hierarchical  approach  (Gagne,  1968),  showed  that  part- 
task  training  can  be  useful  at  some  occasions.  However,  because  of  its  high  costs  and  long  duration,  we  propose  to 
use  part -task  training  only  as  an  additional  part  of  whole -task  based  training.  The  static  aspect  of  the  part -task 
approach  also  was  criticized  for  its  inability  to  adapt  efficiently  to  the  learner’s  needs  during  training. 

Dynamic  whole-task  approaches  including  Intelligent  Tutoring  Systems  (ITS)  have  been  focussing  on  the 
operalization  of  selection  rules.  Many  articles  describe  the  functionalities  of  their  student  models  in  great  detail  but 
do  not  explicitly  describe  the  selection  rules  that  have  been  used.  CASCO  was  identified  as  an  exception  to  this  rule. 
Our  main  critic  on  dynamic  whole -task  approaches  was  that  they  only  use  performance  as  the  determining  factor  for 
task  selection.  We  argued  that  cognitive  load  can  provide  additional  information  about  the  learning  progress  of  a 
trainee. 

Dynamic  part -task  approaches  including  branching  (e.g.,  Campbell,  1962)  were  the  first  to  try  to  diagnose 
the  learner’s  response.  However,  the  programs  that  used  branching  usually  consisted  of  a simple  algorithm  for 
branching  among  a few  fixed  alternative  questions.  This  does  not  enable  one  to  efficiently  adapt  to  the  particular 
needs  of  each  student  (McArthur,  Stasz,  Hotta,  Peter,  & Burdorf,  1988). 

Instead,  we  suggested  a mental-efficiency  based  dynamic  whole  task  procedure  with  the  possibility  for 
additional  part -task  training.  Task  selection  is  based  on  a combined  measure  of  performance  measures  and  cognitive 
load  measures  called  mental  efficiency.  Furthermore,  the  sequence  of  the  learning  tasks  is  not  preset  but  is 
dependent  on  the  mental  efficiency  of  the  individual  trainee.  It  is  important  to  adapt  instruction  to  the  individual 
learner  to  attain  efficient  instructional  methods.  Preliminary  evidence  for  the  success  for  the  use  of  mental  efficiency 
in  dynamic  task  selection  was  found  in  the  study  of  Camp,  Paas,  Rikers  and  van  Merrienboer  (2001). 
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Abstract 

This  paper  focuses  on  the  management  and  change  domains  of  educational  technology,  namely  the  need  for 
educational  technology  (ET)  leadership  in  higher  education.  Leadership  issues  are  examined  with  respect 
to  instructional  development,  faculty  development,  and  instructional  technology/media  management  in 
higher  education  and  the  roles  ET  practitioners  can  ploy.  Other  topics  include:  a description  of  the 
transformative  process  underway  in  higher  education  driven  by  technological  innovations;  raising 
awareness  of  the  need  for  ET  practitioners  and  doctoral  students  to  aspire  for  administrative  leadership 
positions  for  effective  policy  formulation,  strategic  planning,  management,  evaluation,  and  implementation 
of  instructional  development;  and  discussion  (f  the  essential  skills  and  strategies  for  achieving  ET 
leadership  in  colleges  and  universities. 

We  are  in  favor  of  ET  professionals  going  beyond  providing  basic  services  or  merely  running  equipment 
distribution  units.  We  advocate  leading  the  change  in  instructional  development,  technology  integration  in 
the  learning  process,  managing  resources  and  effectively  collaborating  with  faculty  in  designing 
instruction.  We  advocate  for  positions  that  will  permit  the  ET  professional  to  work  with  institutional 
leaders  in  policy-making  and  in  planning  for  instructional  technology  deployment,  faculty  development, 
and  instructional  development. 

The  term  ET  professionals  is  used  to  refer  to  individuals  with  formal  training  in  the  field  of  educational 
technology,  instructional  technology,  instructional  media,  instructional  systems,  educational 
communications  and  all  related  areas,  and  who  have  work  experience  in  the  field. 

Introduction 

“The  real  question  is  not  whether  higher  education  will  be  transformed  but  rather  how  and  by  whom.''"  — James  J. 
Duderstadt  (1999) 

The  emergence  of  new  technologies  lead  by  the  popularity  of  the  microcomputer  and  the  ubiquitous  nature 
of  communication  technologies  has  brought  significant  changes  to  the  field  of  educational  technology  (ET).  These 
changes  are  acutely  felt  in  higher  education,  where  Duderstadt  (1999)  envisions  a significant  transformation  during 
this  decade  as  colleges  and  universities  respond  to  their  internal  changes  (Katz,  1999)  and  the  needs  of  a rapidly 
changing  society  (Surry,  1996;  Surry  & Robinson,  2001).  Some  notable  changes  in  higher  education  include: 
requirements  for  all  students  to  own  laptop  computers;  incentives  for  professors  to  integrate  technology  into  their 
courses;  wireless  network  campuses;  and  classrooms  equipped  for  multimedia  learning.  Technological  innovations, 
in  general,  are  not  new,  but  the  accelerated  rate  of  change  is  moving  towards  a knowledge-driven  future  for  which 
many  colleges  and  universities  are  struggling  to  prepare. 

The  rapid  changes  in  technological  innovations  we  are  witnessing  all  around  us  today  are  affecting 
educational  institutions  and  are  fundamentally  changing  the  educational  needs  of  society  (Surry,  1996).  There  seem 
to  be  the  lessening  of  the  initial  resistance  to  the  adoption  of  the  new  technologies  in  higher  education.  The  effects  of 
the  change  induced  by  the  technological  innovations  in  higher  education  are  becoming  obvious.  Change  is  not  a 
new  phenomenon  as  most  organizations  go  through  such  experiences.  Change  in  higher  education  is  inevitable  and 
as  Duderstadt  (1999)  pointed  out  “the  real  question  is  not  whether  higher  education  will  be  transformed  but  how  and 
by  whom”.  The  change,  in  part,  is  brought  from  the  outside  by  technological  forces  (Daft,  2001).  As  noted  by 
Siegel  (1999)  and  Doll  (1993),  higher  education  is  a complex  and  chaotic  social  system.  The  complex  nature  of 
higher  education  has  affected  the  manner  and  method  of  adopting  the  new  innovation.  Equally,  the  influence  of 
technological  innovations  has  affected  the  way  business  is  done  in  higher  education.  Some  of  the  areas  that  have 
been  impacted  by  the  innovations  are,  faculty  training  in  the  use  of  technology  in  teaching,  innovative  learning  with 
technology,  and  the  management  of  technology  in  higher  education.  Administrators  in  higher  education  are  doing 
their  best  to  manage  the  change  process. 
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Faculty  are  key  stakeholders  in  the  adoption  of  educational  technologies  since  the  changes  brought  about 
by  such  innovations  affect  either  the  way  students  learn  or  how  instructors  teach.  Campus  officials  have  consistently 
rated  assisting  faculty  with  technology  integration  as  one  of  the  most  important  information  technology  issues 
challenging  higher  education  (Green,  2000).  Units  that  support  faculty  in  their  instructional  roles,  such  as  audio 
visual  services,  instructional  media  centers,  faculty  development  centers,  and  centers  for  teaching  excellence,  are 
also  undergoing  changes  in  organizational  structure,  technologies,  and  functional  operation  to  meet  the  emerging 
needs  of  an  educational  technology  powered  campus.  Administrators  within  these  units  are  facing  dual  challenges  to 
manage  a dynamic  change  process  and  handle  the  educational  technology  innovation.  Fullan  (1993)  and  Sparks 
(1993)  assert  that  leaders  in  educational  institutions  need  to  understand  the  change  process  in  order  to  effectively 
lead  and  efficiently  manage  the  transformation  currently  underway.  To  assist  both  faculty  and  administrators  in  this 
process,  ET  practitioners  can  provide  leadership  in  the  adoption  and  use  of  technology  for  the  improvement  of 
teaching  practice  and  learning  processes. 

ET  professionals  in  the  field 

In  a speculative  article  on  the  roles  of  educational  technology  practitioners  in  the  year  2000,  Bratton  (1988) 
wonders  if  there  will  be  many  in  this  field  who  will  rise  to  leadership  roles.  Surry  and  Robinson  (2001)  found  that 
many  colleges  and  universities  are  actively  recruiting  for  positions  that  require  educational  technology  skills.  While 
most  of  those  positions  are  for  instructional  technologists,  a few  are  at  a high  level  of  ET  leadership,  such  as 
director,  dean,  or  vice  president  of  an  academic  computing  unit.  Surry  (1996)  reports  that  educational  technologists 
are  steadily  being  hired  in  higher  education  and  in  a recent  study  by  Surry  and  Robinson  (2001)  they  categorized 
numerous  current  educational  technology  job  postings.  Albright  (1995)  also  reported  that  administrative  positions  in 
higher  education  are  being  advertised.  This  is  unprecedented  and  could  be  attributed  to  the  technology  revolution. 
However,  an  important  question  is,  are  these  positions,  particularly  those  for  senior  managers,  directors,  deans,  and 
assistant  vice  president  positions,  being  filled  by  ET  professionals? 

Who  is  leading? 

Many  graduates  of  educational  technology  programs  who  hold  the  master’s  or  doctorate  degree  seem  to 
take  up  positions  as  instructional  designers  or  faculty  teaching  positions  in  colleges  and  universities  with  titles  such 
as:  Instructional  Technologist;  Instructional  Designer;  Distance  Learning  Co -coordinator;  Instructional  Technology 
Manager/Administrator;  Technical  Support  Specialist;  World  Wide  Web  Specialist;  and  Instructional  Technology 
Librarian  (Surry  & Robinson,  2001).  As  a consequence,  the  early  periods  of  technology  revolution  saw  them  still  in 
their  “traditional”  roles  rather  than  taking  up  positions  to  direct  or  influence  the  adoption  of  innovation  in  higher 
education.  Such  roles  are  non-agenda  setting  (Surry,  1996).  This  is  not  to  say  that  the  positions  they  hold  are 
inadequate  for  leadership,  however,  they  may  be  limiting  in  terms  of  the  influence  they  have  in  directing  educational 
policy  issues  and  in  influencing  change.  The  current  roles  held  by  ET  practitioners  can  be  the  starting  point  for 
advancement  to  higher-level  administrative  and  policy-setting  positions  that  provide  platform  for  greater  influence 
on  the  transformation  of  higher  education. 

The  technology  revolution  has  created  the  need  for  educational  technologists  in  almost  every  sector  of  the 
economy  including  higher  education.  Albright  (1995),  comments  that,  “in  fact  we  are  seeing  significant  evidence  of 
administrative  recognition  (with)  position  announcements  for  a director  or  dean  to  administer  a composite  of 
campus-wide  educational  technology  services”  being  advertised.  That  is  good  news  for  practitioners  in  this  field. 
Additionally,  Surry  (1996)  reports  that  educational  technologists  are  being  steadily  hired  in  higher  education; 
however,  the  responsibilities,  functions,  and  qualifications  required  for  these  roles  are  not  properly  defined.  The 
same  question  remains,  are  these  positions  being  filled  by  ET  professionals?  While  these  positions  are  being 
advertised,  it  will  be  interesting  to  find  out  if  they  are  being  filled  by  ET  graduates  or  otherwise  by  faculty  me  mbers 
from  different  disciplines.  Anecdotal  evidence  suggests  that  many  individuals  filling  these  positions  have  not  had 
the  academic  training  in  educational  technology.  Faculty  from  other  disciplines  seem  to  dominate  administrative 
leadership  and  management  roles  regarding  instructional  development  within  colleges  and  universities,  as  well  as 
faculty  development  areas.  The  implications  are  that  decisions  and  policies  regarding  technology  integration  in  the 
classroom,  helping  faculty  to  improve  teaching,  and  the  instructional  development  process  across  all  academic 
disciplines  in  higher  education  are  not  made  by  those  trained  in  educational  technology. 

Wright  (1997)  found  in  a study  of  faculty  and  instructional  development  programs  in  higher  education  that 
leaders  in  these  areas  seem  to  emerge  from  the  faculty  ranks,  irrespective  of  their  primary  academic  discipline  or 
previous  experiences.  The  study  confirms  the  notion  that  the  appointments  to  such  leadership  positions  are  usually 
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filled  by  faculty,  who  are  prepared  as  specialists  in  their  various  fields  with  little  or  no  training  in  leadership, 
technology  use,  and  in  the  process  of  teaching  and  learning  (Bates,  1999;  Noone  & Swenson,  2001).  Most  are  not 
trained  as  educational  technologists.  Therefore,  faculty  appointments  to  top  ET  positions  may  be  the  result  of  a long- 
standing perception  of  the  roles  of  ET  practitioners  by  faculty  and  higher  education  administrators.  Whereas  faculty 
have  maintained  a strong  position  as  leaders  in  academic  matters,  ET  practitioners  are  perceived  as  providing 
services  to  support  instruction  (Heinich  1995).  There  seems  little  room  for  faculty  to  negotiate  or  share  the  roles. 
Administrators  in  higher  education  do  not  appear  to  have  demonstrated  enough  understanding  of  the  roles  or 
potentials  of  ET  professionals. 

The  Need  for  Leadership 

At  any  level,  there  remains  a significant  role  for  an  educational  technology  leader  in  higher  education  to 
guide  faculty  innovations,  work  collaboratively  with  faculty  and  administrators,  and  lead  the  efforts  towards 
educational  technology  change  in  higher  education.  As  the  transformation  progresses,  faculty  need  training  in  the 
skills  that  are  essential  for  teaching  and  learning  and  with  technology,  support  during  the  development  process,  and 
advice  for  the  effective  integration  of  media  and  information  technologies.  While  a few  ET  practitioners  in  colleges 
and  universities  have  advanced  to  leadership  positions,  entrenched  institutional  policies  and  practices  unique  to 
higher  education  present  challenges  to  overcome  (Bates,  1999;  Green,  1999;  and  Duderstadt  1999). 

A survey  of  instructional  development,  academic  technology,  faculty  development,  and  media  services 
programs  in  higher  education  reveals  a need  for  leadership  and  management  of  these  programs.  It  does  appear  that  a 
problem  that  Spitzer  (1987)  raised  (reported  in  Galbraith  et  al,  1990)  still  persists.  Spitzer  identified  three  factors  he 
“believes  explain  why  educational  technology  has  not  achieved  its  potential.”  One  of  the  reasons  he  identified  was 
the  leadership  problem,  that  is,  the  dearth  of  people  who  will  lead  to  moderate  the  “quagmire  of  individuals,  groups, 
and  interests  that  claim  a part  share  in  the  educational  technology  enterprise”.  He  noted  that  a major 
managerial/leadership  gap  has  not  been  bridged.  Seels  and  Richey  (1994)  describe  management  as  one  of  five 
domains  of  the  instructional  technology  field.  Galbraith,  et  al.  (1990)  add  that,  “The  successful  application  of 
technology  in  education  depends  not  so  much  on  the  technology,  but  how  it  is  managed”  A recent  study  by  Johnson 
(2001)  that  looked  at  issues  in  academic  technology  leadership  in  higher  education  found  few  ET  professional  in  top 
leadership  positions.  Bates  (2000),  who  visited  several  colleges  and  universities  during  a study,  states  this  case 
precisely  and  succinctly,  “In  all  the  organizations  that  I visited  where  technology  was  being  used  successfully  for 
teaching,  strong  leadership  was  a critical  factor.  Without  leadership  and  a strong  sense  of  support  for  change  in  an 
organization,  the  barriers  of  inertia  will  be  too  great”. 

Effective  management  of  the  instructional  development  process  and  technologies  used  in  instruction,  and 
the  preparation  of  faculty,  will  likely  result  in  the  efficient  use  of  the  technologies,  and  in  the  improvement  of  both 
teaching  and  student  learning  processes.  Management  implies  leadership  and  by  virtue  of  their  placement  in 
leadership  positions,  ET  practitioners  are  advantageously  positioned  as  change  agents,  especially  as  it  relates  to  the 
use  of  technology  in  instruction.  As  a result  of  their  preparations  and  understanding  of  educational  systems  change, 
instructional  systems  approach,  and  knowledge  of  learning  technologies,  educational  technologists  can  exert 
considerable  influence  through  technology  in  reshaping  educational  settings.  Roberts  (1994)  argues  that  technology 
and  educational  reform  or  restructuring  is  “inexorably”  linked  and  that  merely  deploying  technology  in  schools 
cannot  cause  educational  reform.  The  changes  called  for  cannot  occur  in  isolation  or  by  simply  acquiring  new 
technologies.  In  the  higher  educational  system,  change  is  best  implemented  from  within.  There  is  need  in  education 
for  systemic  thinking  as  the  newer  technologies  are  introduced.  Introducing  innovation  and  managing  change  in 
higher  education,  we  believe,  requires  individuals  who  have  established  credibility  within  the  institution;  understand 
the  idiosyncrasies  of  academic  environments;  support  the  role  and  use  of  technology;  respect  the  instructional  design 
process,  and  possess  skills  in  change. 

The  focus  here  is  on  leadership.  We  do  recognize  the  importance  of  management  functions  and  other  roles 
that  ET  professional  play,  they  are  all  essential  for  the  success  of  the  learning  enterprise.  While  management  and 
leadership  seem  synonymous,  we  have  emphasized  leadership,  as  many  seem  to  agree  that  it  provides  a broader 
perspective,  fosters  a holistic  view  on  issues,  enlarges  vision,  engenders  viewing  issues  from  systemic  point  of  view, 
and  provides  better  opportunity  for  change  sponsors  and  agents  for  the  betterment  of  the  organization,  and  such  that 
will  appeal  to  the  organizational  stakeholders.  The  leader  directly  or  indirectly  influences  and  motivates  individuals 
with  inspiring  examples.  Leaders  are  expected  to  shape  the  culture  of  an  organization  by  creating  new  visions. 
Kearsley  and  Lynch  (1 994)  add  that  the  success  of  leaders  is  determined  by  their  ability  to  understand  and  influence 
organizational  cultural  mores  and  values.  When  followed,  the  course  that  the  leader  charts  will  ultimately  profit  the 
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organization,  and  in  this  case  higher  education.  Management,  though  equally  good,  tends  to  focus  on  details  in  their 
work  environments  and  to  compartmentalize. 

Unique  Qualities  of  ET  Professionals 

We  do  believe  that  by  the  nature  of  their  training  and  experience,  ET  practitioners  can  work  well  with 
faculty  in  improving  instruction  in  higher  education,  as  well  as  working  effectively  to  manage  educational 
technology  change  in  higher  education.  ET  practitioners  cm  work  well  as  change  sponsors,  change  agents  and 
change  advocates  in  initiating  and  implementing  change  in  higher  education.  Rather  than  be  seen  as  encroaching 
into  the  faculty  domain  of  instructional  delivery,  they  can  co-exist  as  partners  in  the  effort  for  instructional 
development  and  integration  of  technology  in  instruction.  To  better  address  this  issue  will  require  an  understanding 
of  the  preparation,  roles,  function,  and  abilities  of  the  ET  practitioners  in  instructional  development  and  managing 
change.  Their  training  focuses  on  instructional  design  and  development,  media  use  in  instructional  or  performance 
improvement,  educational  and  learning  theories,  technologies  of  instruction,  as  well  as  educational  systems  design. 
Some  programs  are  adding  change  management,  organizational  development,  arid  management  courses.  These  skills 
equip  them  to  view  issues  systemically  and  holistically,  and  to  provide  the  appropriate  interventions. 

The  Educational  Technologist:  A Change  Agent  for  Higher  Education 

This  seems  to  be  the  right  direction  for  higher  education  in  its  efforts  to  either  manage  or  implement 
successful  technological  change  initiatives.  People  who  are  familiar  with  higher  education  seem  to  view  the  issue 
from  the  same  perspective.  Bates  (2000)  advocates  that  the  best  approach  for  colleges  and  universities  is  to  create 
the  position  of  “associate  vice  president,  academic,  with  overall  responsibility  for  academic  technology  issues, 
probably  as  part  of  a larger  unit  for  teaching  and  learning.”  This  position  will  collaborate  with  the  information 
technology  group  and  similar  on-campus  organizations  to  manage  instructional  technology,  administer  faculty 
training,  and  design  instruction  for  better  student  learning.  Dillon  and  Walsh  (1992)  identify  leadership  as  the 
foundation  on  which  change  rests.  Kearsley  and  Lynch,  (1994)  summarize,  “Technology  leadership  is  inherently 
linked  to  innovation  and  this  provides  unique  consideration.  While  leadership  usually  involves  dealing  with  change, 
technology  leadership  deals  almost  exclusively  with  new  procedures,  policies,  and  situations”.  They  identified  the 
potential  benefits  of  good  technology  leadership,  that  includes,  “improved  academic  achievement  by  students, 
improved  student  attendance  and  reduced  attrition,  better  vocational  preparation  of  students,  more  efficient 
administrative  operations,  reduced  teacher/staff  burnout  and  turnover”.  An  addition  to  the  laudable  list  is  better 
support  and  training  for  faculty.  For  educational  technology  leadership,  having  a clear  vision  of  how  the  educational 
technology  innovation  would  be  adopted  to  produce  the  desired  changes  for  the  maximum  benefit  of  the  institution 
is  an  essential  requirement.  The  implications  for  the  ET  practitioner  is  that  only  when  they  aspire  for  and  move  into 
such  leadership  positions  can  they  wield  the  necessary  influence  that  will  support  the  better  adoption  of  new 
technologies  that  will  have  profound  impact  in  the  teaching  and  learning  process  in  higher  education. 

Aspiring  for  Leadership  Positions 

It  behooves  ET  practitioners  to  strive  for  the  leadership  positions  when  such  positions  become  available.  A 
content  analysis  of  recent  position  announcements  in  higher  education  confirms  other  findings  (Albright,  1995; 
Wright  1997;  Surrey  and  Robinson,  2001;  Johnson,  2001)  that  leadership  positions  are  gradually  opening  up.  As 
stated  previously,  ET  practitioners  hold  a small  proportion  of  the  positions.  Romiszowski  (1994)  points  out  that  as 
educational  technology  practitioners,  we  have  a great  deal  of  ejqjerience  in  evaluating  and  designing  systems  for 
patrons  and  little  experience  in  designing  for  ourselves.  It  appears,  from  current  trends  and  the  positions  many 
practitioners  hold,  that  many  of  those  in  leadership  are  not  playing  the  sponsor  role  in  the  change  process.  As 
Salisbury  and  Conner  (1994)  point  out,  “to  successfully  champion  change  in  education,  advocates  must  first  realize 
that  they  cannot  initiate  change  until  they  have  obtained  sponsorship.  Advocates  without  sponsors  result  in 
conferences  and  good  intent  but  not  substantive  change”. 

Being  in  the  right  leadership  position  will  permit  the  ET  practitioner  to  function  not  only  as  change  agents 
and  advocates,  but  also  play  the  role  of  change  sponsors.  The  advantage  of  being  a sponsor  is  that  it  puts  the 
practitioner  in  direct  control  to  initiate  change.  A change  sponsor  is,  “a  person  or  group  who  has  the  authority  and 
legitimizes  a change”  (Lick  and  Kauffman,  2001).  Further,  the  change  agent  is  a person  or  group  who  is  responsible 
for  implementing  desired  change.  Examples  in  higher  education  include:  chancellor,  VP,  Dean,  program/division 


229 


718 


director,  among  others.  Lick  and  Kaufman  (2001)  advocate  that  leaders  need  to  provide  and  implement  “a  detailed, 
structured,  disciplined  transition  plan  for  identifying  and  completing  the  major  change”. 

The  roles  that  ET  practitioners  seem  to  have  played  for  a long  time  are  the  roles  of  change  advocates  and 
targets.  As  advocates,  they  train  and  encourage  faculty  and  others  to  use  technology.  As  targets,  they  change  or 
reposition  themselves  with  any  wave  of  technology  for  pedagogy  that  is  in  vogue.  In  these  roles,  they  have  lacked 
the  authority  to  initiate,  plan,  implement,  and  evaluate  technological  change  in  the  teaching  and  learning 
environment,  in  part  because  they  are  not  playing  the  sponsor  role  in  the  change  process. 

For  educational  technology  leadership,  possessing  a clear  vision  of  how  the  educational  technology 
innovation  would  be  adopted  to  produce  the  desired  changes  for  the  maximum  benefit  of  the  organization  is  an 
essential  requirement.  Kearsley  and  Lynch  (1994)  add  a perspective  that  the  success  of  leaders  is  determined  by 
their  ability  to  understand  and  influence  organizational  cultures.  The  role  of  a leader  is  more  than  management  of 
resources.  As  opposed  to  a manager,  who  may  tend  to  be  compartmental  in  their  operation,  the  leader  is  more 
holistic  and  tends  to  view  issues  from  a systemic  perspective.  Leaders  are  expected  to  shape  the  culture  of  an 
organization  by  creating  new  visions  for  organizational  improvement  in  such  a manner  that  will  appeal  to  the  key 
stakeholders.  The  implication  is  that  when  the  ET  practitioner  assumes  this  kind  of  leadership  position,  they  can  then 
utilize  the  necessary  influence  to  support  effective  adoption  of  educational  technology  systems  in  higher  education 

Moving  into  Leadership 

Implications 

The  dual  role  of  ET  practitioners  in  instructional  design  and  leadership  comes  with  added  responsibilities 
and  calls  for  additional  sets  of  skills  (e.g.,  project  management,  consulting,  change  management)  that  may  not  be  a 
component  of  the  ET  graduate  curriculum  and  new  knowledge  domains  not  usually  associated  with  educational 
technology  (e.g.,  administration  in  higher  education,  educational  leadership,  educational  theory  and  policy).  With 
the  necessary  preparations,  educational  technologists  can  exert  considerable  influence  through  technology  in 
reshaping  educational  settings. 

Moving  into  leadership  positions  as  they  become  available  requires  preparation  on  the  part  of  the  ET 
practitioner.  Although  they  possess  the  basic  skills  as  a result  of  their  training,  ET  professionals  constantly  need  to 
update  their  skills  especially  for  management  and  leadership  in  order  to  align  themselves  to  the  opening  positions. 
The  need  for  constant  update  of  skills  and  for  the  acquisition  of  new  skills  is  essential  in  view  of  the  constantly 
changing  dynamics  in  today’s  work  environment,  including  higher  education  (Gilliland  and  Tynan,  1997).  This  calls 
for  change  in  behavior  in  those  aspiring  for  leadership  or  administrative  positions  in  ET  to  transcend  the  acquisitions 
of  traditional  skills  and  competencies  for  instructional  designers  and  reach  for  the  skills  that  will  propel  them  to 
leadership  positions.  Both  practicing  ET  practitioners  and  those  graduating  from  ET  programs  need  to  be  cognizant 
of  the  requirements  for  the  new  leadership  positions  and  to  prepare  accordingly. 

It  must  be  mentioned  that  moving  up  to  the  leadership  positions  will  have  its  challenges  and  obstacles. 

Such  positions  are  often  fraught  with  politics  and  thereby  requires  understanding  and  adaptation  to  the 
organizational  culture  (Rossett  2000).  As  Creth  (2000)  observes,  “Unlike  deans,  department  heads,  and  faculty  in 
academic  programs,  administrators  in  non-academic  service  areas... do  not  enjoy  the  protection  of  tenure  as  they 
cope  with  the  inherent  risks  of  change.”  However,  the  problems  are  not  more  challenging  than  being  in  supervisory 
or  non  leadership  positions  that  lack  the  provision  of  sufficient  resources  to  discharge  their  duties  and  protection 
from  the  consequences  of  working  in  a complex  environment. 

Benefits 


Much  benefit  accrues  with  ET  professionals  moving  up  into  leadership  positions.  The  institution,  faculty, 
students,  and  the  ET  professionals  will  all  mutually  benefit.  The  processes  that  are  likely  to  be  put  in  place  will 
better  advance  the  missions  of  the  respective  institutions.  As  stated  earlier,  technology  adoption  efforts,  faculty 
development  issues,  and  instructional  development  process  will  be  approached  from  systemic  and  holistic 
standpoints.  Policy  issues  regarding  these  areas  will  involve  those  that  understand  the  ramifications  of  the  policies  in 
the  area,  as  well  as  implications  of  wrong  policies.  This  will  help  to  foster  better  relationship  among  campus  groups 
involved  in  the  use  technology  in  the  teaching  and  learning  process. 

Recommendations:  Preparing  for  educational  technology  leadership 
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Before  assuming  any  leadership  role,  the  educational  technology  practitioner  must  be  adequately  prepared 
with  a broad  array  of  interpersonal  skills  and  abilities  in  management  and  leadership  (Surrey,  2001).  Academic  or 
professional  preparations  beyond  the  usual  boundaries  of  educational  technology  should  include  topics  in: 
organizational  change  and  theory;  higher  education  administration;  educational  theory  and  policy;  workforce 
education;  and  management  science  and  theory. 

ET  programs 

To  better  prepare  students  to  meet  the  new  demands,  educational  technology  programs  may  need  to  focus 
on  those  skills  that  will  equip  graduates  to  assume  leadership  roles  and  function  better  in  higher  education.  In 
addition  to  the  requirements  in  preparing  students  for  faculty  positions,  a track  may  need  to  be  established.  Courses 
that  will  help  students  develop  the  requisite  skills  in  leadership,  management  and  understanding  of  organizational  set 
ups  (Rossett,  2000),  change  management,  educational  systems  design,  and  project  management.  Surrey  and 
Robinson  (2001)  also  suggest  that  for  those  interested  in  positions  in  higher  education,  technological  and 
interpersonal  skills  are  very  much  desired.  Some  of  these  could  be  achieved  by  collaborating  with  other 
departments/programs  to  offer  relevant  courses  or  to  organize  workshops  and  seminars.  These  will  not  be  added  to 
the  detriment  of  the  core  instructional  development  focus  of  the  programs.  Rather,  it  will  be  in  addition  to  all  those 
competences  that  have  been  the  hallmark  of  educational  technology  programs. 

ET  Students 

Students  in  ET  programs  who  desire  to  move  into  leadership  positions  upon  graduation  may  begin  early  in 
their  programs  to  prepare.  In  addition  to  taking  relevant  courses,  they  need  to  be  familiar  with  current  positions.  Job 
descriptions  and  entry  requirements  as  they  prepare.  They  need  to  have  the  traditional  competencies  for  educational 
technologists.  It  will  be  helpful  to  stay  current  with  trends  and  to  position  themselves  for  the  desired  positions. 

ET  Professionals 

This  will  lead  to  the  expansion  of  the  roles  of  ET  professionals.  It  will  result  in  moving  from  service 
support  roles  to  roles  that  will  be  involved  in  looking  at  instructional  and  technological  problem  holistically.  To  be 
in  a better  position  to  be  successful  and  to  handle  the  challenges,  ET  professional  will  need  to  be  prepared.  Some  of 
the  possible  ways  to  prepare  include  being  involved  in  professional  development,  going  for  an  advanced  degree  if 
necessary,  supporting  current  superiors,  being  current,  having  a vision  and  a plan  for  the  new  career  direction,  and 
staying  current  in  the  field.  Try  your  hands  on  playing  leadership  role  before  your  friends  and  others  whenever  the 
opportunity  calls.  The  old  art  of  networking  should  not  be  neglected.  This  is  not  an  exhaustive  list  but  a beginning 
guide  for  the  professionals. 

Conclusion 

Higher  education  is  changing  by  gradually  responding  to  the  change  brought  about  by  technology  and  other 
related  forces.  Colleges  and  universities  have  a distinctive  culture  that  characterize  or  define  them.  Equally,  units 
within  the  organization,  which  could  be  described  as  subsystems,  exhibit  distinctive  culture  that  is  a result  of  their 
profession.  Change  resulting  from  adoption  of  learning  technology  will  evidently  affect  change  in  other  areas 
because  of  the  interrelatedness  of  the  various  subsystems  and  the  complexity  of  higher  education  as  a social  system. 
Such  intricate  and  complex  process  needs  to  be  addressed  by  professionals  who  understand  the  systems  and  can 
provide  a holistic  intervention.  It  is  our  view  that  ET  professionals  can  help  higher  education  significantly  in  this  era 
of  transition. 

Many  institutions  have  invested  heavily  in  information  technology,  and  as  a result  questions  about  the 
effectiveness  of  technology  are  being  asked.  During  the  period  of  acquiring  technologies,  many  institutions  did  not 
have  any  policies  or  any  instructional  technology  plan.  In  the  early  days  of  the  new  media,  many  institutions  did  not 
seem  to  have  any  plans  as  to  what  technologies  are  needed  and  how  to  use  them  in  instruction.  A lesson  to  be 
learned  is  that  merely  acquiring  assorted  technologies  does  not  translate  into  good  instruction.  Equally,  using 
technology  in  an  adjunct  or  desultory  manner  may  not  achieve  the  desired  learning  outcomes.  Having  a sound 
technology  plan  and  working  with  faculty  to  redesign  instruction  to  accommodate  the  new  delivery  systems,  under 
the  leadership  of  ET  professionals  will  be  beneficial  in  enhancing  the  teaching  and  learning  process  as  well  as 
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advance  the  mission  of  the  institution.  With  the  properly  prepared  ET  professional  in  place,  higher  education  will  be 
on  the  road  to  realizing  the  promise  of  technology  in  this  evolving  era  in  education. 

The  increased  involvement  of  ET  professionals  in  educational  technology  leadership  as  well  as  other 
service  areas  will  help  higher  education  as  it  strives  to  enhance  faculty  teaching,  and  technology  skills,  and 
consequently  improve  student  learning.  The  need  for  the  preparation  of  leaders  in  educational  technology  is 
encouraged  by  current  trends  that  suggest  that  some  in  the  field  are  making  inroads  into  the  leadership  and 
management  cadre.  Educational  technology  practitioners  are  gradually  advancing  in  position,  heading  ET 
departments  and  holding  other  administrative  positions. 

It  is  hoped  that  this  paper  will  draw  the  attention  of  ET  practitioners,  faculty,  and  students  to  the  necessity 
to  strive  towards  leadership  positions  in  higher  education.  Educational  technology  graduate  programs  may  in 
response  develop  specializations  in  higher  education  leadership  and  management.  Increased  numbers  of  ET  trained 
administrators  and  faculty  with  interest  in  ET  leadership  in  colleges  and  universities  would  be  a boost  to  the  field. 
The  technological  transformation  of  higher  education  will  proceed  regardless  of  whether  ET  practitioners  are 
leading  the  change.  It  would  be  beneficial  for  all  stakeholders  in  colleges  and  universities  if  ET  graduates  assume 
the  leadership  positions  that  make  key  decisions  that,  in  turn,  influence  future  generations  of  college  graduates,  ET 
practitioners,  and  hopefully  faculty,  who  continue  the  cycle. 

ET  leaders  will  also  help  in  seeking  and  selecting  the  right  ET  professionals  who  are  well  prepared  to  meet 
the  needs  of  the  institutions.  A look  at  many  job  announcements  for  instructional  technologist/designer  position 
seems  to  emphasize  more  on  hardware  and  software  skill,  and  less,  sometimes,  no  instructional  design  skills  are 
listed.  This  may  probably  be  symptomatic  of  the  lack  of  understanding  of  the  roles  and  functions  of  ET  professionals 
and  the  sets  of  skills  and  competencies  they  bring  along.  ET  professionals  in  leadership  can  help  to  address  such 
problems,  as  well  as  work  towards  the  standardization  of  job  descriptions  for  ET  positions. 
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In  1996,  Bonk  and  Cummings  set  out  to  design,  implement,  and  refine  web-based  educational  psychology 
courses  and  laboratory  experiences  offered  to  pre -service  teachers  at  Indiana  University.  Utilizing  the  American 
Psychological  Association’s  14  Learner  Centered  Principles  (1993,  1997),  Alexander  and  Murphy  (1994),  Bonk  and 
Reynolds  (1997)  Bonk  and  Cummings  implemented  the  universities  first  on-line  course  in  educational  psychology. 
Course  evaluation  and  subsequent  refinement  occurred  shortly  thereafter  Bonk  & Dennon  (1999)  and  the  course 
framework  was  considered  strong  enough  for  associate  instructors  to  implement. 

This  paper  focuses  on  the  experiences  of  two  associate  instructors  who  taught  this  web-based  course  in 
educational  psychology,  while  simultaneously  taking  a course  in  Teaching  and  Instruction,  and  consulting  with  the 
course  designers.  While  great  latitude  was  given  in  teaching  the  course,  it  was  accompanied  with  the  need  to  justify 
any  revisions.  For  the  most  part,  each  instructor  maintained  the  basic  framework  during  his  or  her  first  semester  the 
course  was  taught.  Both  associate  instructors  were  graduate  students  in  the  Department  of  Counseling  and 
Educational  Psychology  with  prior  teaching  experience,  although  neither  had  experience  teaching  an  on-line  course. 

Technical  skills  varied 

The  first  step,  according  to  Burger,  was  to  organize  an  approach  to  teaching  educational  psychology  on- 
line. This  involved  consideration  of  the  material  and  the  method.  After  reviewing  several  current  texts,  Burger 
chose  Snowman  and  Biehler’s  Psychology  Applied  to  Teaching  - 9th  ed.  (2000).  She  chose  this  text  because  Dr. 
Bonk,  one  of  the  course  designers,  wrote  portions  of  the  text  pertaining  to  the  use  of  technology,  and  because  the 
text  and  course  materials,  framework,  case  studies,  and  other  activities  were  available  and  integrated,  more  time 
could  be  used  to  review  course  material;  very  helpful  when  teaching  a new  course.  To  ma  intain  the  structured 
feature  of  the  course,  Dakwa  also  selected  the  Snowman  and  Biehler  text.  The  integration  of  technology  into  the 
textbook  by  Bonk  (2000)  made  teaching  with  the  text  on-line  very  appropriate.  Especially  for  first  time  on-line 
instructors. 


Instructional  Theory 

As  instructors,  we  realized  that  specific  instructional  approaches  adopted  could  greatly  influence  student’s 
interest  in  the  classroom  and  consequently  affect  student  learning  of  content.  - Utilizing  a research-based 
instructional  approach,  we  proceeded  to  create  our  on-line  classroom. 

Instruction,  according  to  Driscoll  (1994),  is  to  deliberately  arrange  learning  conditions  in  such  a way  that 
specific  goals  can  be  attained.  To  assist  in  systematizing  these  learning  conditions  some  instructional  theories  have 
been  set  out  to  provide  a framework  in  which  these  learning  conditions  can  be  created.  Reigeluth  (1983)  defines 
instructional  theory  as  the  identification  of  methods  that  best  provide  conditions  under  which  learning  goals  are  most 
likely  to  be  attained.  For  the  instruction  to  be  successful,  it  must  be  compatible  with  the  learning  objectives 
(Reigeluth,  1983;  Driscoll,  1994).  In  our  on-line  class,  our  syllabus  was  very  detailed,  stressing  learning  objectives, 
presenting  assignments  and  all  due  dates  (which  were  flexible),  readings,  and  description  of  all  course  requirements. 
During  the  first  physical  meeting  (it  is  highly  recommended  that  instructors  and  students  meet  at  the  beginning  of 
class  if  possible)  these  objectives  were  discussed  in  detail,  greatly  enhancing  instructor/student  interaction  during  the 
course. 

Instructional  theories  encompass  learning  theories,  in  that  it  is  the  instructional  methods  adopted  by  the 
instructor  that  will  determine  the  outcome  of  the  learning  theory  adopted.  Figure  1 shows  that  the  core  of  the 
instructional  process  is  outcome.  Choosing  the  appropriate  instructional  theories  leads  to  more  effective  use  of 
learning  theories,  and  provides  a clear  path  for  obtaining  instructional  objectives.  It  should  also  be  noted  that 
several  other  factors  contribute  to  the  learning  outcome  e.g.  student  motivation,  whether  class  is  an  elective  or 
requirement. 


Learning  Instructional 


Fi^.  1.  Relationship  between  Instructional  and  Learning  Theories  and  other  factors  in  fluencing  outcome 

Particularly  for  emerging  professionals,  it  is  imperative  to  consider  the  difference  between  desired  learning 
outcomes  and  actual  learning  outcomes  (Reigeluth,  1983).  Actual  outcomes,  Reigeluth  writes,  are  “the  real-life 
results  of  using  specific  methods  under  specific  conditions,  whereas  desired  outcomes  are  goals,  which  often 
influence  what  methods  should  be  selected”  (p.  1 5).  This  is  a crucial  point.  It  is  at  the  beginning  of  designing  the 
classroom  framework  that  the  instructor  needs  to  invest  a large  amount  of  time  selecting  specific  methods  to  be  used 
for  the  desired  learning  outcomes. 

The  Web  has  created  vast  opportunities  for  educators  and  students,  researchers,  and  practitioners. 

According  to  the  US  Dept  of  Education  2000  report,  84%  of  two-and-four  year  colleges  expect  to  offer  distance- 
learning  courses  on  the  web  in  2002.  Clearly,  the  earlier  work  of  web  educators  and  student  response  will  be 
extremely  valuable  in  the  coming  years.  While  the  basic  framework  for  the  on-line  course  in  educational 
psychology  remained  stable,  the  design  for  the  chosen  software  (SiteScape  Forum),  as  well  as  the  strategies  chosen 
to  present  the  material  and  specific  content  for  the  course,  continues  to  evolve.  The  growth  of  teaching  on  the  web 
continues  to  expand  and  it  is  imperative  to  be  aware  of  the  various  pedagogical  strategies  being  developed  for  on- 
line teaching;  strategies  that  have  been  used  to  harness  the  current  technological  advancement  and  provide  efficient, 
effective,  and  creative  opportunities  in  education  and  training. 

Duffy  and  Jonassen  (1 992)  noted  that  objective  perspectives  have  for  some  time  shaped  instructional 
design  practice.  They  suggested  that  constructivism  provides  an  alternative  basis  for  conceptualizing  instructional 
experiences.  In  designing  an  effective  and  efficient  distributed  learning  interface,  the  following  seven  criteria 
should  be  considered:  extensive  interaction;  flexible  structure;  multiple  resources;  transparent  interface;  learner 
control;  attention;  and  satisfaction.  In  addition,  one  might  note  that  the  on-line  format  for  this  educational 
psychology  course  also  enhances  student  information  processing  ability,  strengthens  approach  to  novel  tasks,  and 
extends  the  classroom  beyond  the  four  walls  by  incorporating  rich  resources  from  the  world  wide  web  and  other 
non-traditional  methods. 

When  teaching  in  a medium  that  does  not  support  any  physical  interaction  with  students,  instructors  should 
structure  content  so  that  ideas  are  related  and  attention  is  given  to  the  sequencing  of  information  flow  (Reigeluth  & 
Stein,  1983)  with  special  consideration  for:  selections,  sequencing,  synthesizing,  and  summarizing  (the  four  S’s). 
Structure  and  clarity  is  always  important  in  an  on-line  environment,  but  particularly  so  when  students  are 
experiencing  an  on-line  course  for  the  first  time.  Reigeluth  & Stein’s  (1983)  Elaboration  Theory  of  Instruction 
suggests  that  instructors  first  present  an  epitome,  or  an  instructional  block  of  information,  to  the  students,  along  with 
a motivational  strategy.  This  would  then  be  followed  by  a level- 1 lesson,  elaborating  on  various  aspects  of  the 
epitome.  This  elaboration  serves  to  help  the  student  organize  the  content  of  the  epitome.  Finally,  detailed  content, 
reaching  the  objectives  of  the  lesson  is  presented.  By  systematically  presenting  the  content  and  summarizing  as  the 
lesson  is  presented,  students  are  able  to  assimilate  and  accommodate  the  content  as  the  lesson  proceeds. 

The  ARCS  model  of  motivational  design  integrates  a wide  range  of  theoretical  perspectives  including  (but 
not  limited  to)  social  learning  theory,  decision  theories,  locus  of  control,  and  attribution  theory.  In  addition,  this 
theory  for  instructional  design  acknowledges  environmental  and  humanistic  views,  as  well  as  cognitive  dissonance 
and  learned  helplessness.  The  acronym  ARCS  stands  for  Attention,  Relevance,  Confidence,  and  Satisfaction;  all- 
important  aspects  of  motivation.  The  on-line  environment  presents  particular  challenges  for  instructors  when 
considering  issues  of  motivation.  To  begin,  students  tend  to  perform  better  in  on-line  classes  if  they  have  developed 


a degree  of  self-motivation  and  self-regulation.  It  is  much  too  easy  to  put  off  course  requirements  for  an  on-line 
course.  Students  quickly  learn,  however,  that  is  also  quite  difficult  to  get  caught  up.  During  the  orientation  sessions 
and  especially  the  first  two  weeks  of  a web  course,  an  instructor  must  remain  very  sensitive  to  indications  that  a 
student  is  struggling  with  issues  of  self-discipline.  Early  involvement  and  interesting  activities  help  significantly. 
Having  a choice  in  the  activities  and  projects  also  increases  motivation  and  helps  the  student  become  invested  in 
their  own  learning. 

The  On-Line  Experience 

To  begin,  the  first  semester.  Dr.  Bonk  (course  designer)  and  Burger  (associate  instructor)  co-presented  the 
student  orientation  for  in-resident  students.  Dr.  Bonk  led  the  general  orientation  and  then  each  section  of  the  course 
adjourned  to  separate  computer  labs  for  the  individual  section  orientation  sessions.  General  orientation  addressed 
the  field  practicum  component  of  the  course,  while  section  orientation  in  the  computer  labs  provided  the  opportunity 
for  students  to  become  acquainted  with  each  other  and  course  issues  such  as:  course  design,  requirements,  methods 
for  evaluation,  the  syllabus,  and  technological  issues . A written  version  of  the  orientation  was  posted  on-line  for 
distance  education  students  (and  for  review  for  the  in -resident  students,  as  needed).  Students  spent  a portion  of  the 
orientation  sessions  navigating  the  web  site  to  ensure  they  understood  the  forum  layout  and  how  to  post  responses  or 
modify  them. 

The  following  semester.  Burger  and  Dakwa,  the  associate  instructors,  co-presented  the  course  orientation  in 
the  same  general  manner.  Support  from  Dr.  Bonk  was  obtained  weekly  during  a continuation  of  the  course  in 
Teaching  and  Instruction,  in  addition  to  individual  consultation.  This  course  provided  opportunities  for  associate 
instructors  to  explore  theoretical  as  well  as  practical  issues  of  teaching  and  learning,  and  provided  the  opportunity  to 
raise  particular  issues  or  problems  in  a supportive  and  collaborative  environment. 

A constructivist  approach  to  teaching  and  learning  was  evident  both  in  the  graduate  course  for  the  associate 
instructors,  and  in  the  method  used  to  scaffold  the  instructors  into  the  profession  of  teaching  Simms  and  Ponder 
(1997).  Further,  the  approach  is  evident  in  the  associate  instructors’  on-line  classrooms,  embodied  in  their 
instructional  approach  to  facilitate  learning  of  the  undergraduate  students  enrolled  in  the  on-line  course.  So  often,  a 
new  task  can  seem  overwhelming.  However,  knowledgeable  use  of  Vygotsky’s  Zone  of  Proximal  Development 
(ZPD)  with  students  at  any  level  in  their  development  greatly  lessens  any  sense  of  bewilderment.  Mentoring  and 
apprenticeship  were  very  effective  tools  enabling  an  associate  instructor  to  grow  gracefully  into  the  profession.  We 
made  it  a point  to  have  a one-on-one  meeting  with  each  student  mid-way  through  the  course  to  obtain  student  insight 
and  assist  with  any  difficulties  the  students  might  be  encountering. 

The  starting  point  of  having  a framework  from  which  to  teach  undergraduate  educational  psychology  was 
very  helpful.  The  course  evolved  from  a framework  that  included  an  interactive  textbook,  a syllabus,  case  studies, 
activities,  and  an  on-line  forum  to  serve  as  a point  of  contact  between  students  and  the  associate  instructors.  The 
forum  held  the  course  syllabus,  course  calendar,  reading  schedule,  discussion  groups,  and  places  to  post  the  student 
work,  as  well  as  grading  criteria  and  student  grades.  There  were  never  any  questions  about  what  assignments  had 
been  received,  when  they  were  due,  or  how  many  points  the  student’s  had  earned  for  their  work,  as  this  information 
was  available  via  the  on-line  forum. 

The  opening  screen  for  the  SiteScape  Forum  was  rather  simple,  offering  students  choices  such  as: 
Discussion,  Weekly  Work,  Journal,  Projects,  or  Cafe.  From  there,  if  students  selected  Discussion,  they  would  then 
choose  Discussion  Week  1,  Discussion  Week  2,  etc.  The  Cafe  held  lists  of  which  students  were  responsible  to 
facilitate  each  discussion.  During  the  orientation  session,  students  signed  up  to  co-facilitate  two  weekly  discussions. 
The  Cafe  also  held  “extra’s”  such  as  votes  taken  on  classroom  issues,  current  events  pertaining  to  course  material, 
and  so  on.  Designed  in  this  way,  navigation  was  very  straight  forward,  with  the  majority  of  students  leaving  the 
orientation  sessions  with  complete  confidence  in  their  ability  to  operate  in  an  on-line  environment.  The  course 
structure  encouraged  students  to  contribute  frequently  and  the  tasks  and  activities  were  clearly  specified. 
Occasionally  students  asked  questions  by  email,  which  was  usually  heaviest  at  the  beginning  of  the  term.  Email 
communication  was  also  used  if  students  had  sensitive  material  to  discuss  and  did  not  wish  to  post  particular 
material  on-line  for  the  rest  of  the  class  to  view. 

The  classroom  environment  was  easily  established  and  a team  spirit  was  recognizable  as  we  worked 
together  to  solve  functional  difficulties  that  occurred  from  time  -to-time.  Not  being  a technological  genius  has 
advantages!  The  students  rallied  and  we  worked  together  to  solve  problems  such  as  those  occurred  when  25  people, 
of  varying  experience  or  skill  level,  attempt  to  work  together  on-line. 

Together,  as  a class,  we  adjusted  when  the  publisher’s  website  (containing  optional  weekly  activities) 
experienced  technical  difficulties  and  went  off-line.  We  adjusted  due  dates  as  needed  to  recognize  the  flexibility  so 
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desirable  in  on-line  courses.  We  examined  the  impact  of  due  dates  and  late  work  both  from  a teacher’s  and  a 
student’s  perspective.  Together  we  explored  issues  of  assessment  and  evaluation,  and  all  benefited  from  sharing 
each  other’s  views  enabled  by  the  open,  authentic  learning  environment.  Pre-service  teachers  gained  experience  by 
co-facilitating  weekly  assignments  during  the  on-line  class.  They  created  optional  weekly  activities  for  the  other 
students.  Case  studies,  in  addition  to  field  experiences,  provided  authentic  scenarios  for  student  reflection.  Other 
student’s  of  Dr.  Bonk  had  written  many  of  these  case  studies  previously.  Mistakes  made  were  turned  into 
“teachable  moments”  and  every  opportunity  to  discuss  educational  psychology  in  terms  of  the  learning  we  were 
accomplishing  because  of  our  particular  course  design  greatly  added  to  the  content  of  the  textbook,  journal  articles, 
and  field  experiences  in  elementary  or  secondary  classrooms.  Duffy  & Cunningham,  (1996)  assert  that  “learning 
becomes  a matter  of  change  in  relation  to  the  culture(s)  to  which  one  is  connected  - with  the  gradual  transformation 
of  one’s  means  of  constructing  one’s  world  as  a function  of  the  change  in  membership  in  that  culture”.  Culture  in 
this  sense  is  broad.  The  culture  in  which  the  learner  finds  him-  or  herself  will  determine  the  kinds  of  knowledge 
creation  that  will  occur.  We  hope  that  a culture  of  collaboration  and  willingness  to  explore  divergent  theories  in  a 
safe  environment  will  be  incorporated  and  passed  on  because  of  experiences  such  as  these. 

Open  journals  pertaining  to  the  student’s  field  experiences  illuminated  both  fears  and  hopeful  expectations 
regarding  first  experiences,  as  novice  teachers,  in  the  classroom.  Students  shared  their  views  in  the  open  journals, 
and  emailed  one  another.  An  interesting  activity  that  was  used  as  an  icebreaker  during  orientation,  involved  students 
writing  one  of  their  fears  when  entering  their  classroom  for  the  first  time.  Responses  included  “being  boring,”  “not 
knowing  the  material,”  and  “the  kids  won’t  like  me.”  At  the  end  of  the  course,  when  we  met  to  view  student 
presentations,  the  cards  were  brought  back  out  and  discussed.  Most  students  agreed  that  they  enjoyed  hearing  about 
the  other  students’  fears,  and  were  greatly  relieved  when  most  of  their  fears  did  not  actualize.  While  distance 
education  students  did  not  attend  the  orientation,  or  closing  meetings,  transcripts  of  the  events  were  posted  for  their 
review. 

Finally,  one  insight  gained  from  this  experience,  is  that  it  is  most  important  to  “catch”  students  before  they 
fail.  Sometimes  the  technological  demands  are  too  confusing  for  beginning  students.  I lost  one  student  the  first 
semester,  as  1 didn’t  realize  the  extent  of  her  difficulties  until  she  was  hopelessly  behind.  To  remediate  this,  I 
instituted  a “mid-term  check-up”  in  which  I met  individually  with  each  student.  We  discussed  their  general 
progress,  problems  and  insights,  plans  for  their  class  project,  etc,  and  this  became  a strategy  I will  use  whether 
teaching  on-line  or  in  a traditional  classroom. 

Using  the  American  Psychological  Association’s  Learner  Centered  Principles  and  a socio -cultural 
constructivist  approach  to  on-line  teaching  provides  the  opportunity  for  students,  at  any  level,  to  recognize  “the 
legitimacy  and  limits  of  the  vast  array  of  approaches”  (McCaslin  & Hickey,  2001)  to  teaching  and  learning 
educational  psychology.  Further,  a structured  framework  such  as  that  provided  by  Bonk  and  Cummings  (1998) 
provides  excellent  scaffolding  to  support  an  associate  instructor’s  emerging  skills  when  teaching  undergraduates  in 
an  on-line  environment.  As  the  undergraduates  integrate  knowledge  of  the  content  of  educational  psychology,  in  the 
context  of  an  on-line  environment,  the  elementary  and  secondary  students  they  teach  will  receive  the  benefits  of  an 
expanded  approach  to  traditional  teaching.  In  this  constructivist  environment,  cognitive  flexibility  is  enhanced. 
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Abstract 

In  an  attempt  to  provide  a central  repository  of  online  services,  offer  services  not  currently  being  provided 
and,  through  some  form  of  distance  education,  increase  enrollment  and  therefore  revenue,  a small  northern 
New  York  college  investigates,  recommends  policy  for,  and  implements  an  online  course  management 
system.  This  paper  details  our  procedures,  challenges,  and  experiences  with  the  implementation  of 
BlackBoard  5.0.  Recommendations  are  made  to  help  similar  institutions  avoid  some  of  the  mistakes  we 
made  when  considering  use  of  such  a system.  We  advocate  a systematic  reasoned  approach  to 
implementation. 

Introduction 

This  is  a case  study.  It  is  a story  about  the  planning  decisions  and  implementation  strategies  for  a course 
management  software  package  in  a small  northern  New  York  College.  The  planning  phases  and  first  semester  that 
the  course  management  software  was  put  online  is  under  consideration  in  this  study.  One  of  the  researchers  was 
hired  in  part  because  his  main  suit  was  distance  education  and  is  of  a unique  perspective  to  look  at  the  college's  first 
semester  experiences  as  it  related  to  the  last  two  or  more  decades  of  research  in  to  distance  learning.  The  data 
collected,  outside  of  being  interesting  from  a narrative  perspective,  serves  as  a cautionary  tale  for  institutions  of 
approximately  our  size  (between  4000  and  5000  students).  It  shows -with  the  best  intentions  of  everyone  involved- 
how  a school  unversed  in  the  constructs  and  constraints  of  distance  delivery  both  succeeded  and  failed  during  its 
first  semester  using  course  management  software,  in  our  case  BlackBoard  5.0 

Planning  and  Acquisition 

Although  the  reason  for  bringing  BlackBoard  into  our  school  was  not  clarified  to  the  faculty  in  specific 
terms,  there  was  a general  sense  of  the  needs  of  the  faculty  as  demonstrated  by  their  piecemeal  efforts  at  adding 
electronic  components  to  their  courses.  Some  of  these  efforts  included  placing  course  syllabi  on  the  web, 
communication  among  students  in  a discussion  forum,  and  transferring  files  between  students  and  faculty.  Prior  to 
BlackBoard,  each  of  these  services  was  available  and  supported  separately.  Online  syllabi  were  created  by  the 
faculty  member,  and  placed  in  her  personal  web  space.  Discussion  forums  were  handled  via  newsgroups,  which 
could  be  made  available  off  campus  but  were  this  mean  that  anyone,  not  just  the  students  in  the  class,  could  use 
them.  File  transfer  was  handled  primarily  via  e-mail  attachments  for  off-campus  users,  while  on  campus  each 
instructor  could  create  a shared  space  where  files  could  be  made  available  to  students  as  well  as  space  where 
students  could  submit  their  files  to  the  instructor.  One  of  the  pushes  for  course  management  software  was  that  there 
was  a general  notion  that  it  could  do  all  of  these  things  and  more,  in  an  integrated  package.  Specifics  of  what  “more” 
constituted  was  not  clearly  explicated  or  fully  questioned.  Also  not  fully  questioned  was  how  various  features  were 
implemented. 

The  College  had  a mission  of  increasing  its  reach  to  geographically  diverse  locations  outside  of  the 
traditional  mix  of  mostly  in -state  students.  Along  with  augmenting  the  instruction  given  to  our  current  students, 
there  was  a sense  that  this  would  "open  the  door"  to  future  inroads  in  delivering  distance  education  courses.  This  in 
turn  would  mean  increased  enrollment  and  ultimately  increased  revenue.  Finally,  there  was  a sense  that  other 
institutions  were  using  course  management  software,  and  that  if  we  did  not  adopt  something,  we  would  be  "left 
behind.” 

Among  the  various  competing  course  management  options,  the  college  saw  BlackBoard  as  a solution  that 
addressed  both  the  demonstrated  faculty  needs  and  future  distance  education  possibilities.  Under  consideration  were 
FirstClass,  TopClass,  and  WebCT.  FirstClass  was  seen  as  not  containing  enough  features  such  as  an  online  grade 
book.  TopClass  had  been  tried  previously  in  a small  test,  ending  in  disastrous  results.  WebCT  was  considered  to 
have  a higher  learning  curve  than  BlackBoard.  BlackBoard  was  marketed  as  being  the  most  stable,  having  the  best 
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feature  set,  and  having  the  easiest  learning  curve  of  the  competition.  Price  was  also  considered,  and  the  most  basic 
level  of  BlackBoard,  level  one,  was  quite  inexpensive  when  compared  to  the  other  course  management  systems.  A 
one-year  license  allowing  us  to  put  every  student  and  faculty  member  on  BlackBoard  cost  a mere  $5000.  As  an 
institution  progressed  up  the  levels,  the  price  increased  dramatically,  but  that  was  not  a concern  for  initial 
implementation.  BlackBoard  5.0  was  purchased  in  the  summer  of  2000.  Unfortunately  the  BlackBoard  approved 
server,  did  not  arrive  until  November  of  2000,  too  late  for  implementation  in  the  Fall  2000  semester. 

Implementation 

Having  been  promised  BlackBoard  for  the  fall,  an  incoming  faculty  member,  acting  on  the  advice  of 
representatives  of  BlackBoard,  decided  to  put  a number  of  courses  on  the  BlackBoard.com  site.  BlackBoard  allows 
anyone  to  put  courses  on  their  website  to  demonstrate  the  capabilities  of  their  system  as  well  as  to  act  as  a marketing 
tool.  This  became  problematic  for  a number  of  reasons.  Blackboard  has  enjoyed  quick  growth,  however  that  growth 
has  created  increased  Internet  traffic,  which  puts  quite  a strain  on  its  server,  creating  a situation  of  poor  connectivity. 
Attempting  to  access  a course  on  the  website  during  hours  when  both  the  east  and  west  coast  were  logging  on  often 
resulted  in  not  being  able  to  connect  at  all.  If  a class  was  dependent  on  BlackBoard.com,  it  was  soon  found  that  the 
class  would  have  to  make  other  arrangements.  This  was  both  totally  unacceptable  and  created  a bad  example  for 
both  students  and  faculty.  When  connectivity  could  be  achieved,  the  limited  feature  set  of  the  site,  which  we  later 
found  out  was  utilizing  BlackBoard  level  2,  and  no  technical  support  increased  the  problem.  At  the  end  of  the 
semester  BlackBoard,  who  had  previously  agreed  to  archive  these  courses  for  transferal  to  our  local  server,  would 
not,  the  faculty  member  was  told  by  our  instructional  support  person,  respond  to  repeated  phone  calls. 

BlackBoard  5.0,  freshly  installed  on  our  Dell  server,  was  rolled  out  for  the  Spring  2001  semester.  At  first 
the  Blackboard  server  was  not  meant  to  provide  distance  learning  in  a traditional  sense,  rather  it  was  put  in  place  to 
enhance  the  classroom  experience  (Dede,  1997).  The  initial  meetings  of  the  BlackBoard  advisory  committee  thought 
it  wise  to  allow  a low  number  (no  more  than  five)  of  instructors  use  the  system  in  the  first  semester.  This 
recommendation  was  made  since  it  was  clear  to  the  majority  of  the  committee  that  the  first  semester  was  to  be 
considered  a beta  test,  where  any  kinks  or  bugs  typical  with  implementing  new  software  would  be  experienced  by  a 
small  number  of  faculty.  These  faculty  members  would  then  in  turn  be  responsible  for  reporting  the  problems  to  be 
fixed  before  the  following  semester,  when  BlackBoard  would  be  made  available  to  a larger  group  faculty. 

There  were  two  persons  put  in  charge  of  the  BlackBoard  initiative.  One  person  was  intended  to  provide 
instructional  support  to  the  professors  interested  in  using  BlackBoard;  the  other  would  be  in  charge  of  maintaining 
the  BlackBoard  server.  Unfortunately,  the  instructional  support  person  did  not  share  the  same  vision  as  the  advisory 
board.  BlackBoard  was  offered  by  the  instructional  support  person,  to  all  of  the  roughly  250  full-time  faculty 
members  as  something  to  try  if  they  so  desired.  This  ecumenical  concept  was  contradictory  to  the  advisory 
committee  who  had  warned  that  the  first  semester  should  be  devoted  to  debugging  the  system.  Forty-nine  faculty 
members  decided  to  use  BlackBoard  in  some  capacity  for  their  courses. 

BlackBoard  was  'sold'  to  the  faculty  as  a seeming  panacea,  where  courses  could  easily  be  put  online,  and 
there  would  be  sufficient  support  for  those  that  decided  to  take  the  plunge.  As  it  turned  out,  people  using  more  than  a 
smattering  of  BlackBoard  spent  many  hours  revamping  their  courses  and  a good  deal  more  time  than  many  expected 
in  course  maintenance  (Palloff  8c  Pratt,  1999).  The  instructional  support,  unfortunately,  was  not  sufficient  to  meet 
this  demand,  a situation  quite  common  in  such  situations  (Berge,  1995;  Ellsworth,  1995;  Green,  1998).  The 
instructional  person  did  not  know  enough  about  the  BlackBoard  environment  to  adequately  support  the  large  number 
of  faculty  using  the  system.  This  person  had  no  formal  training  other  than  being  given  the  manual  and  being  sent  to 
a BlackBoard  Conference.  In  a real  sense  training  was  situational  and  ‘on  the  job*.  What  was  expected  to  be 
instructional  support  became  technical  support  since  the  support  person’s  time  was  spent  training  faculty  and  dealing 
with  situational  problems.  On  top  of  that,  the  server  support  person  was  not  ready  to  support  demand  created  by  the 
immediate  presence  of  forty-nine  faculty  members  and  over  1000  BlackBoard  users.  During  the  first  few  weeks  the 
server  froze  if  more  than  a handful  of  people  tried  to  access  the  same  information  at  the  same  time.  This  became 
more  of  a problem  since  all  graduate  courses  start  at  5:30pm  and  technical  support  ends  their  day  at  4:00pm,  so  if 
the  system  became  unusable,  help  was  not  provided  until  the  next  day  leaving  any  number  of  classes  floundering. 
There  was  no  training  for  students  provided  by  anyone  other  than  faculty  who  were  less  than  comfortable  with  using 
BlackBoard  themselves. 

As  the  semester  wore  on,  frustration  grew  as  the  instructors  who  were  sold  on  this  'wave  of  the  future' 
became  frustrated  with  technical  fixes  that  never  came  fast  enough.  To  add  to  this,  they  also  began  to  rethink  why 
they  were  using  BlackBoard  in  the  first  place,  as  the  answer  to  this  salient  question  was  not  apparent.  Some 
instructors  (Winograd,  2000)  became  angry  over  the  situation  and  decided  to  ditch  BlackBoard  altogether,  some 


240 


729 


stuck  with  it,  and  had  good  success  with  it.  The  majority  of  those  experiencing  success  were  early  adopters  of 
technology,  had  a basic  sense  of  what  to  expect  through  experience  with  either  BlackBoard  or  another  course 
management  system,  or  started  small,  using  one  or  two  features  and  building  from  small  victories. 

Recommendations 

It  is  our  hope  that  our  experiences  may  well  help  other  institutions  of  our  size  understand  the  pushes,  pulls, 
and  personality  aspects  of  the  implementation  of  a course  management  system.  Careful  consideration  should  be 
given  to  'jumping  on  the  bandwagon.’  Looking  at  this  from  a critical  perspective  is  invaluable  to  prevent  institutions 
from  becoming  cattle  following  the  herd  (Winograd,  2000). 

It  is  critical,  absolutely  vital,  that  the  demonstration  projects  be  of  the  highest  possible  standard,  since 
failure  or  mediocre  results  will  have  exactly  the  opposite  effect  from  what  is  desired.  For  this  reason  it  is 
imperative  that  financial,  technological,  and  human  resources  be  ruthlessly  focused  (Moore  & Kearsley, 
1996). 

To  this  end,  we  would  like  to  give  some  basic  recommendations  to  those  about  to  go  through  what  we  did: 

1 . Start  with  a Needs  Assessment. 

2.  While  many  of  our  needs  were  self-evident,  they  were  not  always  fleshed  out.  Many  assumptions 
were  made  about  what  faculty  wanted  and  what  the  course  management  software  could  deliver. 
Some  turned  out  to  be  accurate,  although  many  turned  out  to  be  incomplete  or  inaccurate.  Some  of 
the  salient  questions  we  never  asked  directly  included: 

• Do  we  really  need  a course  management  system? 

• What  were  our  goals  and  objectives? 

• What  were  some  experiences  from  other  institutions  faced  with  the  same  issues? 

3.  A needs  assessment  would  not  only  help  us  to  answer  these  questions,  it  also  would  help  us  to 
raise  questions  that  were  not  necessarily  self-evident. 

4.  Have  a timeline  for  partial  and  full  implementation  with  significant  stages  identified.  It  is  critical 
that  the  partial  implementation  phase  be  used  as  a test  phase  for  working  out  the  many  bugs,  both 
from  a technical  support  and  instructional  support  standpoint.  By  building  from  small  successes, 
and  learning  from  problems,  the  pilot  stage  can  lead  to  a successful  full  implementation  phase. 

5.  Have  sufficient,  qualified  technical  and  instructional  support  personnel.  This  is  absolutely  critical, 
yet,  in  reality,  perhaps  the  hardest  to  assure  given  the  limited  human  resources  of  many  campuses. 
It  would  help  if  these  personnel  were  acquired  via  a search,  rather  than  via  administrative 
appointment. 

6.  Make  sure  you  have  strong  administrative  support,  and  that  the  decision  making  process  is  a 
shared  decision,  not  in  the  hands  of  a few  individuals. 


Conclusion 

In  conclusion,  it  could  have  been  better,  and  it  could  have  been  worse.  We  asked  many  of  the  right 
questions,  and  had  in  many  regards  a well-considered  approach.  Our  failure  was  that  although  many  useful 
recommendations  were  made,  very  few  of  them  were  followed.  A reasoned  approach  is  far  more  helpful  than 
jumping  in  and  seeing  what  happens.  In  hindsight,  it  was  a lack  of  coordination  and  consistency  in  the  process  that 
led  to  the  majority  of  our  problems.  It  gives  us  hope  that  others  may  benefit  by  learning  from  our  experiences. 
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Abstract 

The  paper  reports  action  research  on  web-based  courses  conducted  from  1999  to  2001.  The  goal  of  the 
research  was  to  investigate  students ' opinions  toward  such  courses  and  to  examine  the  impact  of  using 
different  strategies  on  online  communities.  Data  collection  relied  on  online  discussion  messages, 
observations,  interviews,  and  surveys.  Both  quantitative  and  qualitative  methods  were  utilized  in  this  study. 
The  results  revealed  that  students  and  the  instructor  were  in  favor  of  web-based  courses  and  that  the  use  of 
moderating  strategies  greatly  influenced  online  communities. 


Introduction 

“How  many  online  (web-based)  courses  does  your  program  offer?”  “How  often  do  I have  to  come  to 
campus  for  my  course  work?”  Such  questions  are  now  being  asked  much  more  frequent  than  ever  before,  and 
answers  to  such  questions  have  become  factors  for  learners  to  select  their  institutions  and  for  institutions  to  attract 
their  students.  This  fact  is  changing  our  society  and  shifting  our  educational  paradigm.  In  response  to  the  paradigm 
shift,  many  educators  offer  web-based  courses.  Some  educators  have  high  regards  for  such  courses  because  they 
offer  opportunities  to  people  who  cannot  receive  education  otherwise  due  to  large  distance  between  home  and 
school.  Some  view  it  as  an  alternative  that  provides  learners  with  options  of  learning.  Some  even  expect  virtual 
classrooms  to  be  the  future  of  education.  Meanwhile,  other  professionals  doubt  the  value  of  such  education  and 
question  its  quality.  Do  students  favor  such  courses?  What  are  benefits  and  barriers  of  such  a delivery  format? 

How  can  we  make  such  courses  effective?  To  answer  these  questions,  more  research  on  this  topic  is  needed. 

"Online  courses,"  "completely  online  courses,"  "web-based  courses,"  "web  supplement  courses,"  "hybrid 
courses,"  and  some  other  terminology  are  used  by  professionals.  Some  educators  consider  courses  containing  online 
features  like  synchronous  or  asynchronous  communication  to  be  online  courses.  According  to  this  definition, 
courses  in  which  instructors  use  these  features  but  still  meet  students  on  a regular  basis  would  be  considered  to  be 
online  courses.  However,  other  educators  disagree  and  only  view  courses  in  which  instructors  meet  students  half  or 
less  than  half  of  the  regular  scheduled  meeting  time  during  a semester  (quarter)  to  be  online  courses.  Further,  even 
other  professionals  only  recognize  courses  in  which  instructors  and  students  do  not  meet  at  all  to  be  online  courses. 
To  avoid  confusion,  the  author  uses  the  expression  "hybrid  courses"  in  this  article  to  refer  to  the  courses  in  which  an 
instructor  and  students  meet  half  or  less  than  half  of  the  regular  scheduled  meeting  time,  and  such  courses  are  the 
courses  involved  in  this  research. 

In  this  article,  the  author  first  introduces  background  that  illustrates  why  the  author  conducted  the  research. 
She  then  describes  her  hybrid  courses  and  strategies  used  in  these  courses  in  order  to  help  readers  understand  the 
structures  of  the  courses.  After  that,  she  reports  the  action  research  including  data  collection,  data  analysis,  and 
results.  Discussions  and  recommendations  are  also  provided  in  this  article. 

Background 

The  author  is  associate  professor  of  Instructional  Technology  at  a state  university  in  the  United  States  of 
America.  The  majority  of  students  in  her  study  was  in-service  teachers  and  worked  full  time.  They  sometimes 
could  not  attend  classes  because  of  their  responsibilities  at  schools,  for  examples,  open  house  day,  parents  day, 
coach  seasons,  or  because  of  their  obligations  to  their  families.  When  they  came  to  class,  they  often  were  exhausted 
and  hungry  because  they  just  rushed  to  the  class  straight  from  their  schools,  where  they  had  been  working  for  at  least 
eight  hours.  They  sometimes  had  to  call  home  to  check  if  their  children  were  fine  or  had  to  leave  classes  early 
because  nobody  was  taking  care  of  his/her  family.  Observing  all  the  demands  that  the  students  encountered,  the 
author  realized  that  these  students  were  different  from  the  students  whom  she  had  taught  before.  The  ones  she  was 
teaching  were  reentry  students  or  adult  learners  while  the  ones  she  had  taught  before  were  traditional  full-time 
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students.  She  wondered  if  a traditional  classroom  provided  her  reentry  students  with  good  learning  conditions  and  if 
another  educational  delivery  format  would  work  better  for  them. 

While  she  was  pondering  this  issue,  she  received  one  of  her  university  grant  proposal  calls,  and  this  grant 
was  designed  to  support  faculty  to  convert  traditional  courses  to  online  or  hybrid  courses.  She  applied  for  the  grant 
with  the  intention  to  examine  if  such  a delivery  format  would  benefit  her  students.  She  received  the  grant  in  the 
spring  of  1999,  converted  a graduate  course  to  a hybrid  course  during  the  summer,  and  implemented  it  during  the 
fall  quarter  of  1999.  She  received  positive  feedback  from  her  students  and  continued  teaching  two  additional  hybrid 
graduate  courses  during  1999-2000.  All  these  three  courses  were  offered  at  two  Campuses  and  offered 
again  during  2000-2001 . During  these  two  years,  she  collected  data  from  her  students  and  conducted  research 
on  online  instruction.  Her  hybrid  courses  and  the  research  are  described  below. 

Instructional  Design  and  the  Hybrid  Courses 

As  Grabowski  (2001)  stated,  instructional  design  is  closely  connected  to  the  beliefs  of  the  course  designer. 
Some  instructors  structure  an  online  course  as  an  independent  study,  and  students  receive  credits  as  long  as  they 
complete  course  assignments.  Accordingly,  students  could  obtain  credits  at  any  time  during  the  semester  and  have 
no  interaction  with  other  students  of  the  course.  On  the  other  hand,  some  educators  design  an  online  course  in  a way 
that  students  have  to  participate  in  ample  interactive  activities  with  others,  such  as  their  classmates.  Instructors 
using  the  former  structure  might  view  independent  study,  one  of  the  four  aspects  of  distance  learning  (Keegan, 

1993),  as  an  important  aspect  while  educators  using  the  latter  one  might  consider  interaction  and  communication  to 
be  essential. 

The  author  of  this  article  supported  the  latter  when  she  designed  her  courses.  She  believed  that  social 
interaction  plays  an  important  role  in  learning  (Vygotsky,  1978)  and  valued  active  learning  and  meaningful  learning 
(Grabe  & Grabe,  2001;  Brown,  1992;  Knapp  & Glenn,  1996;  Means  et  al.,  1993).  Agreeing  with  Palloff  and  Pratt 
(1999),  she  regarded  an  online  community  as  crucial  in  a hybrid  course  and  designed  her  courses  in  a way  that  her 
students  had  to  interact  with  other  students  to  enhance  learning.  Like  Oliva  (2000)  and  Santema  & Genang  (2000), 
she  encouraged  students'  active  learning  and  invited  students  to  construct  course  materials  together  with  her.  She 
met  her  students  four  times  throughout  the  quarter — at  the  beginning,  in  the  middle,  and  at  the  end.  All  assignment 
submissions  and  discussions  were  conducted  online  via  WebCT.  She  constructed  a variety  of  forums  (discussion 
boards)  for  students  to  communicate  with  each  other  and  to  share  resources,  for  example,  forums  for  making 
announcements,  for  asking  questions  and  receiving  help  concerning  technical  issues,  for  submitting  assignments, 
and  for  providing  feedback  and  critiques  to  their  classmates.  In  all  her  courses,  students  had  to  post  their 
assignments  to  the  forums,  review  assignments  of  their  classmates,  and  critique  each  other's  assignments.  These 
activities  provided  students  with  basic  channels  for  communication. 


1999-2000 

2000-2001 

(without  moderating) 

(with  moderating) 

F 

Synchronous 

(Instructor  posting  topics) 

Asynchronous  (Medium-duty  moderating) 

W 

Asynchronous 
(Student  posting  topics) 

Asynchronous  (Heavy-duty  moderating) 

S 

Asynchronous 

Asynchronous  (Light-duty  moderating) 

Table  I.  Different  strategies  used  in  the  hybrid  courses.  F = Fall  quarter,  W = Winter  quarter,  S = Spring  quarter 

In  addition  to  using  assignments  to  enhance  interaction,  the  instructor  also  employed  different  strategies  to 
facilitate  students'  communication.  Table  1 above  shows  strategies  used  in  her  hybrid  courses.  She  used  moderating 
strategies  during  the  second  year  (2000-2001)  while  she  did  not  use  them  during  the  first  year  (1999-2000).  During 
the  fall  quarter  of  1999,  all  students  conducted  synchronous  communication  every  week  and  discussed  topics  that 
were  posted  by  the  instructor.  During  the  winter  quarter  of  1999,  students  used  forums  to  conduct  threaded 
discussion  about  topics  generated  by  their  classmates.  During  both  quarters,  students  obtained  participation  points 
based  on  frequency  and  quality  of  their  posted  messages.  They  were  aware  that  they  had  to  post  at  least  two 
messages  every  week  to  receive  participation  credits.  Such  participation  requirements  were  not  expected  during  the 
spring  quarter  of  1999. 
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During  the  second  year,  the  instructor  used  moderating  strategies  in  the  three  hybrid  courses  in  which 
students  took  turns  to  moderate  their  online  community  for  a week.  It  was  understood  that  the  instructor  would  not 
jump  in  to  answer  questions  unless  it  was  necessary  and  that  the  students  would  receive  credits  based  on  how  we  11 
they  moderated  the  community.  For  example,  if  a question  or  a problem  on  a discussion  board  remained 
unanswered  or  not  acted  upon,  the  moderators  of  the  week  would  be  marked  down.  Moderators  did  not  have  to 
answer  all  questions  or  solve  problems  but  they  needed  to  facilitate  discussions  about  the  issues  within  the 
community. 

Based  on  responsibilities  and  tasks  the  students  conducted,  the  moderating  strategies  varied.  The 
instructor  categorized  them  into  three  types  of  moderating:  mediumduty,  heavy-duty,  and  light-duty.  During  the 
fall  quarter  while  the  medium-duty  moderating  was  conducted,  moderators  posted  discussion  topics,  hosted  online 
discussions,  and  answered  questions  posted  by  their  classmates.  During  the  winter  quarter  while  the  heavy-duty 
moderating  was  conducted,  moderators  not  only  took  on  the  same  responsibilities  as  in  the  previous  quarter  but  also 
assigned  readings,  generated  rubrics,  and  further  evaluated  their  classmates’  online  performance.  During  the  spring 
quarter  while  the  light-duty  moderating  was  conducted,  moderators  only  needed  to  host  online  discussion  and 
answer  questions  posted  by  their  classmates. 

The  Research  Study 

The  goals  of  the  research  were  to  investigate  students’  opinions  toward  hybrid  courses  and  to  examine  the 
impact  of  using  different  strategies  on  online  communities. 

Subjects 


The  participants  of  the  study  were  the  students  in  the  twelve  hybrid  courses  during  1999-2001.  They  were 
in-service  teachers  who  were  pursuing  their  Master’s  degree  in  Instructional  Technology  at  the  university  and  had 
little  experience  with  hybrid  courses. 

Methodology 

At  the  beginning  of  each  course,  the  instructor  explained  to  the  students  the  course  and  their 
responsibilities,  especially  their  duties  related  to  the  different  strategies  used  in  an  online  community.  During  the 
courses,  the  instructor  posted  questions  for  discussion.  Examples  of  questions  included  (1)  What  are  benefits  and 
barriers  of  hybrid  courses?  (2)  What  are  advantages  and  disadvantages  of  conducting  synchronous  and  asynchronous 
communication  in  hybrid  courses?  (3)  Do  you  like  the  moderating  strategies  used  in  this  course?  and  so  forth. 

During  the  last  meeting  of  each  course,  students  filled  in  a survey  that  contained  10  Likert  scale  (1-4)  questions  and 
open-ended  questions.  The  Likert  scale  questions  were  guided  to  examine  the  following  points:  (1)  Compared  to  a 
traditional  class,  did  students  feel  that  they  learned  as  much  as,  or  even  more,  in  the  hybrid  course?  (2)  Compared  to 
a traditional  class,  did  students  feel  that  they  spent  as  much  as,  or  even  more,  time  preparing  for  the  class?  (3) 
Compared  to  a traditional  class,  did  students  feel  that  they  were  motivated  as  much  as,  or  even  more,  to  learn  in  this 
course?  (4)  While  taking  the  course,  did  students  have  sufficient  access  to  the  instructor?  (5)  While  taking  the 
course,  did  students  have  sufficient  interaction  with  other  students  in  this  course?  (6)  Given  the  choice  between 
traditional  courses  and  hybrid  courses,  did  students  prefer  a hybrid  course  if  the  course  content  were  suitable  for  a 
hybrid  course?  (7)  Did  students  wish  that  more  hybrid  courses  were  offered  in  the  Masters  program  at  the 
university?  (8)  Would  students  enjoy  taking  another  hybrid  course?  (9)  Were  students  concerned  about  the  quality  of 
hybrid  courses?  (10)  Did  students  like  the  delivery  format?  (11)  How  many  sessions  in  which  the  teacher  and  the 
students  meet  would  be  appropriate  for  a hybrid  course?  Open-ended  questions  of  the  first  year  focused  on  benefits 
and  barriers  of  hybrid  courses  while  those  of  the  second  year  emphasized  the  impact  of  moderating  on  online 
communities. 

In  addition  to  online  messages  and  surveys,  data  was  also  collected  from  observations  and  interviews.  The 
author  observed  students  and  activities  online  as  well  as  during  class  meetings.  Interviews  were  informal  and 
occurred  when  there  was  a need  for  clarifying  students’  comments. 

The  author  tabulated  the  survey  data.  She  also  downloaded  students'  messages  related  to  the  research, 
color-coded  messages,  and  categorized  them  into  appropriate  folders  based  on  the  topics,  for  example,  benefits  of 
web-based  courses  and  barrier  of  web-based  courses. 
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Results  and  Discussions 


The  table  below  reports  students’  opinions  toward  the  hybrid  courses  offered  from  1999  to  2000.  As 
mentioned  above,  the  first  course  was  funded  by  an  internal  grant  and  was  used  as  a pilot  to  examine  if  the  students 
preferred  a hybrid  course  to  a traditional  one.  The  survey  used  in  the  first  course  was  modified  and  improved  for  the 
upcoming  quarters,  which  explains  why  some  numbers  in  the  table  are  missing. 


First  Year 

Pilot  (F) 

2"“  (W) 

3^“  (S) 

Learn  more 

2.13 

3.08 

3.46 

Spent  more  time 

1.75 

3.51 

3.58 

More  motivated 

3.30 

3.52 

Access  to  instructor 

2.25 

3.58 

3.39 

Interaction  with  students 

3.06 

3.22 

Prefer  a WB  course 

3.38 

3.40 

3.68 

Wish  for  more  WB  courses  in  program 

2.88 

3.40 

3.55 

Enjoy  taking  another  WB  course 

2.75 

3.58 

3.55 

Concerned  about  quality  of  WB  course 

2.05 

1.64 

Like  the  delivery  method 

3.31 

3.20 

3.52 

.F  = Fall  quarter,  W = Winter  quarter,  S = Spring  quarter 

Table  2:  Students ' responses  during  1999-2000.  WB  = Web-Based  (hybrid)  course 


The  results  indicated  that  students  were  in  favor  of  a hybrid  course.  The  students  felt  that  they  learned  as 
much  as  or  even  more  in  such  course  and  that  they  were  more  motivated.  They  wished  that  more  hybrid  courses 
would  be  offered  in  the  academic  program,  and  they  preferred  a hybrid  course  to  a traditional  course. 

Most  of  the  means  on  table  2 went  up  when  time  progressed.  The  instructor  felt  that  this  fact  was  partially 
due  to  the  experiences  she  and  her  students  gained  from  the  hybrid  courses.  She  also  felt  that  certain  characteristics 
were  necessary  for  people  to  succeed  in  such  courses,  for  example,  being  self-disciplined.  Since  the  students  and  the 
instructor  did  not  meet  every  week,  students  had  to  be  self-disciplined  and  be  able  to  complete  tasks  without  much 
supervision.  More  than  half  of  the  students  revealed  in  their  surveys  and  interviews  that  online  learners  should  be 
self-disciplined  and  complete  tasks  on  time.  Four  students  who  received  incomplete  during  the  first  quarter  pointed 
out  a need  of  such  characteristic.  One  of  them  said,  *T  have  difficulty  in  the  online  [hybrid]  course  because  I’m  not 
so  disciplined  and  often  postpone  my  work.  In  a traditional  course,  I would  be  reminded  every  week  when  I go  to 
class.  But  this  is  not  the  case  for  taking  an  online  [hybrid]  course.  ” 

Although  people  who  tend  to  delay  their  work  might  have  difficulty  surviving  in  hybrid  or  online  courses, 
the  instructor  believed  that  preparing  students’  mindset  before  they  took  such  courses  could  be  helpful.  Therefore 
during  the  first  meeting  of  her  courses,  she  always  asked  students  who  had  taken  her  hybrid  course(s)  before  to  share 
with  other  students  tips  of  being  online  learners.  **Don’t  postpone  your  work”  was  mentioned  repeatedly  by  the 
students. 

As  mentioned  earlier,  students  conducted  synchronous  communication  in  the  first  course  and  asynchronous 
communication  in  the  second  course.  In  both  courses,  students  received  participation  credits.  In  the  third  class, 
asynchronous  communication  was  utilized  but  no  participation  credit  was  issued.  The  different  strategies  used  in 
these  three  classes  had  an  impact  on  the  online  communities.  The  research  results  indicated  that  synchronous 
communication  strengthened  students’  sense  of  belonging.  A student  stated,  “Although  we  do  not  see  each  other 
every  week,  the  real-time  communication  makes  me  feel  we  belong  to  the  same  class.”  Such  a sense  of  belonging 
was  less  common  when  asynchronous  communication  was  used  during  the  second  quarter.  During  that  quarter,  a 
few  students  addressed  that  they  missed  real-time  communication  and  requested  the  instructor  to  sometimes  conduct 
such  communication  while  they  were  enjoying  the  flexibility  that  asynchronous  communication  provided. 

Observing  the  three  courses,  the  instructor  found  that  the  online  community  of  the  third  course  seemed  to  be  loose 
and  thought  that  issuing  no  participation  credits  might  have  contributed  to  the  loose  community.  The  instructor 
suggests  online  instructors  employ  asynchronous  communication  and  synchronous  communication  alternatively  and 
use  participation  credits  to  motivate  students  participating  in  online  communities. 

Students  listed  several  benefits  of  a hybrid  course:  flexible  schedule,  being  able  to  work  at  any  time  and  at 
any  place,  and  being  able  to  choose  the  best  conditions  for  learning.  They  also  mentioned  that  hybrid  courses  saved 
them  gas  and  time  on  commuting  and  allowed  them  more  access  to  instructor  and  to  their  fellow  classmates. 
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Barriers  also  existed  in  a hybrid  course.  The  participants  missed  face-to-face  communication  and  personal 
contact.  Students  with  low  technology  skills  felt  pressured  and  anxious.  Such  pressure  and  anxiety  might  create  a 
negative  impact  on  learning.  Despite  these  barriers,  students  expressed  that  they  would  still  choose  a hybrid  course 
over  a traditional  one  if  they  had  an  option. 

Like  the  students,  the  instructor  enjoyed  the  flexibility  hybrid  courses  provided  and  missed  face-to-face 
contact  with  her  students.  Unlike  the  students,  she  experienced  tremendous  pressure  of  responding  to  students* 
messages  and  of  their  expectations  of  receiving  responses  instantly.  Meeting  students  once  a week  in  a traditional 
course  became  meeting  students  24  hours  a day,  seven  days  a week  online.  In  addition,  it  was  time  consuming  and 
stressful  to  communicate  with  a couple  of  students  who  often  got  confused  and  repeatedly  asked  the  same  questions 
no  matter  how  clear  information  was,  for  example,  on  when  an  assignment  was  due  and  when  the  next  meeting 
would  be.  Such  stress  was  eased  during  the  second  year  when  moderating  was  used  in  her  courses. 

During  2000-2001,  the  instructor  utilized  medium-duty  moderating  in  the  first  (Fall)  quarter,  heavy-duty 
moderating  in  the  second  (Winter),  and  light-duty  moderating  in  the  third  (Spring)  quarter.  As  mentioned  earlier, 
the  differences  among  the  three  types  of  moderating  were  based  on  responsibilities  and  tasks  students  conducted  in 
the  hybrid  courses.  Data  collected  from  survey  open-ended  questions,  online  discussions,  observations,  and 
interviews  indicated  that  both  the  students  and  the  instructor  favored  the  use  of  moderating  in  the  courses.  The 
students  felt  sense  of  ownership  of  their  online  communities  and  learned  from  their  peers  by  observing  how  their 
peers  hosted  the  communities.  They  received  answers  much  faster  than  before  because  every  member  of  the 
communities  tried  to  help  answer  questions.  The  instructor  also  favored  the  moderating.  She  noticed  that  she  was 
less  stressed  responding  to  students  compared  to  the  first  year  and  that  the  students  received  responses  faster  and 
became  very  active  in  the  online  communities.  The  communities  became  very  dynamic,  and  she  felt  the  courses 
sometimes  could  smoothly  move  forward  like  a car  with  a "cruise  control". 


Second  Year 

1*‘  (F) 

2"“  (W) 

3'“(S) 

Learn  more 

3.73 

3.15 

3.46 

Spent  more  time 

3.90 

3.93 

3.73 

More  motivated 

3.80 

3.33 

3.56 

Access  to  instructor 

3.83 

3.62 

3.43 

Interaction  with  students 

3.50 

3.45 

3.46 

Prefer  a WB  course 

3.81 

3.42 

3.73 

Wish  for  more  WB  courses  in  program 

3.85 

3.63 

3.63 

Enjoy  taking  another  WB  course 

3.72 

3.66 

3.70 

WB  providing  better  quality 

3.70 

3.45 

3.70 

Like  the  delivery  method 

3.70 

3.27 

3.66 

F = Fall  quarter,  W = Winter  quarter,  S = Spring  quarter 

Table  3:  Students^  responses  during  2000-2001.  JVB  = Web-Based  (hybrid)  course. 


Among  the  three  different  moderating  strategies,  the  students  favored  the  medium-duty  moderating  the 
most  and  the  heavy-duty  moderating  the  least.  During  the  winter  (heavy-duty)  quarter,  the  responsibilities  of  the 
students  and  their  activities  appeared  too  complicated.  Moderators  of  every  week  tried  to  do  a good  job  by 
assigning  readings,  setting  objectives  for  the  week,  and  facilitating  discussion;  often  their  readings  and  objectives 
were  too  many  to  be  accomplished  within  a week.  Students  conducted  many  different  activities  when  time  moved 
on.  At  the  end,  they  were  distracted  by  the  objectives  of  different  moderators  and  forgot  the  objectives  of  the 
course.  In  addition,  moderators  often  had  to  participate  in  activities  of  a week  while  they  were  still  evaluating  their 
peers'  online  performance  of  the  week  when  they  were  the  moderators.  This  was  indeed  a hectic  quarter,  and  the 
instructor  learned  a big  lesson  from  the  experience.  Appropriate  amount  of  online  communication  could  enhance 
learning  while  too  much  communication  might  cause  learners  to  withdraw  from  the  community  (Palloff  and  Pratt, 
1999).  Students'  dislike  of  such  moderating  is  clearly  revealed  in  the  table  above.  A number  of  the  means  during 
the  winter  quarter  dropped  and  consistently  appeared  to  be  the  lowest  among  the  three  courses.  Students  spent  a lot 
of  time  (mean=3.93)  on  the  course  but  did  not  necessarily  learn  more  (mean=3.15).  They  liked  the  course  the  least 
(mean=3.27)  compared  to  the  other  two  courses  of  the  year. 

Although  the  Winter  quarter  was  hectic,  students  still  favored  hybrid  courses.  Issues  addressed  during  the 
first  year,  like  benefits  of  such  courses  and  their  wish  to  have  more  hybrid  courses  in  the  program,  repeatedly 
appeared  in  the  second  year.  Students  also  liked  the  fact  that  they  had  to  post  their  assignments  and  review  their 
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peers’  work.  By  doing  so,  they  learned  much  from  their  peers.  They  liked  meeting  three  to  four  times  per  quarter 
and  did  not  seem  to  favor  an  online  course  without  a face-to-face  meeting. 

A hybrid  course  did  provide  flexibility  and  convenience  to  learners,  especially  to  learners  at  remote  areas. 
While  hybrid  courses  are  blooming  in  many  places,  the  author  thinks  that  hybrid  courses  should  not  (1)  be 
independent  studies  in  which  no  interaction  among  students  are  necessary,  (2)  be  only  task-oriented  in  which  social 
learning  is  neglected,  and  (3)  lower  the  quality  of  education.  Instead,  the  courses  should  be  structured  to  raise  the 
quality  of  education  because  learners  have  (1)  options  to  choose  their  best  learning  conditions  and  (2)  opportunities 
to  enhance  their  learning  using  resources  beyond  boundaries  of  time  and  space. 

Conclusion 

The  paper  reports  action  research  on  online  courses  conducted  from  1999  to  2001.  Data  were  collected 
from  three  courses  (12  course  sections)  at  two  campuses  during  the  two  years.  Data  collection  replied  on  online 
discussions,  observations,  interviews  and  surveys.  Both  qualitative  and  quantitative  research  methods  were  used  in 
this  study. 

The  results  indicated  that  students  and  the  instructor  were  in  favor  of  hybrid  courses  and  that  the  use  of 
different  strategies  had  an  impact  on  online  communities.  Using  synchronous  communication  and  asynchronous 
communication  alternatively  could  enrich  online  communities,  and  moderating  strategies  with  careful  design  and 
organization  worked  well  in  hybrid  courses. 

Instructors  should  employ  a variety  of  strategies  to  build  up  and  nurture  an  online  community  that  may  lead 
to  a success  of  hybrid  courses.  This  new  delivery  method  provides  students  with  options  of  choosing  their  best 
learning  conditions  and  with  opportunities  to  enhance  their  learning  using  resources  beyond  boundaries  of  time  and 
space.  One  can  see  its  potential  and  positive  impact  on  our  education  and  society. 
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Abstract 

How  can  instructional  designers  create  executive  education,  seminar  style  (non-credit)  courses 
that  are  short  (one -two  hours)  but  still  provide  a meaningful  learning  experience  for  managers 
and  executives  in  an  organization?  Engaging  and  motivating  learners  in  short  courses  without 
traditional  assessment  formats  or  external  motivators  such  as  grades  or  academic  credit  is  hard  to 
do  and  presents  a challenge  for  designers  who  have  “grown  up”  developing  academic  courses  or 
professional  training  for  traditional  programs.  This  paper  presents  several  design  solutions  that 
have  been  successful  in  this  context.  Unique  instructional  features,  specific  instructional  design 
challenges,  and  a program  of  ongoing  research  are  also  described. 


Introduction 

Executive  education  courses  are  used  to  teach  managers  and  executives  the  skills  they  need  to  succeed  in 
corporate  leadership.  The  audience  for  these  courses  is  typically  the  managers  across  a company  or  business  unit 
within  a corporation.  Common  topics  include  leadership  and  management  styles,  strategies,  and  tactics,  motivating 
employees,  project  management,  business  strategy,  and  many  others.  Executive  education  courses  differ  from 
traditional  business  academic  courses  in  that  they  are  generally  shorter,  and  do  not  offer  academic  credit  for 
successful  completion.  We  have  created  many  executive  education  courses  in  our  role  as  instructional  designers  for 
UNext,  Inc.  and  Option  Six,  Inc.  This  paper  describes  two  of  the  most  successful  course  models  we  have  developed 
and  implemented.  We  will  explain  the  pedagogical  approaches  we  have  used,  describe  relevant  course  features, 
discuss  the  instructional  design  challenges  we  have  encountered,  and  briefly  explain  our  ongoing  research  agenda. 

Executive  Education 

In  the  past,  it  was  all  too  common  for  a company  to  promote  an  employee  to  a management  position 
without  providing  any  training  or  education  targeted  at  the  new  roles  and  responsibilities  of  this  new  position.  The 
new  manager  was  expected  to  transfer  their  proven  “line”  expertise  into  the  realm  of  management  without 
retraining.  Unfortunately  this  approach  does  not  work  well  for  many  new  managers.  They  are  suddenly  expected  to 
understand  cross -functional  business  concepts  such  as  those  in  managerial  and  financial  accounting,  marketing, 
sales,  and  managing  people.  Training  and  expertise  in  one  area  of  a business  as  a line  worker  (e.g.  as  a sales 
associate,  marketing  analyst,  or  manufacturing  supervisor)  does  not  necessarily  provide  the  ability  to  excel  or  even 
function  in  the  cross-functional  world  of  management. 

It  is  important  for  managers  to  be  able  to  understand  the  basic  concepts  and  language  used  in  the  other 
areas  of  their  company.  Why  is  it  important  to  work  within  the  information  technology  constraints  dictated  by  the 
network  manager?  Why  is  it  important  to  use  the  “company  colors”  on  all  new  products?  How  do  the  operations  of  a 
certain  business  unit  affect  the  corporate  financial  statements?  Why  is  this  important  to  the  health  of  the  company? 
These  are  questions  that  executive  education  courses  are  designed  to  help  new  managers  and  executives  answer  for 
themselves.  They  don’t  need  a thorough  conceptual  understanding  of  accounting  systems  or  network  security 
measures,  such  as  they  might  receive  in  a certification  or  academic  MBA  course,  and  a one  to  two  hour  course  or 
suite  of  courses  is  often  sufficient. 

Executive  education  courses  have  often  been  delivered  in  traditional  training  environments,  such  as 
corporate  training  classrooms  or  off -site  seminar  rooms.  The  online  courses  we  create  are  experienced  in  an 
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anytime,  anywhere  format  from  the  manager’s  networked  computer,  at  work  or  home,  whenever  they  have  the  time 
in  their  busy  schedule.  We  have  found  most  participants  are  able  to  devote  an  hour  or  two  at  a time  to  this  kind  of 
training.  We  attempt  to  keep  each  suite  of  courses  short  enough  so  that  a new  manager  could  finish  the  entire  suite  in 
one  business  day.  In  many  situations,  one  dedicated  “learning”  day  is  more  effective  than  trying  to  stretch  the  course 
out  and  keep  participants  motivated  over  several  weeks.  Occasionally,  a business  will  have  experience  and  resources 
to  implement  a synchronous  online  seminar  setting,  a “virtual  classroom.”  If  this  is  the  case,  we  design  courses  that 
exploit  this  capability.  It  should  be  noted,  however,  that  the  instructional  experience  (information,  learning, 
knowledge,  etc.)  in  each  course  is  designed  to  be  completely  accessible  even  if  the  synchronous  component  fails  or 
is  inaccessible  to  some  participants.  In  fact,  this  is  one  of  the  design  challenges  we  discuss  later  in  this  paper. 

Instructional  Approaches 

We  have  developed  and  implemented  two  major  models  of  instruction  for  executive  education  courses. 
While  we  have  used  other  instructional  approaches  for  specific  training  needs,  we  believe  our  two  primary 
approaches  fit  most  of  the  learning  content  and  situations  that  we  encounter.  In  each  course  model,  we  have 
implemented  the  learning  values  of  experiential  learning  (learning  by  doing)  (Dewey,  1897;  others?).  We  believe 
that  active  learners  are  much  more  likely  to  learn  than  passive  ones,  so  each  course  uses  many  interactive  elements 
(cite  interactivity  in  learning  research). 

Cognitive  apprenticeship 

The  first  model  we  used  for  executive  education  is  based  on  the  cognitive  apprenticeship  theory  of  learning. 
In  this  theory,  founded  primarily  upon  Vygotsky’s  notions  of  sociocultural  learning  and  the  zone  of  proximal 
development  (ZPD),  learning  is  accomplished  by  participants  as  they  are  guided  through  the  study  and  experience  of 
new  information  and  skills  by  cognitive  experts  (Rogoff,  1990;  Vygotsky,  1978).  The  expert  begins  with  a large 
amount  of  control  over  the  learning  process  and  content,  and  gradually  turns  over  more  and  more  control  of  the 
learning  to  the  learner.  By  the  end  of  the  experience,  the  learner  is  ready  to  perform  the  new  skills  or  apply  the  new 
information  on  their  own,  in  their  own  environment,  as  they  begin  to  develop  their  own  expertise  through  authentic 
practice. 

In  the  training  environments  we  create,  the  online  course  is  the  expert  that  guides  the  student  through  the 
learning  experience.  We  structure  the  overall  course  experience  and  embed  various  course  elements  that  support  and 
guide  the  students’  learning  throughout.  Beginning  with  a motivating  and  engaging  lead-in  to  the  content,  we  use 
predictable  course  segments  that  move  from  high  “scaffolding”  and  gradually  reduce  the  amount  of  scaffolding  as 
the  course  progresses  (Bruner,  1983;  Rogoff,  1990;  Stone,  1993;  Vygotsky,  1978).  In  the  end,  the  student  is  given 
tools  and  references  to  use  on  their  own  in  their  own  work  environment.  Next,  we’ll  briefly  explain  the  course 
structure  we’ve  created.  Please  refer  to  figure  1 to  see  an  example  of  this  course  structure  from  an  actual  course. 

Each  course  is  bundled  with  related  courses  in  a course  suite.  Usually,  there  are  four  to  five  courses  in  one 
suite.  Suites  can  be  completed  within  one  business  day,  in  approximately  eight  to  ten  hours.  In  figure  1,  the  courses 
in  the  “Learn  to  Speak  Accounting”  suite  are  shown  on  the  left  side  of  the  navigation  frame  (on  the  left).  While  there 
is  no  technical  barrier  to  taking  any  course  in  any  order,  there  is  an  implied  order  in  the  navigation  system.  In 
practice,  it  usually  makes  good  “sense”  to  take  the  courses  in  their  listed  order,  but  we  allow  the  learner  the  freedom 
to  choose.  Each  course  is  further  divided  into  common  sections.  Once  again,  the  technical  system  does  not  require 
the  student  to  follow  the  implied  (physical)  order  as  they  work  through  the  course,  but  the  course  content  and 
connecting  language  used  strongly  suggest  the  implied  order.  These  sections  are  listed  in  order  on  the  right  side  of 
the  navigation  frame  in  figure  1 . 

Why  Learn  It 

The  first  section  of  each  course,  the  Why  Learn  It  section,  is  designed  to  motivate  the  student’s  further 
study  of  the  course  content.  Using  a combination  of  text,  images,  vide,  and  multimedia,  the  overall  importance  of  the 
content  is  explained.  Often,  we  use  a short  case  experience  to  engage  the  student,  asking  them  to  read  or  view  a 
situation  and  make  a few  basic  decisions.  This  experience  is  intended  to  show  them  that  they  may  do  not  have  all  the 
answers  or  expertise  (e.g.  how  to  motivate  different  employees)  they  need,  and  that  this  course  can  help  them 
develop  their  own  expertise. 

How  It’s  Done 
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The  second  section  of  each  course  is  the  “How  It’s  Done”  section.  He  purpose  of  this  section  is  primarily 
expository  - explaining  how  a concept  works  in  practice,  how  processes  are  carried  out,  or  perhaps  presenting  a 
body  of  information  that  will  be  used  in  some  subsequent  practice  or  activity.  The  How  It’s  Done  section  uses  video 
segments,  animations,  text,  and  graphics  to  present  information.  A constant  instructional  practice  is  to  include  many 
relevant  examples  and  cases  of  real  companies  and  situations  to  maintain  high  student  interest  and  increase  the 
practicality  of  the  instruction.  Throughout  this  section,  the  course  designer  is  still  in  control  of  the  learning  process 
for  the  most  part.  While  no  particular  order  is  technically  mandated,  students  generally  read  the  text  from  start  to 
finish,  interacting  with  the  course  to  watch  videos  and  animations  and  read  extra  material  in  pop-up  windows  as  they 
progress. 

You  Do  It 

The  next  section,  “You  Do  It,”  provides  an  opportunity  for  the  student  to  apply  the  information  they  have 
read,  watched,  and  begun  to  learn  in  the  How  It’s  Done  section.  The  You  Do  It  section  is  focused  on  guided  student 
activity.  The  course  sets  up  a “practice  field”  (cite  Duffy)  in  which  the  student  applies  information  in  a constructed 
learning  space.  Activities  take  the  form  of  website  visits,  interactive  animations,  text  -based  problem  scenarios,  and 
others.  The  student  is  given  more  control  over  their  own  learning  now,  with  opportunities  to  repeat  activities, 
sometimes  with  different  conditions.  Some  courses  present  several  different  activities  that  the  student  can  choose 
from.  We  have  seen  students  use  “optional”  activities  in  many  different  ways.  Some  students  try  every  activity  once 
and  move  on.  Other  students  try  each  activity  over  and  over  until  they  master  each.  Still  other  students  only  work 
through  one  activity  and  then  continue. 

Explain  It 

The  “Explain  It”  section  presents  the  student  with  a chance  to  explain  what  they  have  learned  by  answering 
several  free  response  questions.  Students  submit  their  answers  and  receive  direct  feedback  after  just  a short  wait  (20- 
30  minutes).  Questions  usually  require  the  student  to  use  simple  analysis,  synthesis,  or  application  skills  to  create 
fully  correct  answers.  These  questions  provide  an  opportunity  for  the  student  to  check  their  understanding  and 
demonstrate  that  they  are  learning  course  content,  and  not  just  clicking  through  the  course  without  much  thought. 

Now  Think  Again 

The  last  section  of  each  course,  “Now  Think  Again,”  challenges  the  student  to  think  about  the  course 
content  in  a new  way  - often  in  the  context  of  his  or  her  own  work  environment  or  from  an  alternative  perspective 
(as  an  employees  versus  as  a manager,  for  example).  The  course  provides  printable  documents  the  student  can 
“takeaway”  from  the  course  to  help  them  implement  new  knowledge  and  skills  in  their  work.  Suggestions  for  further 
reading  and  helpful  website  references  complete  the  course  content,  providing  resources  for  the  student  to  use  after 
the  course  experience  is  finished.  Many  students  report  that  this  is  one  of  the  most  useful  sections  of  the  course, 
especially  links  to  relevant  websites.  In  this  section,  the  student  is  in  almost  full  control  of  his  or  her  own  learning. 
The  course  content  takes  the  form  of  a library  of  resources  for  the  student  to  use  as  he  or  she  applies  new  knowledge 
on  their  own  in  their  own  unique  situation. 
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Figure  1.  Cognitive  apprenticeship  course  model:  Learn  to  Speak  Accounting, 
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Start  counting . But  since  this  is  not  the  case,  you  may 
find  the  following  exercise  in  determining  your  level  of 
economic  wealth  interesting, 


The  previous  exercise  focused  primarily  on  individual 
wealth  and  the  elements  of  resources  and 
obligations  that  can  be  used  to  compute  the  wealth 
of  an  individual.  AC  the  end  of  that  exercise,  you  were 
exposed  to  tiie  resources  and  obligations  of  a 
company . You  are  probably  familiar  with  the  concept 
of  a company.  Fiat,  Club  Med,  and  Microsoft  are  all 
examples  of  companies.  Their  business  operations 
differ  greatly;  so  what  characteristic  makes  them 
companies  to  begin  with?  Accounting  views 
companies  as  wealth-creating  entities. 
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From  the  perspective  of  financial  accounting, 
companies  create  wealth  by  engaging  in  financing 
activities  (a  transfer  of  wealth  into  the  business), 
investing  activities  (converting  capital  to  plant  and 
equipment),  and  operating  activities  (producing 
products  and  services  that  can  be  sold  to  create 
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Additional  Course  Features 


This  course  model  takes  advantage  of  other  features  that  contribute  directly  to  its  effectiveness.  We  use 
short  video  segments  of  experts,  usually  notable  business  professors,  that  highlight  key  points  in  the  content  or 
answer  frequently  asked  questions.  Links  to  video  segments  are  located  in  the  right  margin  of  the  course  content 
page  (see  figure  1.).  Transcripts  of  each  video  segment  are  included  — just  in  case  there  are  technical  limitations  that 
preclude  a student  from  viewing  one  of  the  videos.  We  use  Flash  animations  to  motivate  student  interest,  explain 
complex  concepts,  and  provide  interactive  exercises  (see  figure  2.).  We  provide  transcripts  for  animations  that 
explain  concepts,  since  they  often  include  core  course  content,  and  students  occasionally  experience  technical 
difficulties  when  viewing  animations.  Finally,  we  use  many  “pop-up”  windows  to  display  additional  information  to 
the  student.  These  pop-ups  are  indicated  by  standard  hyperlink  formatting,  underlined  blue  text  (see  figure  1.).  The 
types  of  information  we  present  in  pop-ups  include;  answers  to  course  questions,  additional  reading  material 
(articles,  case  studies,  etc.)  and  glossary  definitions.  Pop-ups  increase  the  level  of  learner-content  interactivity, 
improving  the  overall  student  experience  (Gilbert  & Moore,  1998;  Hannifin,  Hill,  & Land,  1997;  Trentin,  2000). 
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Figure  2.  Interactive  Animation 
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Sample  Course:  Learn  to  Speak  Accounting 

One  of  the  courses  created  using  this  instructional  model  is  titled  “Learn  to  Speak  Accounting.”  This  course 
is  designed  to  help  non-accounting  managers  understand  basic  accounting  principles  so  they  can  communicate  with 
corporate  accounting  staff  and  manage  their  own  business  functions  with  more  fiscal  soundness.  The  target  audience 
for  this  course  is  non-accounting  managers:  engineers,  manufacturing,  marketing,  etc.  managers  who  do  not  have  an 
accounting  background  yet  have  (new)  fiscal  responsibilities  that  include  making  decisions  that  affect  or  that  are 
affected  by  accounting  systems.  Sometimes,  a new  manager  who  does  not  have  a financial  background  may  tend  to 
make  business  decisions  for  their  unit  or  group  without  taking  into  account  various  accounting  factors  that  could 
help  them  understand  the  flow  of  wealth  within  their  company.  A core  assumption  (and  assertion)  of  the  course  is 
that  understanding  these  factors  can  help  managers  make  better  business  decisions. 

A New  Approach  to  Teaching  Accounting 

This  course  presents  the  concepts  of  accounting  in  a new  way.  Instead  of  beginning  the  study  of  accounting 
from  the  bottom  up,  by  discussing  debits,  credits,  and  accounts,  this  course  takes  a “top  down”  approach,  beginning 
with  the  grand  issues  of  accounting,  such  as  how  to  read  annual  reports.  Specifically,  the  course  begins  by  showing 
how  accounting  tracks  the  resources,  obligations,  and  wealth  of  a company.  These  are  presented  in  the  company’s 
annual  report,  an  important  document  for  business  professionals.  As  the  student  examines  the  different  parts  of  the 
annual  report,  the  key  concepts  of  accounting,  such  as  different  accounts  and  how  transactions  affect  those  accounts, 
are  revealed.  This  top  down  approach  is  effective  as  it  gets  the  student  involved  in  a (more)  familiar  practice  such  as 
reading  annual  reports  and  then  presents  the  more  specific  (and  unfamiliar)  concepts  of  accounting  that  relate  to  the 
report. 

Key  Course  Features 

Throughout  the  suite  of  LTSA  courses,  there  are  many  opportunities  for  the  student  to  practice  applying  the 
information  and  concepts  introduced  in  the  course.  Many  of  these  practice  opportunities  take  the  form  of  interactive 
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(Flash)  animations  or  visiting  live  websites  and  analyzing  real  accounting  information  from  existing  companies. 
Additionally,  each  How  It’s  Done  section  in  this  suite  utilizes  an  interactive  (Flash)  animation  that  requires  the 
student  to  apply  the  course  concepts  in  a realistic  business  situation.  See  figure  2.  for  an  example  of  one  of  these 
interactive  tools.  This  practice  is  focused  on  developing  basic  skills  and  understanding  of  the  concepts  just 
presented.  Finally,  in  each  You  Do  It  section  in  this  suite  we  challenge  the  student  by  asking  him  or  her  to  apply  the 
new  information  he  or  she  has  just  read  and  explored  through  a visit  to  a real  company’s  website.  A sample  of  the 
tasks  he  or  she  may  be  asked  to  complete  includes  looking  for  general  corporate  financial  performance  in  a 
company’s  annual  report,  exploring  the  detailed  financial  statements  found  in  real  annual  reports,  and  comparing 
methods  of  accounting  as  evidenced  in  a company’s  annual  report  with  common  or  accepted  methods  discussed  in 
the  course. 

Problem  Centered  Learning 

A second  course  model  we  have  used  uses  a problem  centered  approach  to  design  and  instruction.  Starting 
off  by  considering  the  problem  the  student  is  facing,  or  in  some  instances  the  “guiding  question”  the  student  is  trying 
to  answer,  is  helpful  both  for  the  student  as  they  enter  the  course,  but  also  for  the  design  team  as  they  create  the 
instruction.  Since  most  learning  takes  place  as  the  result  of  a gap  between  what  a learner  knows  and  what  a learner 
wants  to  know,  starting  with  what  exactly  the  learner  wants  to  know  (problem  to  be  solved,  major  question  to  be 
answered)  makes  sense  (Savery  & Duffy,  1996;  Merrill,  2000).  After  starting  with  the  problem,  the  instruction  must 
present  the  student  with  the  resources  and  guidance  needed  to  solve  the  problem  or  answer  the  guiding  question. 
Providing  multiple  opportunities  for  practice  and  supporting  the  student  as  they  leave  the  course  and  apply  their  new 
knowledge  in  their  normal  jobs  is  the  final  phase  of  this  instruction.  In  essence,  this  course  design  includes  four 
phases;  problem  (Why?  What  context?),  information  (What?  How?),  practice  (I  try  with  you),  and  transfer  (I  go  it 
alone).  This  overall  approach  to  instruction  includes  some  elements  of  cognitive  apprenticeship  as  well,  since  the 
student  receives  fading  support  or  scaffolding  as  they  progress  through  the  course. 

Sample  Course:  Post-purchase  Marketing 

This  one  to  two  hour  course  was  designed  to  teach  the  concept  of  marketing  after  the  sale  - post-purchase 
marketing,  to  various  levels  of  employees  in  a business.  Everyone  who  comes  into  contact  with  customers,  or  whose 
work  directly  impacts  a customer’s  experience  after  they  have  purchased  a product  (or  service)  from  the  company, 
should  understand  these  concepts.  On  a large  scale,  this  course  is  designed  to  help  an  entire  organization  understand 
the  importance  of  after  the  sale  marketing  efforts.  See  figure  3 for  the  problem-setting  page  of  the  course. 

The  course  sections  are  accessed  using  the  navigation  panel  on  the  left.  In  this  course,  there  is  an 
Introduction  section,  a problem  setting  section,  four  sections  devoted  to  new  content  information,  a problem  solution 
(practice)  section,  and  a final  section  devoted  to  supporting  the  student’s  transfer  of  their  new  knowledge  from  their 
course  experience  to  their  own  workplace. 
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Figure  3.  Problem-centered  Learning  Course:  Post-purchase  Marketing 
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Improving  Customer  Satisfaction 
Fostering  Customer  Loyalty 
MarHeting  to  the  Individual 
Creating  a Plan  For  DYI 
CreaUng  Your  Ov/n  Plan 


In  this  course,  you  will  take  the  role  of  an  employee  at  DYI 
Software  Solutions,  a maker  of  home  office  software.  After 
gathering  information  about  DYI's  marketing  problem  and  learning 
more  about  postpurchase  tactics,  you  will  have  a chance  to  create 
a solution  for  DYI. 


SoNwofe  Solutions 


Here's  the  situation:  The  company's  marquee 
product,  DYI  OfficeChores,  sold  extremely  well 
initially,  but  sales  of  its  first  upgrade,  released 
one  year  ago,  have  been  poor.  As  DYI  gears  up 
to  release  OfficeChores  3.0  in  the  coming 
months,  the  company  wants  to  improve  its 
sales  outlook.  You  have  been  called  to  a 
meeting  with  various  department  heads  to 
discuss  what  is  wrong  with  the  marketing 
strategy.  Watch  the  video  to  attend  the  meeting. 


Video  »»  View  video  | Tr^nsoriut  | Preliminary  Surv-’ev  Findlnae 

After  attending  the  meeting,  your  assignment  is  clear:  Help  develop 
a marketing  strategy  that  addresses  three  major  areas  of 
postpurchase  marketing  that  were  identified  by  the  market  research 
company:  customer  service,  loyalty,  and  customization.  Funding  is 
always  a concern,  so  prioritize  the  parts  of  your  strategy  with  cost 
in  mind. 

The  next  sections  of  the  course  will  help  you  explore  postpurchase 

tartirc  wnii  will  npprl  tn  i inHprctanrl  anH  inrnrnnratp  in  vniir 


A New  Approach  to  Teaching  Marketing 

The  Post-purchase  Marketing  course  takes  a different  approach  to  teaching  marketing.  Instead  of  starting 
the  course  with  the  concepts  of  post-purchase  marketing,  the  student  is  presented  with  a business  problem  that  needs 
resolution.  A fictional  company,  DYI  Software,  is  experiencing  reduced  sales  and  low  customer  satisfaction.  The 
problem  appears  to  be  a failure  in  post-purchase  marketing  efforts.  The  student  is  given  the  task  of  learning  about 
post-purchase  marketing  and  then  coming  up  with  a new  marketing  strategy  for  the  company.  As  the  student  mo  ves 
through  the  rest  of  the  course,  they  are  presented  with  the  key  concepts  of  post-purchase  marketing  to  help  them  in 
formulating  their  own  strategy.  Examples  and  interactive  practice  elements  give  the  student  the  opportunity  to  apply 
principles  learned  before  embarking  on  the  strategy  project  for  DYI.  Throughout  the  course,  the  student  has  the 
opportunity  to  interact  with  other  students,  posting  questions  and  ideas  in  the  course  discussion  forum. 

Key  Course  Features 

This  course  uses  many  of  the  same  instructional  methods,  tools,  and  techniques  as  the  cognitive 
apprenticeship  courses  do.  One  additional  feature  we  added  to  this  course  we  call  “embedded  self  assessments” 
(Beatty,  Branon,  & Wilson  2001).  This  feature  was  added  to  assist  the  student’s  self-regulatory  behavior  (Schunk  & 
Zimmerman,  1998)  during  their  learning  experience  (see  figure  4).  Embedded  self-assessments  (ESA)  are  short 
opportunities  for  students  to  assess  their  understanding  while  they  are  learning  new  course  content.  They  are 
“embedded”  in  that  they  are  included  within  blocks  of  content,  not  grouped  at  the  end  of  a content  section  as  is 
traditionally  done  in  instructional  texts.  We  believe  that  ESAs  help  students  understand  whether  they  are  truly 
learning  the  course  material  before  they  get  to  the  comprehensive  assessment  at  the  end  of  the  course,  before  they 
attempt  to  apply  course  content  in  a practice  situation.  Additionally,  students  have  reported  that  ESAs  help  them  stay 
focused  on  the  learning  task  at  hand. 
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Figure  4 shows  an  exa  mple  of  the  type  of  ESA  used  in  the  post-purchase  marketing  course.  In  general,  we 
ask  the  student  to  answer  a free-response  question,  and  then  provide  an  opportunity  for  them  to  check  their  answer 
with  an  “expert”  response.  Another  type  of  ESA  we  have  used  is  multiple  choice  with  immediate  feedback.  We  have 
found  that  different  students  have  different  ESA  style  preferences  (Beatty  et  al.,  2001),  and  different  course  content 
fits  different  ESA  styles  as  well.. 

Figure  4.  Embedded  Self-assessment 


Self-assessment 

I ^ 1 . 

Answer  the  question  below  in  the  space  provided.  When  finished,  click  "Submit."  Then  compare 
your  answer  to  the  expert  answer  provided. 

Since  DYI  is  a technology  company,  what  are  some  ways  it  can  employ  technology  to  help 
improve  customer  service  and  satisfaction?  List  at  least  three  ways  technology  could  be  used, 
and  briefly  explain  how  each  way  might  help  improve  customer  satisfaction. 

Your  answer: 

One  uay  technology  could  be  used 
is  for  it  to  be  the  center  of  the  ~ 
customer's  experience.  Another  uay 
it  could  be  used  is  to  form  the 
communication  backbone  between  DYI 
and  its  consumers.  Finally, 
technology  could  facilitate  the  ^ 

exchange  of  new  product  ideas  Q 


Submitg» 

Compare  your  answer. 

I 1 

Technology  could  be  employed  in  a variety  of  ways  to  improve  customer  service  and 
satisfaction.  Customer  service  employees  can  definitely  use  technology  to  access  detailed 
product  and  service  information  specific  to  the  customer.  This  can  help  customers  feel  like  the 
company  understands  them  as  individuals  and  can  help  solve  problems  more  quickly  with 


ID  Challenges 

One  course  model  fits  all ... 

One  of  the  challenges  to  building  executive  education  is  that  training  departments  often  want  a “on-size- 
fits-all”  approach  to  courses.  Having  a single  course  model  is  easier  to  manage  and  track  but  the  reduction  in 
flexibility  can  detract  from  learning  and  motivation.  For  instructional  designers,  this  can  mean  having  to  work  with 
a less  than  ideal  pedagogical  structure  for  some  topic  areas.  In  the  future.  Learner  Management  Systems  (LMSs) 
should  become  more  complex  in  how  they  track  learners,  thus  making  it  possible  that  this  problem  can  be  reduced. 

Working  with  Subject  Matter  Experts  (SMEs) 

Finding  SMEs  can  be  a major  hurdle  for  high-level  online  executive  education.  Ideally,  a SME  should  be 
someone  who  is  an  exe  cutive  and  has  experience  in  the  narrow  topic  being  covered.  The  obvious  issue  is  that 
people  with  such  credentials  are  extremely  busy  running  companies  or  consulting  and  the  incentive  to  have  them 
develop  an  online  course  has  to  be  high.  Even  when  a person  has  been  located  with  the  correct  skills  and 
knowledge,  they  probably  do  not  understand  the  instructional  design  process. 

Working  with  SMEs  has  always  been  a challenge  for  instructional  designers.  When  developing  short, 
specialized  e-leaming,  however,  the  difficulty  can  become  acute.  Because  SMEs  are  experts,  it  is  often  difficult  for 
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them  to  write  high-level  pieces  that  need  to  be  delivered  in  a short  period  of  time.  Additionally,  the  expert  status  of 
a SME  means  that  they  are  not  used  to  working  in  a team-based  development  environment.  The  process  of 
negotiating  can  require  astute  political  skills  on  the  part  of  the  instructional  designer,  especially  when  determining 
what  content  is  ‘critical’  for  an  executive  level  course. 

Can  accounting  really  be  made  motivating? 

Unless  a person  works  as  an  accountant,  a course  in  accounting  is  probably  not  one  that  would  be  taken  by 
choice.  Accounting  skills,  however,  are  important  for  managers  at  nearly  every  level.  They  might  not  need  to  be 
able  to  handle  accounting  functions  directly,  but  having  a working  knowledge  of  accounting  fundamentals  can 
improve  their  ability  to  run  their  department.  By  understanding  this  perspective,  an  instructional  designer  can  build 
a course  that  is  interesting  and  motivational  for  a variety  of  executive  learners. 

Learner  persistence 

Even  if  a course  is  short  and  filled  with  motivating  animations,  videos,  and  content,  business  managers  are 
always  very  busy.  The  temptation  can  be  to  move  through  the  training  module  as  quickly  as  possible.  Simply 
paging  through  screens  does  not  mean  that  learning  will  occur  yet,  emphasizing  this  to  someone  with  limited  time 
can  be  difficult  in  an  online  environment.  Keeping  people  on  task  and  focused  will  in  an  online  environment  will 
likely  continue  to  be  a vexing  problem  for  designers. 

Ongoing  Research 

We  are  engaging  in  ongoing  research  projects  at  Option  Six  to  help  us  understand  online  learning  better  and 
focus  on  creating  more  effective  learning  environments  for  each  of  the  instructional  situations  that  we  encounter. 

The  corporate  environment  we  find  ourselves  in  is  very  supportive  of  focused  research  that  contributes  to  more 
effective  learning,  better  overall  training  solutions,  and  ultimately  more  satisfied  customers.  The  projects  we  are 
currently  pursuing  include: 

The  use  of  self-assessments  in  Executive  education  courses  - looking  at  student  perceptions  of  learning, 
and  actual  conceptual  learning  in  courses  that  use  embedded  self-assessments. 

Self-regulation  in  self-paced  e-leaming  - how  can  we  better  support  learners  through  the  instructional 
design  and  technical  features  of  an  online  course? 

The  use  of  blended  learning  solutions  - when  should  students  interact  asynchronously?  When  should  the 
course  include  synchronous  meetings?  Should  part  of  the  training  be  implemented  face-to-face? 

Social  interaction  in  online  learning- when  should  social  interaction  be  used  in  an  online  course?  What 
conditions  should  be  considered  when  choosing  specific  instructional  methods? 

Conclusion 

In  closing,  we  have  found  that  several  course  instructional  models  have  been  effective  in  creating 
Executive  Education  courses  that  are  based  on  solid  instructional  theory,  that  students  like,  and  that  actually  teach 
students  the  course  content.  Creating  effective,  engaging  courses  is  challenging  and  difficult.  Following  solid 
instructional  design  blueprints  while  still  allowing  for  appropriate  divergence  and  variability  based  on  content, 
student,  and  client  needs  has  been  a successful  strategy  for  us.  We  are  continuing  to  study  the  course  models  and 
essential  course  features  we  have  created  and  used  in  our  attempts  to  understand  and  implement  the  most  effective 
instructional  approaches  for  business  audiences.  Expanding  our  instructional  design  research  to  include  the 
emerging  issues  and  concerns  of  business-focused  e-leaming  promises  to  be  an  enjoyable  and  rewarding  venture,  in 
both  a theoretical  and  practical  sense.  The  opportunities  are  endless. 
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Abstract 

The  purpose  of  this  paper  is  to  analyze  the  implementation  process  of  Taiwan  's  constructivist-approach 
elementary  mathematics  curriculum  in  terms  of  Reigeluth  *s  guidelines  for  the  system  design  process. 

Introduction 

The  implementation  of  Taiwan’s  nationwide  constructivist-approach  elementary  mathematics  curriculum 
has  been  ongoing  for  five  years.  Not  surprisingly,  as  time  passes,  problems  related  to  this  implementation  process 
become  increasingly  evident.  This  situation  is  complex  because  it  not  only  reveals  those  difficult  issues  directly 
related  to  the  epistemology  and  instructional  methods  of  mathematics,  but  also,  more  seriously,  those  ones 
associated  with  the  troubles  of  Taiwan’s  entire  educational  system.  From  the  perspective  of  systems  thinking, 

Taiwan  elementary  educational  system’s  mindset  as  a whole  is  troubled,  and  is  unable  to  coordinate  with  the 
surrounding  community  in  order  to  better  educational  standards. 

Taiwan’s  case  is  worth  examination  as  it  serves  as  an  exemplary  case  of  difficulties  associated  with  the 
redesign  of  an  educational  system.  The  goal  of  this  paper  is  to  analyze  the  problems  linked  to  the  reform  and 
implementation  of  Taiwan’s  elementary  constructivist  mathematics  curriculum.  This  paper  will  begin  with  a brief 
introduction  on  the  new  trend  in  Taiwan’s  elementary  constructivist  mathematics  curriculum.  I will  then  discuss  the 
difficulties  associated  with  implementation,  and  the  problems  caused  by  those  persons  in  the  education  community 
who  have  a distorted  perspective  on  the  reform.  Finally,  I will  examine  these  problems  utilizing  Reigeluth’s 
guidelines  for  the  system  design  process  for  more  comprehensive  and  systemic  understanding  of  the  situations. 

An  overview  of  the  new  trend  in  Taiwan’s  elementary  mathematics  curriculum 

Moving  away  from  the  objectivist  tradition  of  didactic  and  authoritative  mathematics  education,  the 
Republic  of  China’s  Ministry  of  Education  mandated  a constructivist  approach  to  the  province’s  elementary 
mathematics  curriculum  agenda  (Ministry  of  Education,  R.O.C.,  1993).  Taiwan’s  elementary  mathematics 
curriculum  adopts  two  major  views  of  constructivist  approach:  the  individual  cognitive  mainly  driving  from  Piaget 
(1977)  and  his  followers,  and  social-cultural  mainly  driving  from  Vygotsky  (1978)  and  his  followers  (Taiwan 
Province  Public  School  In-Service  Teacher  Training  Service,  1995). 

Individual  cognitive  view  “...emphasizes  the  constructive  activity  of  the  individual  as  he  or  she  tries  to 
make  sense  of  the  world.  Learning  is  seen  to  occur  when  the  learner’s  expectations  are  not  met,  and  he  or 
she  must  resolve  the  discrepancy  between  what  was  expected  and  what  was  actually  encountered.  Thus,  the 
learning  is  in  the  individual’s  constructions  as  he  or  she  attempts  to  resolve  the  conflict,  or  alternatively  put, 
individuals  literally  construct  themselves  and  their  world  by  accommodating  to  experiences... From  this 
perspective,  the  importance  of  the  teacher  and  other  students  is  as  a source  of  perturbation  or  puzzlement  as 
a stimulus  for  the  individual’s  learning... Hence,  within  this  framework,  the  focus  is  on  the  individual 
within  the  group,  and  cognition  occurs  in  the  head  of  the  individual.”  (Duffy  & Cunningham,  1996,  p.l75) 
Social-cultural  view  “...emphasizes  the  socially  and  culturally  situated  context  of  cognition... This 
approach  examines  the  social  origins  of  cognition,  for  example,  the  impact  of  an  individual’s  appropriation 
of  language  as  a mediating  tool  to  construct  meaning.  Collective  actions  become  the  focus,  as  in  RogofTs 
(1994,  p.  209)  learning  communities,  where  ‘learning  occurs  as  people  participate  in  shared  endeavors  with 
others,  with  all  playing  active  but  often  asymmetrical  roles  in  social-cultural  activity.’  It  is  the  changes  of 
ways  in  which  one  participates  in  a community  which  are  crucial,  not  individual  constructions  of  that 
activity... Learning,  then,  is  a process  of  acculturation,  and  thus  the  study  of  social  and  cultural  processes 
and  artifacts  is  central...”  (Duffy  & Cunningham,  1996,  pp.  175- 176) 

Publishers  designed  their  elementary  mathematics  instructional  methods  based  on  these  two  approaches. 
These  instructional  methods  were  implemented  nationwide  in  1996,  beginning  with  first-grade  students. 
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Examining  the  problems  of  implementation 

Since  1996,  obstacles  faced  by  proponents  of  reform  have  become  increasingly  great.  These  problems  have 
already  hindered  the  reform’s  effectiveness,  and  have  even  distorted  the  Ministry  of  Education’s  original  intentions 
(Chen,  2001;  Chen,  2001;  Huang,  2001).  As  a matter  of  fact,  the  original  intentions  of  this  reform  were  to  provide 
learners  with  opportunities  to  truly  understand  mathematics,  to  discuss  mathematics  and  to  collaborate  in  efforts  to 
solve  problems,  just  as  the  community  of  mathematicians  did  (Huang,  1996).  Unfortunately,  such  intentions  did  not 
come  into  fruition,  as  many  teachers  rejected  the  new  instructional  methods  (Chen,  2001;  Chen,  2001;  Huang, 

2001).  Opposition  was  so  great  that  there  were  even  teachers  who  planned  an  early  retirement  in  order  to  avoid 
change.  A significant  number  of  people  in  the  education  community  (By  this  term,  I mean  officers,  professors  of 
education,  researchers,  and  teachers  not  directly  involved  in  this  reform)  were  unaware  of  this  nationwide  mandatory 
reform.  Many  parents  were  against  it,  and  most  community  members,  like  those  in  the  education  community,  did  not 
know  of  it. 

The  obstacles  above  are  by  no  means  caused  only  by  issues  of  the  constructivist  instructional  method  itself. 
They  are  closely  linked  to  the  issues  of  systemic  thinking  and  the  system  design  process.  Due  to  the  lack  of  insight 
in  viewing  this  reform  as  one  related  to  a systemic  transformation  (a  systemic  design  and  implementation  process), 
this  reform  has  encountered  enormous  obstacles.  The  current  reform  of  Taiwan’s  elementary  curriculum,  like  many 
other  past  educational  reforms,  has  fallen  into  the  rut  of  using  a piecemeal  approach  in  a desperate  attempt  to 
improve  fragmented  and  outdated  educational  reforms.  If  educators  truly  wish  to  succeed  in  creating  a better 
educational  program,  they  must  abandon  such  an  antiquated  approach  and  instead,  focus  their  attention  on  the 
systems  design  approach. 

One  must  understand  that  the  reform  of  one  or  two  educational  paradigms  often  affects  other  aspects  of  the 
educational  system.  The  reform  ofTaiwan’s  elementary  mathematics  education  is  no  exception.  As  Reigeluth  states: 

For  a fundamental  change  in  education  to  be  lasting  and  effective,  it  must  be  a systemic  transformation-one 
in  which  changes  pervade  all  aspects  of  the  educational  system  to  account  for  inter-dependencies  among 
parts  of  the  system.  If  the  change  is  truly  pervasive,  it  will  have  an  impact,  not  only  on  learning  in  the 
classroom,  but  throughout  the  school  and  administration,  as  in  the  community  where  it  occurs.  (Reigeluth, 
1990,  p.) 

The  systems  design  approach  directs  educational  systems  design  toward  a sweeping,  comprehensive 
transformation.  Unless  we  use  a systemic  approach  to  review  the  problems  and  solutions,  we  will  not  be  able  to  give 
appropriate  suggestions  for  solutions. 

In  the  following  section  I will  use  Reigeluth,  Norris,  and  Ryan’s  “Major  Guidelines  for  the  Redesign 
Process”  to  analyze  the  design  and  implementation  ofTaiwan’s  elementary  constructivist  mathematics  curriculum.  I 
believe  this  approach  will  guide  our  mindset  to  view  Taiwan’s  case  in  a radical  yet  systemic  way,  and  help  us  to 
build  a more  comprehensive  blueprint  for  restructuring  the  educational  system  so  that  it  may  support  a constructivist 
mathematics  instruction. 

Analysis 


Table  1:  MAJOR  GUIDELINES  FOR  THE  REDESIGN  PROCESS 


(Adopted  from  Reigeluth,  1991,  p.  9) 

Guidelines 

Initiating  the  process: 

1.  Appraise  the  climate  for  systemic  change. 

2.  Make  sure  you  have  powerful  instigation. 

3.  Build  a school  and  community  coalition  of  restructurers. 

4.  Build  public  support  for  the  change. 

5.  Document  the  experience  from  the  beginning. 

Design  the  new  system: 

6.  Involve  all  stakeholder  groups  in  deciding  on  and  carrying  out  the  design  process. 

7.  Get  good  leadership  and  outside  facilitation. 

8.  Develop  a common  mindset  for  the  new  paradigm. 

9.  Design  a feasible  model  for  the  system. 
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10.  Establish  and  maintain  communication  links. 

Planning  the  implementation: 

11.  Experiment  with  tryouts  and  pilots,  and  make  revisions. 

12.  Designate  time  and  funds  to  develop  and  participate  in  staff  development. 

13.  Designate  time  and  funds  to  procure  learning  resources,  and  equipment  and  to  remodel  facilitation. 

14.  Obtain  active  resource  support  from  all  stakeholders. 

Implementing  the  design: 

15.  Allow  parents  to  choose  to  send  their  children  to  the  new  system. 

16.  Institutionalize  the  change  process  with  regular  formative  evaluations  by  the  stakeholders. 

17.  Maintain  communication  with  stakeholders,  non-participants  and  outsiders  interested  in  systemic  restructuring. 

1 8.  Allow  three  years  before  doing  a summative  evaluation. 


The  above  chart  outlines  the  eighteen  guidelines  proposed  by  Reigeluth,  Norris,  & Ryan  (1991)  to  direct 
the  four  phases  of  the  educational  redesign  process.  In  the  following  section,  I will  elaborate  on  the  condition  for 
each  guideline. 

Initiating  the  Process 

Guideline  1:  Appraise  the  climate  for  systemic  change 

Most  of  the  critical  players  in  Taiwan’s  educational  reform  did  not  harbor  negative  feelings  toward  the 
existing  system  before  the  reform  started;  subsequently,  they  were  seemingly  unaware  of  the  urgency  for  a reform  of 
the  mathematics  curriculum.  The  motivation  for  change,  in  large  part,  can  be  attributed  to  suggestions  from  two 
groups:  a large  number  of  college-level  mathematics  education  professors  and  researchers,  and  a handful  of 
elementary  school  principles  and  mathematics  teachers  who  were  aware  of  international  trends  in  elementary 
mathematics  education.  These  persons  proposed  to  Taiwan’s  Ministry  of  Education  that  Taiwan’s  official 
elementary  mathematics  curriculum  be  revised;  the  Ministry  of  Education  authorized  them  to  do  so  and  appointed 
key  figures  to  edit  mathematics  textbooks  and  resources. 

Judging  from  the  above,  it  is  apparent  that  the  reform  was  not  a grass-roots  action  lead  by  the  needs  of 
elementary  school  teachers,  administrators,  and  parents.  Thus,  many  teachers  continue  to  use  traditional  instructional 
methods  to  teach  the  new  mathematics  curriculum.  There  is  no  climate  for  change  for  the  mathematics  instructional 
method,  and  needless  to  say,  there  is  no  fertile  ground  for  a systemic  change  of  the  entire  educational  system  that 
would  support  the  constructivist  mathematics  curriculum. 

Guideline  2:  Make  sure  you  have  powerful  instigation 

Usually,  elementary  school  principals  enjoy  a great  deal  of  power  because  they  have  been  directly 
appointed  by  Taiwan’s  Ministry  of  Education,  not  the  school  board.  Moreover,  the  Ministry  of  Education  authorizes 
them  to  decree  school  policies  and  the  administration  of  educational  reform.  As  principals,  the  above  tasks  constitute 
their  primary  Job  responsibilities,  and  are  the  basis  for  promotions.  Therefore,  school  principals  are  the  ones  who  are 
most  vocal  in  their  support  of  educational  reforms.  In  contrast,  staff  members,  general  administrators  and  teachers 
tend  to  be  passive  because  of  their  fear  of  change  and  their  unwillingness  to  accept  the  additional  burdens  associated 
with  such  change.  Consequently,  the  principals  often  have  a difficult  time  finding  teachers  willing  to  participate  in 
the  reform.  Principals  often  try  to  find  highly  motivated  and  respected  teachers  to  serve  as  instigators  of  the  reform. 
If  the  principals  are  unable  to  persuade  a sufficient  number  of  such  educators,  they  often  order  young  teachers,  who 
tend  to  be  more  obedient  to  the  principal  than  experienced  teachers.  In  Taiwan,  community  members  (parents, 
business  leaders  and  other  influential  persons)  seldom  become  involved  actively  in  school  activities.  The  bulk  of 
school  funding  comes  from  central  and  local  taxes,  not  from  businessmen’s  donations. 

Guideline  3:  Build  a school  and  community  coalition  of  restructurers 

As  stated  above,  community  members  were  far  removed  from  school  activities;  therefore,  there  were 
virtually  no  partnerships  between  schools  and  community  members  in  the  school  reform  process. 

Guideline  4:  Build  public  support  for  the  change 
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Reform  took  place  almost  completely  in  professional,  educational  circles.  Community  members,  even  the 
students*  parents,  did  not  know  much  of  the  reform.  Over  the  past  few  years,  the  media  has  only  sporadically 
reported  these  events;  such  issues  tended  to  appear  solely  in  scholarly  periodicals  that  deal  with  education. 
Meanwhile,  the  constructivist  mathematics  curriculum  quietly  developed  on  school  camp  uses. 

Guideline  5:  Document  the  experience  from  the  beginning 

Taiwan’s  official  model  of  constructivist  mathematics  curriculum  was  tested  in  several  schools  over  a four- 
year  period  before  it  was  deemed  fit  for  application  nationwide.  A great  deal  of  qualitative  and  quantitative  research 
has  documented  the  different  phases  of  implementation,  for  example:  teachers*  changes  in  pedagogy  (Ko,  1996),  the 
process  of  instruction  (Tzeng,  1997),  and  students’  achievements  and  shifting  attitudes  toward  mathematics  (Chung, 
1997).  The  research  results  came  in  both  print  and  non-print  forms,  and  were  quite  useful.  Unfortunately,  because  of 
limited  quantity  and  circulation,  teachers,  parents  and  administrators  did  not  have  many  opportunities  to  access 
them. 

Designing  the  new  system 

Guideline  6:  Involve  all  stakeholder  groups  in  deciding  on  and  carrying  out  the  design  process 

As  stated  above,  only  a few  mathematics  professors,  researchers,  administrators  and  teachers  initiated  and 
developed  this  new  curriculum.  The  stakeholder  groups  in  the  majority  of  schools  were  not  involved  in  the  design 
process.  Thus,  they  were  neither  familiar  with  nor  enthusiastic  about  the  reform;  they  could  not  play  the  role  of 
advocators  for  the  new  curriculum. 

Guideline  7:  Get  good  leadership  and  outside  facilitation 

Owing  to  the  limited  number  of  persons  trained  in  the  new  curriculum,  there  were  not  enough  teachers  to 
demonstrate  it  in  the  2,583  elementary  schools  throughout  Taiwan.  Normally,  teachers  who  were  somewhat  familiar 
with  the  new  methods  simply  would  supervise  their  colleagues,  or  teachers  from  the  same  school  would  form 
collective  training  sessions,  through  which  they  could  learn  from  each  other. 

Guideline  8:  Develop  a common  mindset  for  the  new  paradigm 

Unfortunately,  most  of  those  involved  in  the  reform  viewed  it  more  in  terms  of  curriculum  and  instructional 
methods,  rather  than  in  terms  of  a fundamental  change  of  mindset  about  education.  The  reformers  failed  to  create  a 
special  atmosphere  for  sharing  common  values  and  goals. 

Guideline  9:  Design  a feasible  model  for  the  system 

Educators  published  several  instructional  models  on  constructivist  mathematics.  These  models  were 
feasible,  and  came  with  teacher’s  manuals,  so  they  were  not  too  difficult  to  teach  from.  However,  I think  the 
problems  are  not  in  these  instructional  models,  but  the  problems  behind  the  instructional  models  and  those  on  the 
periphery.  By  “behind,”  1 mean  the  epistemology  and  assumption  of  learning.  Peripheral  problems  are  training  and 
diffusion  issues.  These  have  been  or  will  be  discussed  in  each  guideline,  so  I will  not  elaborate  on  them  here. 

Guideline  10:  Establish  and  maintain  communication  links 

Communication  links  entail  internal  communications  such  as  holding  regular  public  meetings  and  utilizing 
various  media,  and  external  communications  such  as  linking  up  with  other  schools  and  joining  networks  (Reigeluth 
et  al,  1991).  Not  surprisingly,  these  links  were  scarce  within  schools  and  between  schools. 
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Planning  the  implementation: 


Guideline  11:  Experiment  with  tryouts  and  pilots,  and  make  revisions 

Among  the  publications  implementing  elementary  constructivist  mathematics,  only  the  textbooks  designed 
and  developed  by  Taiwan’s  government  agency  were  used  in  the  four-year  pilot  program.  The  rest  of  the  textbooks 
did  not  go  through  this  process,  therefore,  they  created  a significant  amount  of  confusion  among  teachers.  This 
situation  proves  that  the  testing  process  is  necessary. 

Guideline  12:  Designating  time  and  funds  to  develop  and  participate  in  staff  development 

Neither  substantial  time  nor  sufficient  funding  has  been  provided  to  facilitate  the  diffusion  of  the  reform. 

Guideline  13:  Designate  time  and  funds  to  procure  learning  resources,  and  equipment  and  to  remodel 
facilities 


The  lack  of  time  and  funds  was  regarded  as  the  most  frustrating  situation  for  teachers.  Teachers  complained 
that,  even  though  there  were  enough  constructivist  mathematics  instruction  resources  for  them  to  use,  there  was  a 
lack  of  quality  instructional  media  to  go  with  each  lesson  in  the  new  curriculum. 

Guideline  14:  Obtain  active  resource  support  from  all  stakeholders 

This  reform  was  mandatory,  so  all  stakeholders  were  supposed  to  contribute  as  many  human  and  material 
resources  as  possible  to  the  implementation  efforts.  Teachers  also  had  the  obligation  to  contribute  to  the 
implementation  efforts.  However,  this  was  a mandatory  and  not  a grass-roots  action;  as  a result,  not  all  stakeholders 
and  teachers  were  involved  in  the  implementation.  Some  stakeholders  rejected  the  training  and  opted  for  early 
retirement. 

Implementing  the  design: 

Guideline  15:  Allow  parents  to  choose  to  send  their  children  to  the  new  system 

Since  this  reform  was  mandatory,  all  Taiwanese  elementary  schools,  private  and  public,  took  the  identical 
approach  to  instruction.  Parents  did  not  have  a say  in  the  matter. 

Guideline  16:  Institutionalize  the  change  process  with  regular  formative  evaluations  by  the  stakeholders 

Reigeluth  mentioned  three  kinds  of  strategies  for  this  guideline:  evaluate  the  system  continuously,  include 
all  stakeholders  in  a shared  governance  system,  and  consider  an  equitable  choice  plan  and  incentive  system  to 
encourage  continual  systemic  change.  1 do  not  know  if  these  strategies  occurred  in  Taiwan’s  case;  1 need  more  time 
to  investigate  these  issues.  Regardless,  the  reform  often  times  dies  off  within  a few  years  due  to  a lack  of  these 
strategies. 

Guideline  17:  Maintain  communication  with  stakeholders,  non-participants  and  outsiders  interested  in 
systemic  restructuring 

Taiwanese  schools  often  are  isolated  from  the  surrounding  communities.  These  institutions  lack  a 
comprehensive  channel  of  communication  between  those  inside  and  outside,  participants  and  non-participants,  or 
even  among  stakeholders.  Schools  tend  to  introvert  themselves,  and  this  closed  system  is  not  conducive  to  reform. 

Guideline  18:  Allow  three  years  before  doing  a summative  evaluation 

To  my  knowledge,  there  is  no  plan  yet  to  conduct  a nationwide  formative  or  summative  evaluation  of  this 
reform.  Small-scale  formative  evaluations  on  students’  achievements  in  mathematics  and  attitudes  toward  the  new 
curriculum  were  carried  out  (Chung,  1 997). 
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Conclusion 


In  this  paper,  I started  with  the  significance  of  the  study  of  the  implementation  of  Taiwan’s  constructivist- 
approach  elementary  mathematics  curriculum.  I briefly  described  its  new  trends  and  revealed  the  problems  related  to 
such  educational  reform  by  examining  the  difficulties  associated  with  the  implementation  of  this  curriculum.  In 
order  to  achieve  an  appropriate  perspective  to  view  these  problems,  it  was  necessary  to  utilize  a systems  design 
approach  rather  than  a piecemeal  approach.  Reigeluth’s  guidelines  for  the  design  process  were  used  to  analyze  the 
situation.  These  guidelines  guided  us  to  view  these  situations  both  comprehensively  and  systematically  through  four 
phases  of  design  process.  Therefore  we  did  not  miss  any  essential  process  to  be  trapped  into  a partial  perspective, 
like  the  blinds  touched  an  elephant.  Further,  they  will  continue  to  guide  us  on  how  to  take  action  for  future  reform 
action. 
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Introduction 

In  recent  years,  academic  deficiencies  of  U.  S.  science  students  have  been  of  primary  concern  to  both  the 
U.  S.  Department  of  Education  (Henry,  1997)  and  the  National  Research  Council  (NRC)  (1996).  More  specifically, 
Georgia  students  scored  in  the  lowest  25%  of  students  tested  in  40  states  and  51%  scored  at  the  “below  basic”  level 
of  understanding  science.  In  light  of  such  reports,  strategies  to  improve  science  teaching  and  learning  have  become  a 
focus  of  professional  development  efforts  around  the  country. 

Due  to  the  dynamic  nature  of  teaching  and  learning  science  in  our  modem,  technological  world,  it  would  be 
reasonable  to  assume  that  teachers,  not  just  their  students,  should  be  dedicated  learners.  The  job  of  teaching  must  be 
redefined  to  include  continuous  teacher  learning  (Berlin,  1996).  According  to  the  National  Science  Education 
Standards  (NSES)  (1996),  learning  has  a two-fold  purpose  in  science  education:  (1)  to  keep  current  in  science  and 
(2)  to  deepen  the  understanding  of  effective  science  teaching  strategies.  Such  educational  strategies  should  be 
aligned  with  the  NSES  to  include  science  as  inquiry,  utilization  of  technology,  and  collaboration  among  educators. 
Additionally,  successful  professional  development  activities  provide  explicit  connections  to  teacher  needs  as  well  as 
a sense  of  ownership  in  the  learning  process. 

The  NSES  clearly  outlined  objectives  for  teaching  and  learning  science.  Paramount  to  all  science  education 
was  science  as  inquiry.  Inquiry,  the  watchword  of  constructivist  reform,  dictates  that  any  knowledge  and 
understanding  must  be  (1)  actively  acquired;  (2)  socially  constructed;  and  (3)  created  and  then  recreated  for 
individualized  meaning  (Perkins,  November  1999).  NSES  guidelines  vary  greatly  from  the  passive  mode  of  learning 
that  pervades  education  today,  and  logically  teachers  would  require  substantive  support  and  education  to  master  the 
inquiry  techniques  advocated. 

As  students  become  more  fluent  with  technology,  it  is  imperative  that  educators,  as  well,  increase  their 
technological  comfort  level  and  rise  to  meet  the  needs  of  their  students.  According  to  Jane  Healy,  as  cited  in  Tell, 
(October,  2000),  technology  is  a great  gift  to  education  and  affords  teachers  and  students  opportunities  to  initiate 
important  learning.  Such  learning,  while  easily  aligned  with  constructivist  ideals,  must  be  planned  and  guided  by 
teachers.  Fundamental,  then,  to  any  professional  development  effort  is  providing  school  staff  with  ongoing  training 
to  increase  teacher  fluency  as  well  as  link  technology  with  authentic  work  (Dooling,  October,  2000). 

Finally,  as  mandated  by  the  NSES,  collaboration  should  be  at  the  heart  of  teaching  and  learning.  As 
mentioned  previously,  constructivist  learning  is  a highly  social  process.  Knowledge  and  understanding  for  students 
and  teachers  are  constructed  in  dialogue  with  others  (Perkins,  1999). 

A regional  university  of  the  University  System  of  Georgia,  Valdosta  State  University  (VSU),  fulfills  the 
academic  needs  of  the  South  Georgia  area.  The  area  serviced  by  VSU  contains  a high  proportion  of  minorities  and 
students  from  economically  disadvantaged  families.  Student  performance  on  the  state  mandated  science  assessment, 
the  Georgia  High  School  Graduation  Test,  was  well  below  achievement  levels  compared  other  subject  areas 
(Georgia  Department  of  Education,  1999).  VSU  must  reach  out  to  science  teachers  in  the  area  to  improve  teaching 
skills  if  their  students  are  to  become  productive,  contributing  members  of  local  communities.  It  was  with  these 
needs  in  mind,  that  the  inquiry  learning  and  technology  utilization  project  for  middle  grades  and  high  school 
teachers  was  developed. 

The  educational  significance  of  this  study  was  to  advance  the  existing  body  of  knowledge  and  improve 
science  classroom  instruction  by  assisting  middle  and  high  school  teachers  to  (1)  become  knowledgeable  and 
proficient  with  inquiry -based  teaching  consistent  with  both  state  and  national  education  reform  efforts  (GIMS,  1996; 
NRC,  1996),  and  (2)  obtain  necessary  experience  and  skills  to  incorporate  instructional  technologies  into  the  * 
inquiry -based  teaching  format.  Due  to  the  one-year  length  of  the  project,  teachers  were  supported  through  concerns 
identified  with  the  three  distinct  stages  of  change  implementation:  preparation,  acceptance,  and  commitment 
(Rogers,  1996;  Salisbury,  1996)  allowing  discipline- wide  adoption  and  considerable  change  in  practice  to  be 
achieved. 
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Worthwhile  educational  changes  require  new  knowledge,  attitudes,  and  behavior  (Fullan,  1991).  A striking 
departure  from  the  classroom  norm,  constructivist  theory  and  science  as  inquiry  demands  the  active  participation  of 
the  learner,  not  passive  receptivity.  The  authoritative  role  previously  played  by  teachers  has  been  transformed  into 
one  of  facilitator  (Mancino,  1995).  Additionally,  while  there  are  growing  pressures  for  teachers  to  embrace 
technology  in  the  classroom,  where  there  exists  no  specific  mandate  to  do  so,  there  will  be  continued  reluctance  on 
the  part  of  teachers  to  change  from  what  is  familiar  and  comfortable  (Robinson,  2000). 

Professional  Development 

Dorit  (1999)  hypothesized  that  professional  development  programs  should  put  teachers  in  the  role  of 
learners  in  an  attempt  to  facilitate  epistemological  change  and  influence  their  use  of  technology  and  constructivist 
teaching  practices.  Within  the  realm  of  a multimedia  constructivist  program,  science  teachers  worked  cooperatively, 
with  minimum  supervision,  and  conducted  investigations  regarding  the  utilization  of  a multimedia  package  in  their 
classroom.  The  study  suggested  that  collaboration,  time  for  reflection,  and  active  teacher  participation  as  learner  all 
contributed  to  the  success  of  inquiry -based  learning  and  a change  in  teacher  thinking. 

Effective  professional  development  for  science  education  should  draw  from  a synthesis  of  standards  posited 
by  the  NRC  (1 996)  (Loucks-Horsley,  Stiles,  and  Hewson,  1996).  Included  in  these  standards  is  the  development  of  a 
learning  community.  Stein  (1998)  provided  a view  of  professional  development  at  its  best:  a systemic  investment  in 
building  teacher  capacity  through  collaboration.  In  this  case,  professional  development  was  seen  as  everyone’s  job 
and  everyone  viewed  themselves  responsible  not  only  for  their  learning  but  also  for  the  learning  of  colleagues.  Such 
a collaborative  effort  led  to  teacher  change,  a newly  implemented  constructivist  approach  to  teaching  and  learning, 
and  improved  student  performance  district- wide. 

Teachers,  like  students,  learned  science  as  inquiry  best  by  doing  science  as  inquiry  by  investigating  for 
themselves,  and  building  their  own  understanding  (Supovitz,  Mayer,  and  Kahle,  2000).  A standards-driven  inquiry- 
based  professional  development  effort  in  Ohio  produced  strong,  positive,  and  significant  growth  in  teachers’ 
attitudes  toward  inquiry -based  instruction,  as  well  as  their  use  of  inquiry  in  the  classroom.  Not  only  did  change 
occur,  it  was  sustained  for  up  to  three  years  following  involvement. 

In  a five-year,  longitudinal  study,  Berlin  (1996)  evaluated  the  effects  of  action  research  on  teacher  attitude 
toward  inquiry,  as  well  as  improved  teaching  and  learning.  Berlin’s  model  mimicked  the  stages  of  the  learning 
cycle:  Awareness,  Research  and  Development,  and  Application.  Not  only  were  science  educators  introduced  to  the 
philosophy  of  science  as  inquiry,  professional  development  strategies  were  inquiry-based.  To  repeat,  teachers 
learned  inquiry  by  doing  inquiry.  Quantitative  and  qualitative  data  suggested  that  inquiry -focused  action  research 
enhanced  teacher  attitudes  towards  both  educational  research  and  inquiry -based  instruction.  Additionally,  action 
research  facilitated  the  implementation  of  educational  innovations  in  the  classroom  and  improved  teaching  and 
learning. 

The  results  of  past  learning  cycle  institutes,  sponsored  by  the  National  Science  Foundation,  have 
documented  long  term  changes  in  teaching  strategies  of  the  participants  (Marek,  Haack,  and  McWhirter,  1994). 

Most  significantly,  93%  of  participating  science  teachers  continued  to  use  learning  cycles  in  their  science  programs 
nearly  a decade  after  the  institute.  Teachers  stated  that  the  learning  cycle  procedures: 

• Involved  students  actively  in  the  learning  process 

• Produced  deeper  understanding  and  greater  retention  of  concepts 

• Developed  students’  thinking  and  communication  skills 

• Included  teaching  process  as  well  as  content 

• Made  science  relevant  and  meaningful  to  students. 

As  mentioned  previously,  teachers  learned  inquiry  by  doing  inquiry.  If,  indeed,  the  learning  cycle  so  greatly 
impacts  student  learning,  one  would  expect  similar  results  on  teacher  learning  as  well. 

Methods  and  Procedures 

Participants 

The  participants  of  the  research  project  included  50  public  school  teachers  from  four  counties  in  South 
Georgia,  certified  to  teach  either  middle  grades  or  high  school  science.  A Florida  group  included  20  teachers 
certified  to  teach  grades  4-12. 

Procedure 
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Exploration  phase.  Science  teachers  met  for  five  days  during  the  summer  from  9 a.m.  to  3 p.m.  During  the 
first  ninety  minutes  they  attended  seminars  lead  by  science  education  professors  to  examine  (a)  the  structure  of 
science;  (2)  the  nature  of  human  learning;  and  (3)  authentic  assessment  strategies  for  student  evaluation  in  learning 
cycle  curricula.  Teacher-participants  spent  the  rest  of  each  day  in  laboratory  session  lead  by  project  staff  These 
sessions  included  (1)  technology  activities  designed  to  familiarize  teachers  with  the  use  of  various  educational 
technologies  and  (2)  science  laboratories  modeling  the  learning  cycle  procedure  led  by  in-service  middle  and  high 
school  teachers  experienced  in  inquiry  teaching. 

Application  phase.  Once  school  started  in  August,  teacher-participants  met  with  project  staff  every  alternate 
Saturday  for  eight  weeks  from  8 a.m.  until  3 p.m.  Teachers  received  copies  of  learning  cycle  curriculum,  and  with 
the  assistance  of  project  staff,  modified  two  weeks  of  current  science  curriculum  into  inquiry -based  lessons. 
Successes  and  difficulties  of  implementation  were  discussed  on  a regular  basis. 

Follow-uP  phase.  For  the  remainder  of  the  school  year  a member  of  the  project  staff  conducted  four 
observations  of  each  participating  teacher  utilizing  the  learning  cycle  in  the  classroom.  Following  each  observation, 
a meeting  between  teacher  and  staff  member  was  conducted  to  discuss  implementation  of  inquiry -based  teaching 
procedures,  incorporation  of  technology,  assessment,  or  other  factors  associated  with  teaching  change.  In  addition  to 
individual  meetings,  two  meetings  with  all  teachers  and  project  staff  were  conducted  to  brainstorm  solutions  to 
problems  and  to  share  successes. 

Instrumentation 

An  Advisory  Panel,  consisting  of  a scientist,  educational  technologist,  certified  Teacher  Support  Specialist, 
two  middle  school  and  two  high  school  teachers,  and  two  science  education  professors,  developed  a preliminary 
survey  administered  previous  to  participation  in  the  summer  workshop  and  designed  to  measure  teacher  attitudes 
and  pedagogy.  A follow-up  survey  was  designed  to  measure  attitudinal  change  immediately  following  the  Saturday 
workshops.  Teacher  observations  were  conducted  using  a checklist  developed  in  conjunction  with  survey  data  and 
teachers  submitted  reflective  responses  during  follow  up  sessions.  Student  achievement  was  ascertained  via  pre  and 
post  subject-matter  tests  administered  by  participating  teachers. 

Results 

Students 


Student  pre  and  post-test  data  indicated  adequate  to  high  levels  of  science  achievement.  More  importantly, 
teachers  noted,  via  reflective  writings,  an  increase  in  student  enthusiasm.  Students  appeared  genuinely  eager  to  be 
engaged  in  science  and  actively  participated  in  providing  suggestions  for  classroom  investigations.  One  teacher 
noted  how  the  inquiry  format  facilitated  learning  for  all  students.  Non-reading  middle  grade  students  could  readily 
participate  in  hands-on  science  activities  while  she  monitored  their  learning  through  performance  based  assessments 
rather  than  traditional  paper-and-pencil  items.  Another  teacher  commented  on  the  development  of  classroom 
cooperation  through  increased  use  of  inquiry  activities.  Inclusion  students,  usually  excluded  from  science  learning 
due  to  the  traditional  lecture  format,  were  now  able  to  actively  participate  within  cooperative  inquiry  groups. 

The  spirit  of  cooperation  and  inquiry  spilled  over  into  other  learning  opportunities  as  well.  A student  in  a 
tech-prep  biology  class  instructed  classmates  how  to  use  PowerPoint  to  present  data  collected  during  an  inquiry 
activity  while  another  student  taught  classmates  in  ecology  how  to  use  the  school’s  digital  camera  to  record  data  for 
a science  investigation.  Cooperative  activities  lead  to  instances  of  shared  responsibility  in  other  areas  as  well.  When 
a student  acted  inappropriately  on  the  school  grounds  while  looking  for  rocks,  the  teacher  took  that  opportunity  for 
the  class  to  look  for  solutions  to  resolve  the  incident. 

At  first  glance,  many  classrooms  seemed  noisy  and  disorderly.  In  reality,  after  implementing  inquiry 
activities  and  increasing  student  input,  teachers  had  fewer  discipline  problems.  Active  engagement  in  learning 
science  really  was  just  that  and  students  who  were  busy  doing  science  did  not  have  time  to  cause  problems. 

Teachers 

Teacher  response  was  overwhelming  positive.  Answers  to  attitudinal  survey  questions  ranged  from  “agree” 
to  “strongly  agree”.  Not  only  did  the  participants  state  that  the  professional  development  addressed  their  needs,  they 
also  stated  at  the  end  of  the  weeklong  meeting,  that  they  agreed  with  the  educational  approaches  presented  and 


267 


would  be  able  to  incorporate  both  inquiry  and  technology  into  their  daily  lessons.  Individual  comments  included 
such  things  as  “empowering”,  “energizing”,  and  “a  reminder  that  science  is  supposed  to  be  active  and  exciting”. 

Additional  meetings  and  correspondence  with  teachers  indicated  unwavering  support.  At  the  initial  2 -week 
interval  meeting,  one  group  of  middle  school  teachers  had  converted  10  weeks  of  lesson  plans  to  inquiry  format. 

Lack  of  textbook  support  materials  caused  some  teachers  to  voice  concern,  but  the  support  of  the  project’s  master 
teachers  spurred  them  on  the  inquiry  path.  One  early  disbeliever,  by  the  third  meeting,  had  taught  an  entire  three- 
week  science  unit  using  the  inquiry  format.  She  gave  credit  for  her  conversion  to  the  master  teacher  who  helped  her 
design  the  unit. 

If  there  was  a single  result  that  stood  out  among  all  others  it  was  the  evolution  of  teachers  to  teacher- 
collaborators.  After  the  summer  session,  teachers  shared  a vision  of  change  in  the  science  classroom.  Incorporating 
technology  and  modifying  existing  curricula  to  an  inquiry  format  is  a huge  task  for  one  person.  Recognizing  that 
there  is  strength  in  nunnbers  and  acknowledging  the  skills  of  some  to  lead  the  way,  the  groups  of  teachers  from  each 
system  became  collaborators.  As  2002  is  also  science  textbook  adoption  year,  teachers  have  decided  to  collaborate 
on  the  selection  of  textbooks  to  facilitate  incorporating  the  goals  of  this  project.  There  are  plans  to  develop  a web 
site  for  sharing  of  ideas  and  inquiry  formatted  lesson  plans  as  well. 

Individually,  one  teacher  commented  on  how  the  newly  generated  student  enthusiasm  for  science  as 
energized  her  as  well.  A physical  science  teacher  discussed  how  this  format  seemed  more  efficient  to  her  since  many 
of  her  students  rarely  read  the  material  in  the  text  anyway.  Once  their  interest  was  piqued  by  the  inquiry  activities, 
they  were  more  likely  to  go  back  and  read  pertinent  textual  information.  Another  science  teacher  recently  relegated 
to  P.E.  duty,  incorporated  inquiry  activities  into  the  health/P.E.  arena  by  guiding  students  through  an  investigation  of 
respiration,  pulse  rate,  and  physical  activity.  She  commented  on  what  a great  opportunity  that  provided  for 
integrating  health,  science,  and  mathematics  in  an  otherwise  academic -free  zone. 

On  a broader  scale,  teachers  acknowledged  the  importance  of  the  support  offered  by  the  project  staff  One 
teacher  commented  on  previous  staff  development  activities:  “They  spend  a day  or  two  with  you  and  then  they 
leave.  You  don’t  know  what  to  do  and  no  one  else  does  either.”  In  contrast,  the  continued  support  by  the  project 
leaders  in  this  instance  was  empowering.  The  same  teacher  added:  “I  know  I can  count  on  this  group  (project 
leaders)  if  I need  help.” 

The  support  by  the  project  members  and  co-workers  has  inspired  several  veteran  teachers  to  venture  into 
uncharted  territory.  Traditional  lecture -driven  classrooms  were  coming  alive  with  the  noisy  hum  of  inquiry 
activities.  Teachers  who  never  wanted  a computer  in  their  room  now  were  asking  for  several  student  computers  to  be 
installed.  One  school  system,  in  support  of  their  science  educators,  applied  for  and  received  a $125,000  math  and 
science  technology  grant.  This  money  will  be  spent  on  installing  a presentation  station  in  each  science  classroom  as 
well  as  purchasing  a set  of  15  laptop  computers  with  Internet  access. 

Conclusion 

Transformation  may  be  the  watchword  of  education  today.  As  we  charge  into  the  information  age,  priorities 
and  paradigms  shift.  What  was  valued  a few  decades  ago,  mass-produced  knowledge,  has  been  replaced  by 
individually  constructed  knowledge.  Active  replaces  passive.  Teachers,  too,  must  become  active  learners  to  master 
the  changing  scene  in  education,  a scene  that  requires  science  as  inquiry  and  the  integration  of  technology.  The  key 
concepts  described  in  the  NSES  formed  the  foundation  for  this  project  through  the  use  of  inquiry -based  learning 
cycle  teaching  procedures,  technology  integration,  and  system-wide  collaboration  of  science  teachers.  Worthwhile 
educational  change  requires  new  knowledge,  attitudes,  and  behaviors  (Fullan,  1991).  The  inquiry -technology 
integration  project  seemed  to  effectively  supply  these  key  requirements  to  science  educators  in  South  Georgia. 
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Integrating  Internet-based  Learning  In  An  Educational  System:  A Systemic  Approach 


Marshall  G.  Jones 
Winthrop  University 

Stephen  W.  Harmon 
Georgia  State  University 

The  first  time  we  taught  an  online  class,  we  didn’t  tell  anybody  that  was  what  we  were  going  to  do.  In  1996 
when  we  first  conceived  of  teaching  a class  on  internet-based  learning  using  the  medium  of  internet-based  learning, 
we  weren’t  entirely  sure  how  we  would  go  about  it.  And  given  the  myriad  of  rules  and  regulations  associated  with 
public  higher  education  such  as  contact  hours,  new  course  approvals,  and  teaching  evaluations  as  they  relate  to 
tenure  and  promotion,  we  thought  it  best  that  we  keep  things  quiet  less  they  go  very,  very  wrong.  Whether  or  not  our 
first  class  went  very,  very  wrong  is  open  to  debate,  but  our  online  course  is  no  longer  a secret  (Harmon  & Jones,  in 
press;  Harmon  & Jones,  1999;  Jones,  Harmon,  & Lowther,  2002;  Jones  & Harmon,  In  Press).  Online  courses  are 
being  taught  at  established  universities  and  newly  founded  virtual  universities.  Department  chairs.  Deans,  and 
Presidents  are  all  anxious  to  put  classes  online.  Position  announcements  specifically  request  people  with  online 
teaching  experience,  and  course  loads  are  defined  by  online  courses.  For  better  or  worse,  online  courses  are  here, 
apparently  to  stay,  and  university  teaching  may  be  fundamentally  changed  as  a result. 

But  teaching  is  only  one  part  of  higher  education.  For  those  of  us  working  in  the  academy,  we  recognize 
immediately  that  what  we  teach  and  how  we  teach  impacts  our  time  and  resources.  And  time  and  resources  impact 
everything  else  we  are  asked  and  required  to  do.  Consequently  a fundamental  change  in  teaching  will  also  change 
how  we  do  other  things  in  higher  education,  such  as  research  and  service. 

While  we  have  defined  varying  levels  of  Web-based  instruction  (WBI)  (Harmon  & Jones,  1999),  here  we 
would  like  to  take  a broader  look  at  WBI  as  it  applies  to  higher  education.  Specifically,  we  are  attempting  here  to 
define  the  educational  system  under  girding  higher  education.  Our  purpose  here  is  to  identify  the  various 
components  of  the  system  to  illustrate  how  and  where  they  interact,  overlap,  and  come  together  so  that  we  may 
better  understand  how  WBI  may  impact  higher  education.  To  that  end,  we  must  first  define  an  educational  system  in 
order  to  define  our  educational  system,  higher  education. 

Defining  an  Educational  System 

Banathy  ( 1 995)  argues  that  modern  society  imposes  on  us  complexities  that  cannot  be  dealt  with  by 
traditional  reductionist  methods  of  science.  In  this  era  of  rapid  information  flow  human  systems  interact  with  their 
environment  in  fluid  and  dynamic  ways.  There  are  too  many  interacting  variables  to  allow  easy  problem  solving; 
perhaps  the  best  we  can  hope  for  is  "problem  management."  But  in  order  to  achieve  even  this  level  of  control  we 
must  first  understand  our  systems  and  how  they  operate.  This  need  becomes  even  greater  in  light  of  the  additional 
complexity  potential  resident  in  the  vast,  chaotic  amalgam  that  is  the  World  Wide  Web.  Realizing  that  entire 
textbooks  have  been  written  on  systems  and  systems  theory  (i.e.  Banathy,  1968;  von  Bertalanffy,  1968),  our  goal 
here  is  to  provide  a few  simple  guidelines  for  consideration  when  implementing  WBI. 

Principle  1:  A system  is  a set  of  organized  components  working  toward  a common  goal. 

Every  educational  system  is  working  towards  a common  goal,  whatever  that  goal  may  be  (Banathy,  1991). 
For  a corporate  environment,  that  goal  may  be  to  increase  revenue.  If  we  consider  a retail  environment  for  a 
moment,  the  training  of  new  sales  people  will  put  more  people  on  the  floor  working  with  customers  effectively, 
which  may  increase  customer  satisfaction,  which  can  generate  increased  sales,  which  in  turn  drives  profits  and 
ultimately  corporate  revenue.  But  training  can  be  costly  for  the  company.  The  organization  must  pay  not  only  the 
wages  of  trainers  and  students,  but  also  the  costs  of  developing  the  training  and  the  associated  costs  of  travel, 
lodging,  per  diem  and  lost  productivity.  An  organization  must  decide,  based  on  its  goals,  what  training  should  look 
like,  how  much  it  can  cost,  and  how  ultimately  it  meets  the  goals  of  the  system.  Being  able  to  align  strategic  goals 
and  missions  of  the  system  to  the  training  environment,  assures  replicability,  and  allows  for  precise  troubleshooting 
when  some  component  of  the  instruction  fails  (Dick  & Carey,  1996). 
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Principle  2:  A change  to  one  component  of  a system  may  cause  a change  in  every  other  component  of  that 
system. 


Understanding  this  idea  is  central  to  an  effective  use  of  Web-based  Instruction.  For  most  of  us,  WBI  is  a 
very  different  way  of  doing  business  than  that  to  which  we  are  accustomed.  When  we  make  the  often  radical  change 
to  WBI  we  run  the  risk  of  causing  drastic  changes  in  the  rest  of  our  organization.  Offering  a course  on  the  Web 
requires  us  to  reconsider  aspects  of  the  system  that  we  often  take  for  granted.  How  will  the  change  affect  textbooks? 
What  will  become  of  custodial  services?  What  happens  to  corporate  and  university  libraries  when  students  are  no 
longer  on  campus?  Will  scheduling  change  from  the  purview  of  facilities  to  the  purview  of  information  systems? 
These  and  numerous  other  issues  will  need  to  be  considered.  Rememb  er  the  lesson  of  chaos  theory,  that  a butterfly 
flapping  its  wings  in  Brazil  can  create  a hurricane  in  Texas  (Gleick,  1987),  only  changing  to  WBI  is  not  so  much 
like  a butterfly  flapping  its  wings  as  it  is  a jet  plane  revving  its  engines. 

Principle  3:  Every  educational  system  is  different. 

While  general  systems  theory  holds  that  there  are  elements  common  to  systems,  every  system  is  unique. 
Every  system  has  particular  features  about  it  that  provide  its  strengths  and  weaknesses.  There  do  exist  macro  level 
pieces  of  a system  that  may  be  common  to  any  educational  system.  Students,  instructors,  assessment,  delivery,  and 
management  can  be  common  to  all  environments.  However,  different  environments  will  have  different 
requirements.  For  example,  the  instructors  in  a corporate  environment  may  have  no  other  responsibilities  beyond 
those  associated  with  the  classes  they  teach.  In  higher  education,  however,  instructors  often  have  not  only  teaching 
responsibilities,  but  research  expectations  and  committee  assignments  as  well.  While  both  may  be  involved  in 
instruction,  the  other  components  of  the  system  will  vary.  Analysis  of  your  own  system  and  its  particular 
requirements  is  important  to  before  training  begins. 

It  is  important  to  note  that  we  are  working  within  the  framework  of  Higher  Education  here.  While  we  have 
written  on  WBI  in  other  environments,  (Jones,  Harmon,  & Lowther,  2002)  we  are  writing  here  specifically  to  higher 
education.  We  chose  higher  education  as  an  example  for  a number  of  reasons.  One,  it  is  an  environment  that  we 
work  in  and  understand.  Two,  it  is  an  environment  that  is  often  imitated  in  other  educational  institutions.  We  feel 
that  much  of  what  we  say  could  apply  to  any  educational  system,  but  because  of  space  we’ll  limit  our  discussion  to  a 
university  environment. 

Defining  the  mission 

To  begin,  we  focus  not  on  student  learning,  but  on  the  mission  of  the  institution,  whatever  it  may  be.  All 
components  of  the  system  should  work  together  to  promote  this  mission.  Student  learning  may  represent  only  one 
part  of  it.  We  divide  our  model  into  four  primary  components,  three  of  which  are  based  on  the  common  faculty 
perspective  of  the  mission,  namely  teaching,  research  and  service,  supplemented  by  the  fourth  factor, 
management. 

Teaching 

The  teaching  component  includes  everything  having  to  do  with  students,  ranging  from  public  relations, 
which  might  include  such  things  as  athletic  teams,  to  on-campus  housing,  dining,  and  transportation  systems.  We 
believe  that  we  can  capture  the  complexity  of  this  component  with  three  primary  elements.  (See  Table  1)  Note  that 
as  with  all  other  pieces  of  our  system,  activities  within  one  component  often  overlap  with  other  components. 


Table  I.  Elements  of  the  teaching  component 


Element 

Definitions 

Student  Recruitment 

Deals  with  attracting  students  to  the  university.  Impacts  student  services,  quality 
of  academic  programs,  types  of  athletic  programs  and  other  factors  which  may 
influence  a student’s  decision  to  attend  a university. 

Student  Retention 

Keeping  students  enrolled  in  the  university.  Deals  with  issues  such  as  campus 
life,  academic  assistance,  and  generally  keeping  the  quality  of  the  student 
experience  high. 

Student  Matriculation 

Ensuring  that  students  progress  and  graduate  in  a timely  fashion.  Deals  with 
access  to  classes  and  curriculums  that  are  accessible  and  possible  to  finish  in  a 

271 


760 


reasonable  amount  of  time. 


Research 

At  many  universities  the  research  component  may  be  tacitly  viewed  as  the  most  important  component  of 
the  system.  Even  at  primarily  teaching  institutions,  tenure  and  promotion  decisions  may  rest  significantly  upon 
research  (Boyer,  1997).  The  degree  to  which  research  is  seen  as  a driving  force  at  a university  will  vary  considerably 
depending  upon  the  institution's  mission  and  goals.  However,  research  will  retain  a prominent  role  in  the  academic 
community,  and  the  best  researchers  will  accumulate  the  most  prestige  and  resources.  Table  2 defines  major 
elements  of  the  research  component. 


Table  2.  Elements  of  the  Research  component 


Element 

Definitions 

Research  Funding 

Identifying  funding  sources,  writing  and  obtaining  grants.  Also  includes 
administration  of  grants  and  any  profits  realized  from  research. 

Research  Investigation 

Defining  and  conducting  research.  The  establishment  and  execution  of  a 
productive  research  agenda. 

Research  Dissemination 

Making  public  the  results  of  research  through  journals,  conferences,  and 
increasingly,  the  WWW. 

Service 

Service  is  generally  the  least  rewarded  of  university  tasks  but  ironically  often  seems  to  require  the  mo  st 
time.  Typically  speaking,  service  happens  inside  the  university,  through  such  outlets  as  faculty  governance,  and 
outside  of  the  university  as  outreach  to  the  local  community  or  to  your  community  of  professionals.  Table  3 
illustrates  the  two  typical  types  of  university  service. 


Table  3.  Elements  of  the  Service  component 


Element 

Definitions 

Governance 

Participating  in  the  self-governing  process  by  serving  on  such  bodies  as  departmental 
committees  or  the  University  senate. 

Outreach 

Service  to  one’s  profession  or  community.  This  may  include  such  things  as  holding 
offices  in  professional  organizations,  serving  as  editors  for  journals,  or  volunteering  time 
for  professionally  related  community  service. 

Management 

The  fourth  major  component  of  our  system  is  often  the  least  regarded  by  faculty  and  students  because  it 
appears  to  have  little  direct  impact  on  the  teaching  and  learning.  However,  management  serves  to  insure  that  the 
proper  resources  are  available  and  are  applied  to  the  efficient  functioning  of  the  other  components.  Management 
can  be  the  glue  that  holds  the  rest  of  the  system  together.  Major  elements  of  the  management  component  are  listed  in 
table  4. 


Table  4.  Elements  of  the  management  component. 


Element 

Definitions 

Faculty  & staff  recruitment  & 
retention 

Locating  and  keeping  qualified  personnel.  Includes  developing 
and  implementing  attractive  compensation  packages.  Impacts  the 
institution’s  reputation. 

Logistical  operations 

The  day-to-day  functioning  of  a university.  Everything  from 
ordering  the  right  books  in  the  bookstore  to  making  sure  the 
trash  gets  emptied. 

Program  Evaluation 

Making  sure  that  the  system  itself  is  both  appropriate  and 
working  correctly. 

Finance 

Finance  is  an  aspect  of  the  management  component  includes 
such  things  as  budget,  publicity,  and  alumni  relations.  Budget  is 
at  its  most  basic,  the  acquisition  and  distribution  of  funds  (i.e. 

Ill 


76i 


working  with  legislatures  and  donors)  and  managing  tuition, 
fees,  and  salaries.  Since  publicity  is  usually  considered  as 
goodwill  and  has  financial  value  we  include  it  here.  Publicity 
would  include  everything  from  University  logos  to  PR 
campaigns.  Alumni  relations  involve  maintaining  contacts  with 
past  graduates  of  the  university  for  purposes  of  maintaining  the 
reputation  of  the  institution  and  in  attracting  donations. 


We  propose  these  areas  as  an  example  of  the  major  components  of  an  educational  system.  Other  aspects  of 
the  system,  such  as  stakeholders  and  the  interactions  among  the  components  are  beyond  the  scope  of  this  paper. 
While  we  realize  that  we  have  not  captured  every  element  of  every  component,  we  do  feel  that  these  are 
representative  and  give  us  a starting  place  for  looking  at  how  WBI  might  impact  the  educational  system.  However 
before  we  look  at  its  impact  on  the  system,  we  mu  st  first  look  at  how  the  Web  might  be  used  within  the  system. 

Five  Levels  of  Web  Use 

When  faced  with  the  prospect  or  requirement  or  of  using  the  Web  in  education,  many  people  assume  that 
they  are  being  asked  to  create  an  online  environment  that  will  be  a stand-alone,  self-sustaining  educational  product. 
While  this  may  be  the  goal  of  some  environments,  it  need  not  be  the  goal  of  all.  Harmon  and  Jones  (1999)  suggest 
five  levels  of  use  of  the  Web  common  in  schools,  colleges,  and  corporations.  These  levels  represent  a continuum 
from  basic  occasional  use  to  advanced  continual  use.  We  feel  that  these  levels  go  a long  way  towards  helping 
stakeholders  understand  how  the  Web  might  be  used  in  an  education  or  training  setting.  Each  level  provides  for 
particular  uses  and  classifications  of  interaction  between  the  students  and  teachers,  and  between  the  humans  and  the 
technology.  Table  Five  defines  and  summarizes  each  of  the  levels. 

Table  5.  Levels  of  WWW  use  in  education.  From  Harmon  and  Jones  (1999). 


Level  Of  Web  Use 

Description 

Level  0 : No  Web  Use 

The  default  level.  Implies  no  Web  use  at  all. 

Level  1 : Informational 

Providing  relatively  stable  information  to  the  student  typically  consisting 
of  instructor  placed  items  such  as  the  syllabus,  course  schedules,  and 
contact  information.  This  sort  of  information  is  easily  created  by  the 
instructor  or  an  assistant,  requires  little  or  no  daily  maintenance,  and 
takes  up  minimal  space  and  bandwidth. 

Level  2 : Supplemental 

Provides  course  content  information  for  the  learner.  May  consist  of  the 
instructor  placed  course  notes  and  other  handouts.  A typical  example 
would  be  a PowerPoint  presentation  saved  as  an  HTML  document  and 
placed  on  the  Web  for  students  to  review  later. 

Level  3 : Essential 

The  student  cannot  be  a productive  member  of  the  class  without  regular 
Web  access  to  the  course.  At  this  level  the  student  obtains  most,  if  not  all 
of  the  written  course  content  information  from  the  Web. 

Level  4:  Communal 

Classes  meet  both  face-to-face  and  on-line.  Course  content  may  be 
provided  in  an  on-line  environment  or  in  a traditional  classroom 
environment.  Ideally,  students  generate  much  of  the  course  content 
themselves. 

Level  5 : Immersive 

All  of  the  course  content  and  course  interactions  occur  on-line.  Does  not 
refer  to  the  more  traditional  idea  of  distance  learning.  Instead,  this  level 
should  be  seen  as  a sophisticated,  constructivist  virtual  learning 
community. 

It  is  our  position  that  these  levels  can  help  any  organization  define  how  it  plans  to  use  the  Web  in 
education.  Knowing  at  which  level  you  are  working  will  help  you  understand  how  your  work  can  affect  the  entire 
educational  system.  To  illustrate  this,  we  propose  a model  for  considering  systemic  issues  in  higher  education  as 
they  relate  to  WBI. 

The  Model 
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To  illustrate  our  proposed  method  for  accounting  for  all  systemic  issues  within  higher  education  as  it 
relates  to  WBI,  we  offer  the  model  shown  in  Figure  1.  The  model  can  be  read  by  taking  each  component  of  a higher 
education  system,  teaching  (T),  research  (R),  service  (S),  and  management  (M)  and  filtering  it  through  the  five 
levels  of  WBI,  represented  numerically.  It  is  possible  to  take  a single  component  of  the  system  and  fun  it  through  all 
levels  or  a single  level.  Likewise,  the  entire  system  can  be  run  through  a single  level  or  through  all  levels 
collectively. 


Level 
of  WBI 


Component 
of  Higher  Ed 
System 


Figure  1.  A model  used  to  address  Higher  Education  System  Components  as  they  relate  to  WBI. 


To  further  explain  how  the  model  might  be  used,  we  will  take  each  component  of  a higher  education 
system  and  discuss  it  as  it  relates  to  each  of  the  five  levels  of  WBI,  excluding  Level  0.  It  should  be  noted  that  we 
would  recommend  a careful  analysis  of  your  individual  environment.  What  we  provide  in  the  following  pages  is  a 
generic  discussion  of  issues  in  a higher  education  system  as  they  relate  specifically  to  WBI. 


Level  1:  Informational  Web  Use 


The  easiest  level  to  reach,  informational  use  of  the  Web  may  make  the  greatest  impact  on  the  Management 
component.  Teaching  itself  may  not  be  impacted  greatly,  but  student’s  decisions  on  courses  may.  This  level 
involves  merely  providing  basic  information  to  stakeholders  via  the  Web 

Teaching 

Students  today  are  thought  of  as  consumers.  Before  they  spend  money  on  a class,  or  make  a decision  on  a 
degree  program,  they  may  do  considerable  research.  While  putting  a syllabus  online  may  not  impact  instruction 
much,  it  will  impact  access  to  course  information.  This  may  provide  better  visibility  to  low  enrollment  classes  or 
provide  general  publicity  for  a particular  degree  program.  It  may  also  impact  faculty  time  and  teaching  expenses. 
Because  the  course  syllabus  is  online,  faculty  do  not  have  to  spend  as  much  time  answering  questions  about  classes 
to  prospective  students,  or  students  may  have  more  focused  questions  for  the  faculty  me  mber. 

Research 
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At  this  level,  research  may  not  be  impacted  greatly  for  the  faculty  member,  but  institutional  research  could 
have  a field  day.  Data  could  be  tracked  on  the  relationship  of  access  to  syllabi  to  course  enrollment  and  course 
evaluations.  This  is  the  kind  of  data  that  can  help  justify  technical  and  support  positions  as  well  as  capital  upgrades 
to  aging  computing  infrastructure.  In  addition,  the  university  may  wish  to  provide  an  online  listing  of  faculty 
research  interests,  perhaps  thereby  generating  some  research  opportunities,  fostering  collaborations,  or  providing 
assistance  in  research  funding. 

Service 


While  level  1 may  impact  committee  work  slightly  (i.e.  in  terms  of  organizational  charts  and  committee 
memberships  placed  online),  the  greatest  impact  on  service  at  this  level  may  come  to  faculty  who  specialize  in 
technology.  Technology  faculty,  (i.e.  faculty  in  Instructional  Technology,  Information  Science,  or  Computer 
Science),  are  often  asked  to  take  on  the  responsibility  of  creating  and  ma  intaining  programmatic  Web  pages.  The 
danger  in  this  is  that  technology  faculty  are  typically  in  academic  units  and  not  service  units.  Technology  faculty  can 
no  more  be  expected  to  do  other  peoples  Web  pages  for  them  than  the  accounting  faculty  could  be  ejected  to  do 
everybody’s  taxes.  Having  the  expertise  does  not  mean  that  you  have  the  time  or  inclination.  If  all  faculty  units 
would  be  required  to  do  this  as  part  of  their  service  requirements,  then  their  contracts  should  be  evaluated  to  make 
sure  that  proper  consideration  is  made  for  this  work  during  tenure,  promotion,  and  merit  decisions. 

Management 

Making  full  use  of  the  WWW  in  higher  education  will  improve  the  university’s  goodwill  or  public  image. 

At  the  time  of  this  writing,  any  institution  that  makes  it  an  institutional  priority  to  have  every  course  syllabus  online 
could  be  seen  as  a leader  in  technology.  As  it  becomes  more  convenient  for  current  and  prospective  students  to 
locate  course  information  and  make  better-informed  decisions  about  classes  and  degree  programs  the  reputation  of 
the  university  grows.  With  it  grows  enrollment,  tuition  dollars,  and  funding.  But  this  level  may  be  more  difficult  to 
achieve  than  some  think.  For  example,  at  one  large  urban  institution  that  we  know  of,  there  are  1,155  undergraduate 
classes  and  1,105  graduate  classes  listed  in  the  respective  catalogs.  To  put  all  course  syllabi  on  line,  somebody  must 
make  sure  that  all  of  these  syllabi  are  found,  approved,  and  posted,  not  to  mention  updated  as  needed.  This  is  a 
daunting  task  regardless  of  whether  the  responsibility  is  localized  within  a department  or  centralized  by  an  outside 
office.  Even  though  Level  1 is  the  easiest  to  achieve  from  a technical  standpoint,  it  would  still  require  much  effort 
from  a planning  standpoint. 

At  Level  1,  institutions  must  make  critical  and  defining  decisions  about  intellectual  property  and  faculty 
load.  Professors  often  teach  classes  focusing  on  their  research,  so  a particular  course  may  rely  heavily  on  a particular 
person  being  there  to  teach  it.  As  more  information  becomes  public  an  important  question  must  be  answered:  Who 
owns  the  course?  Does  the  university  own  it  because  the  university  employed  Professor  Smith,  or  does  Professor 
Smith  own  it  as  intellectual  property?  Putting  materials  online  is  not  something  most  people  have  scheduled  time  for 
in  their  careers.  If  a faculty  member  makes  the  time,  should  they  be  compensated  for  doing  this?  If  it  is  required  of 
faculty  to  put  course  materials  on  line,  should  they  be  given  reduced  teaching  loads?  Should  they  be  paid  more 
money?  Should  a faculty  member  be  paid  royalties  if  the  university  continues  to  use  the  material  after  the  professor 
leaves?  These  are  critical  questions  that  will  need  to  be  addressed. 

Level  2:  Supplemental  Web  Use 

Supplemental  Web  use  appears  to  make  its  greatest  impact  on  actual  classroom  issues  today.  These 
classroom  issues  in  turn  impact  the  educational  system  in  a variety  of  ways. 

Teaching 

The  most  common  use  of  supplemental  Web  use  is  putting  course  notes  and  handouts  on  line.  At  this  level 
the  course  schedule,  which  we  distinguish  here  from  the  syllabus,  will  go  online  as  well.  One  advantage  of  putting 
the  schedule  on  line  is  that  courses  may  change  with  the  needs  of  the  students.  Course  scheduling  changes  can  be 
made  and  posted  on  a weekly  basis.  This  will  require  more  planning  on  the  part  of  the  instructor,  and  regular  access 
by  the  students. 

Many  faculty  are  taking  course  handouts,  such  as  Power  Point  slides,  and  placing  them  online  as  course 
notes  or  handouts  to  be  reviewed  later.  However,  a surprise  to  some  people  is  that  once  students  get  the  handouts 
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they  stop  having  a reason  to  come  to  class.  Rightly  or  wrongly,  lecture  notes  are  often  seen  as  the  sole  course 
content.  Historically  students  go  to  class  to  get  information  they  will  need  to  pass  tests  on  didactic  content.  If  that 
didactic  content  is  going  to  be  placed  on  line,  then  they  may  indeed  have  few  reasons  to  go  to  class  unless  the  nature 
of  the  class  changes.  So  at  its  most  basic  point.  Supplemental  Web  use,  perhaps  more  than  any,  will  make  the 
greatest  impact  on  what  will  happen  inside  the  physical  classroom. 

As  a professor,  if  I no  longer  give  lectures  on  a daily  basis,  what  do  I do?  If  the  students  no  longer  take 
notes,  what  do  they  do?  Obviously  this  does  not  sound  the  death  knell  for  classroom  instruction  or  even  the  lecture. 
What  it  might  do  is  breath  new  life  into  the  classroom.  Good  teaching  is  not  merely  good  public  speaking.  Good 
teaching  is  about  providing  opportunities  for  students  to  become  engaged  in  the  learning  process  (Jonnassen,  1996). 

Research 

The  biggest  impact  on  research  at  the  supplemental  level  is  in  dissemination.  Researchers  can  post  their 
preliminary  results  online  and  get  immediate  feedback  from  their  peers.  They  can  access  their  peers’  findings  that 
may  affect  their  own  work  without  having  to  wait  months  or  even  years  before  seeing  them  in  a journal.  Indeed,  it  is 
for  these  sorts  of  activities  that  the  Web  as  initially  created.  (Crossman,  1997)  Of  course  this  easy  access  als  o 
removes  the  safety  net  of  the  referring  process  creating  its  own  set  of  problems. 

In  addition,  any  change  in  the  amount  of  time  faculty  spend  teaching  will  have  a direct  impact  on  the 
amount  of  research  and  publishing  they  can  do.  While  certain  faculty  may  be  able  to  use  their  classes  as  research 
settings,  clearly  all  will  not.  Faculty  who  make  the  effort  to  work  on  their  teaching  are  rarely  rewarded  for  it  (Boyer, 
1997).  But  teaching  is  a significant  type  of  scholarship,  and  one  that  should  be  treated  as  such  (Boyer,  1997). 
University  tenure  committees  must  begin  to  recognize  this  and  reward  it  not  only  as  good  teaching,  but  as  good 
scholarship  as  well. 

Service 


If  what  we  know  as  “traditional”  classes  change,  then  we  can  expect  growing  pains  as  this  change  takes 
place.  If  class  formats  begin  to  change,  then  institutions  can  expect  successes  at  some  points,  and  dismal  failures  at 
others.  As  faculty  learn  to  teach  differently,  students  will  need  to  learn  to  learn  differently.  Changes  in  structure  will 
nearly  always  be  met  with  resistance  initially  (Rogers,  1983).  At  the  service  component,  expect  greater  time  spent  in 
grade  appeals.  As  faculty  spend  more  time  working  on  teaching,  expect  appeals  on  tenure  decisions.  The  self- 
governance  function  itself  may  become  more  open  to  scrutiny  as  meeting  agendas  and  minutes  are  published  online 
All  of  this  will  impact  the  amount  of  university  service  done  through  self-governance. 

Management 

In  the  management  component,  this  level  will  ultimately  be  championed  if  for  no  other  reason  than  to  save 
printing  and  copying  costs.  Freeing  up  money  from  departmental  photocopying  alone  may  be  a tremendous  financial 
boon.  However,  the  administration  will  need  to  be  tolerant  of  experimentation.  Teaching  evaluations  may  be  poor 
for  a period  of  time  as  faculty  learn  to  teach  differently,  and  students  strive  to  understand  a shift  in  the  focus  of  the 
classroom  experience.  However,  the  change  may  bring  the  university  greater  recognition  and  ultimately  better 
faculty,  more  students,  and  increased  funding. 

Level  3:  Essential  Web  Use 

As  far  as  teaching  and  learning  goes,  putting  course  information  on  the  Web  at  level  3 makes  two 
significant  assumptions:  (1)  Faculty  and  students  have  appropriate  computing  and  internet  access  (2)  Faculty  and 
students  have  the  requisite  skills  or  support  to  use  the  Web.  At  many  universities,  these  two  assumptions  are  far 
from  being  realities. 

Teaching 

Obviously  if  the  student  needs  Web  access  to  be  a contributing  member  of  the  class  then  the  student  will 
need  Web  skills.  The  question  then  becomes  where  and  when  does  the  student  get  these  skills?  There  have  been 
arguments  made  that  newer  students  may  bring  these  skills  to  campus  with  them.  If  not,  then  the  university  may 
need  to  rethink  the  core  curriculum  for  new  students.  Freshmen  are  taught  how  to  write  and  how  to  read  critically, 
through  “101”  courses.  Perhaps  information  technology  literacy  should  also  become  a “101”  course  for  students 
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who  do  not  possess  these  skills.  At  Level  3,  we  will  very  definitely  encounter  the  issues  of  technology  haves  and 
have  nots  (Trotter,  1996).  But  students  are  not  the  only  ones  who  suffer  from  poorly  developed  computer  skills. 
Many  faculty  do  not  have  adequate  equipment  in  their  offices,  let  alone  the  skills  to  produce  essential  materials. 

Research 

As  with  Level  2,  teaching  should  be  seen  here  as  a type  of  scholarship,  one  that  can  be  measured  against 
particular  criteria  to  ensure  that  the  same  rigor  that  is  applied  to  the  teaching  process  that  is  applied  to  the  referee 
process  (Boyer,  1997).  Faculty  may  begin  to  publish  research  results  solely  in  on-line  refereed  journals.  Promotion 
and  tenure  committees  may  need  to  be  educated  as  to  the  value  of  these  sorts  of  research  outlets,  and  to  help  them 
tell  the  difference  between  an  on-line  journal,  and  simply  a paper  that  has  been  posted  on-line.  While  I might  be  able 
to  publish  on-line  at  will,  a posted  paper  is  quite  different  than  a published  paper. 


Service 


This  level  also  impacts  the  service  components  as  well.  For  with  new  teaching  strategies  there  will  also  be 
new  policies  and  committees.  The  administration  will  need  to  form  new  committees  and  new  mechanisms  for 
dealing  with  information  from  these  committees.  Existing  committees  may  find  more  work  for  themselves  as  well. 
Grade  appeals  are  likely  to  be  common  for  a period  of  time  as  faculty  and  students  work  towards  understanding  new 
strategies.  Additionally,  the  faculty  reward  structure  may  need  to  be  revisited.  The  business  of  putting  education  on 
line  is  more  than  simply  converting  syllabi  and  handouts  to  HTML  formats.  It  will  involve  a comprehensive  look  at 
the  way  that  students  study  and  teachers  teach.  There  will  be  new  excuses  to  deal  with  (the  server  ate  my 
homework?)  and  new  ways  of  meeting  with  students  (such  as  online  office  hours).  The  copy  and  paste  commands, 
along  with  copious  online  term  paper  sites  (i.e..  http://homeworksucks.com)  have  made  plagiarism  much  easier. 
Faculty  will  need  support  and  release  time  to  do  the  job  well.  Students  will  need  time  and  comprehensive 
information  regarding  what  the  changed  expectations  are.  Finding  both  support  and  time  may  be  difficult  at  many 
institutions. 

In  addition,  in  the  professional  service  element,  faculty  may  spend  more  time  working  for  their  professional 
organization  virtually.  Indeed,  annual  conferences  may  begin  to  occur  solely  online,  freeing  up  time  and  travel 
money.  However  as  time  increases  so  does  the  amount  of  work  required.  Faculty  my  find  themselves  stretched 
thinner  and  thinner  as  more  organizations  seek  their  aid  in  online  endeavors. 

Management 

Providing  for  these  assumptions  will  require  significant  attention  from  the  Management  component  of  the 
system.  At  the  Management  component,  this  will  make  its  most  significant  impact  in  the  element  of  faculty  and  staff 
recruitment  and  Finance.  In  order  to  use  the  Web  at  level  3,  the  institution  must  make  a commitment  to  a faculty  and 
staff  that  can  manage  the  technology.  It  is  common  knowledge  that  at  many  institutions  there  are  faculty  who  are  not 
going  to  make  any  changes  in  the  way  that  they  teach.  Some  institutions  are  relying  on  attrition  and  aggressive 
faculty  training  and  support  to  move  their  institution  into  a technology  using  institution.  Moreover,  the  increased 
burden  on  the  network  infrastructure  that  essential  Web  use  entails  may  overwhelm  some  systems.  Management 
will  have  to  work  and  plan  carefully  to  ensure  that  university  computing  resources  can  meet  the  demand. 

Level  4:  Communal  Web  Use 

At  the  most  basic  level.  Level  4 will  need  to  change  the  expectations  of  students  and  faculty.  If  classes  meet 
both  face  to  face  and  on  line,  then  this  may  impact  logistical  operations  under  the  management  component. 

Teaching 

Learning  to  teach  online  is  quite  an  experience.  At  this  level  faculty  need  not  only  help  in  putting 
information  on  line,  but  help  in  managing  the  environment  as  well.  Students  begin  to  generate  some  of  the  course 
content  themselves  requiring  faculty  to  evaluate  new  content  and  students  to  take  responsibility  for  their  own 
learning.  This  level  may  require  the  most  fundamental  shift  in  faculties  and  students  mind-sets. 
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Faculty  may  begin  richer  and  more  frequent  collaborations  with  other  researchers  at  this  level  of  Web  use. 
Distance  becomes  important  only  as  a function  of  time  of  day.  Researchers  can  begin  to  undertake  projects  of  a 
scope  undreamed  of  before.  Consider  for  example  efforts  to  crack  cryptographic  codes  that  use  idle  computing  time 
on  millions  of  desktop  computers  worldwide.  Thanks  to  the  Web  (and  internet)  researchers  can  now  do  for  an 
insignificant  fraction  of  the  cost  and  time  what  before  would  have  taken  supercomputers  working  thousands  of  hours 
and  costing  millions  of  dollars.  Massive  and  convenient  collaboration  opens  the  door  to  more  productive  research  in 
all  fields. 

Combining  research  and  teaching  becomes  possible  for  a number  of  faculty.  Education  faculty  may 
certainly  conduct  research  on  the  benefits  and  limits  of  WBI  (Reeves  & Reeves,  1997),  but  other  faculty, 
particularly  those  in  the  social  sciences,  may  find  research  opportunities  as  well.  Special  educators  may  look  at  the 
benefits  of  WBI  for  students  with  special  needs  (Holzberg,  1996).  Sociologists  may  look  at  the  development  of 
online  relationships.  Information  scientists  may  develop  and  test  new  theories  and  devices,  and  other  scholars  can 
find  outlets  here  as  well.  The  biggest  concern  here  is  not  that  you  can  do  research  with  your  teaching,  but  rather  that 
within  the  existing  system  you  get  credit  for  doing  the  things  that  advance  the  mission  of  the  institution.  Again  we 
point  to  Boyer  (1997)  as  having  much  to  say  on  this  subject. 

Service 


Meetings  may  now  begin  to  take  place  entirely  online.  If  nothing  else,  the  onerous  task  of  scheduling 
meetings  will  be  eased  at  this  level  through  the  use  of  asynchronous  work.  Further,  records  of  meetings  will  be 
highly  detailed  and  accurate. 

Management 

The  management  component  of  the  system  can  benefit  from  the  use  of  the  WWW  at  this  level.  Classes 
typically  scheduled  weekly  in  a room  may  not  need  the  room  every  week.  Depending  on  how  classes  are  scheduled, 
it  may  be  possible  to  teach  two  or  more  classes  in  the  same  room  on  the  same  night  at  the  same  time.  This  could  be  a 
tremendous  boon  for  high  use  classrooms  such  as  computer  labs.  Other  benefits  are  just  as  real  if  less  obvious. 
Georgia  State  University  is  actively  encouraging  faculty  to  hold  classes  online  as  part  of  an  effort  to  reduce  pollution 
from  traffic  in  the  City  of  Atlanta. 

Communal  Web  use  may  also  change  the  way  students  view  schedules  and  classes.  Many  students  don’t 
feel  comfortable  with  distance  education,  but  they  still  live  too  far  from  campus  to  drive  to  every  class  meeting. 
Online  classes  may  mean  less  driving  time  for  students,  which  in  turn  means  greater  convenience  for  non-traditional 
students,  which  in  turn  provides  the  university  with  a reputation  of  supporting  non-traditional  students.  These  can  be 
good  things.  As  fathers  of  young  children,  the  authors  have  appreciated  the  flexibility  of  conducting  classes  online 
from  home.  It  means  that  at  the  very  least  we  can  be  there  to  say  goodnight  after  a long  graduate  class.  Students  also 
appreciate  this  flexibility.  However  most  universities  have  contact  hour  requirements  that  state  (roughly)  that  the 
student  and  teacher  must  be  in  the  same  room  for  a set  number  of  hours  over  the  period  of  the  semester.  Strict 
interpretations  of  existing  contact  hour  requirements  will  make  level  4 use  impossible  and  mean  that  faculty  and 
students  cannot  take  advantage  of  this  level  of  Web  use. 

In  addition,  as  classes  begin  to  move  entirely  online,  the  whole  issue  of  what  comprises  a university  begins 
to  change.  No  longer  is  a university  a place.  Far  from  the  Ivory  Towers  in  isolated  campuses  of  yore,  universities 
will  be  thrust  in  among  all  of  the  other  unwashed  data  on  the  net.  Traditional  rivalries  and  competitors  may  be 
replaced  with  competitors  from  across  the  country  or  around  the  world.  Management  will  have  to  carefully  consider 
market  segments  and  demographics  in  much  the  same  way  broadcasting  networks  do  now. 


Level  5;  Immersive  Web  Use 

This  level  is  steeped  in  the  constructivist  paradigm.  In  it,  faculty,  staff  and  students  work  to  assist  each 
other  as  they  approach  their  respective  tasks  individually.  Socially  constructed  knowledge  exists  here,  but  is 
secondary  to  individual  constructions. 

Teaching 
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At  level  five  students  and  teachers  interact  with  each  other  and  directly  with  the  knowledge  base.  The 
classroom  becomes  a true  learning  community  in  which  knowledge  is  acquired,  created  and  distributed  on  an 
egalitarian  basis.  The  faculty  member  becomes  more  of  a mentor  than  an  instructor  and  students  and  faculty 
contribute  to  the  literature  in  a field  as  well  as  learn  from  it.  Naturally,  this  level  is  best  suited  to  more  advanced 
students  and  courses. 

Though  it  may  seem  counter  intuitive,  the  more  students  a course  has,  the  lower  the  level  of  WBl  that  is 
appropriate.  A fully  immersive  WBl  environment  as  described  in  level  5 requires  significantly  more  preparation 
time  and  classroom  management  time  for  the  instructor  than  a traditional  course.  In  our  experience,  a large  part  of 
this  time  comes  from  interacting  with  individual  students.  We  believe  that  a single  instructor  cannot  manage  more 
than  10  to  20  students  in  this  type  of  environment.  For  every  additional  10  to  20  students  the  instructor  should  have 
some  sort  of  teaching  assistance.  On  the  other  hand,  for  courses  with  hundreds  of  students  it  makes  more  sense  to 
distribute  as  much  information  as  possible  on-line.  Therefore,  for  large  numbers  of  students  levels  1,2,  and  3 are 
indicated.  But  levels  four  and  five,  in  which  more  interaction  is  required,  should  be  reserved  for  smaller  classes. 
This  is  often  times  a struggle  as  you  begin  to  think  about  implementing  Web-based  instruction. 

Research 

Research  becomes  more  integrated  with  teaching  as  students  as  well  as  faculty  begin  publishing  scholarly 
work  online.  Teaching  at  this  level  may  well  follow  the  conservatory  method  or  that  of  a hard  science  research 
group,  whose  learning  comes  in  the  context  of  carrying  out  a research  agenda. 

Service 


Service  at  this  level  is  conducted  entirely  online.  Virtual  meetings  and  telecommuting  are  the  rule  rather 
than  the  exception.  Virtual  "town  meetings"  may  take  the  place  of  some  committee  work  where  the  entire  university 
community  participates  in  discussion  of  and  action  on  an  issue.  There  is  a danger  here  of  grid  locked  chaos,  but  the 
example  of  the  governance  of  the  internet  itself  suggests  that  immersive  service  can  work  quite  well. 

Management 

The  educational  system  may  change  strikingly  at  Level  5.  There  are  organizations,  and  entire  institutions 
emerging  that  do  nothing  but  Immersive  Web  use  (The  Western  Governors  University,  Nova,  The  University  of 
Phoenix).  In  this  case,  the  system  itself  will  evolve  around  the  particular  needs  and  particulars  of  the  university. 
However,  if  a traditional  university  begins  to  do  this  type  of  instruction,  then  many  elements  will  be  impacted. 
Course  evaluations  will  need  to  be  reconsidered  to  deal  with  a student  body  at  a distance.  Office  hour  requirements 
may  need  to  be  rethought  to  capture  the  nature  of  a new  kind  of  student  interaction. 

This  decision  will  impact  the  management  of  the  institution  significantly.  Administrators  who  are  pressed 
for  classroom  space  and  the  need  to  generate  more  tuition  related  dollars  often  encourage  faculty  to  take  more 
students  in  on-line  environments.  Administrators  need  to  understand  the  nature  of  the  environment  so  that  they 
might  help  make  better  policy. 

Conclusion 

Too  often  in  the  history  of  educational  technologies  has  innovation  been  carried  on  in  a piecemeal  and 
haphazard  fashion.  Many  creative  ideas  have  been  postponed  or  abandoned  altogether,  not  because  they  were  bad  or 
unworkable,  but  because  they  were  implemented  in  ways  that  at  best  made  success  questionable,  and  at  worst 
guaranteed  failure.  Web-based  education  will  not  go  away.  It  has  gained  too  much  momentum;  it  has  reached 
Rogers’  (1983)  critical  mass.  It  may  founder  in  higher  education  for  quite  some  time  before  it  is  embraced  fully  and 
used  effectively  as  a teaching  and  learning  tool.  Or  it  may,  on  the  other  hand,  revolutionize  the  business  of  higher 
education.  It  may  enable  faculty  and  students  to  embrace  new  methodologies  of  learning  and  instruction  and  ma  ke 
possible  rapid,  efficient,  and  effective  learning  environments  of  a sort  never  seen  before. 

It  will  probably  do  both.  Some  institutions  will  be  incredibly  successful  at  adopting  Web-based  instruction. 
Some  institutions  will  be  dismal  failures.  We  believe  the  difference  between  them  will  be  the  manner  in  which  they 
go  about  that  adoption.  Those  organizations  that  undertake  the  adoption  of  Web-based  instruction  in  a systematic 
and  systemic  fashion  stand  a good  chance  of  becoming  the  premier  institutions  of  the  next  few  decades.  They  will 
consider  the  impact  of  Web-based  instruction  in  all  areas.  They  will  acknowledge  that  Web-based  instruction  means 
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different  things  to  different  people,  that  there  are  levels  of  WBI.  They  will  probably  not  move  as  a whole  from  one 
level  in  our  continuum  to  the  next.  Instead,  different  parts  of  an  institution  will  be  at  different  levels  of  the 
continuum  at  different  times.  It  is  likely  that  some  parts  of  an  institution  will  move  down  levels  rather  than  up 
levels.  This  could  occur  as  they  experience  disenchantment  with  the  Web  or  technical  frustration.  Successful 
institutions  will  on  average,  we  believe,  continue  to  move  up  the  continuum  toward  immersive  Web  use.  Higher 
education  in  this  country  will  eventually  take  place  predominantly  on  the  Web.  It  may  take  some  time,  but  we 
academics  are  not  easily  discouraged.  Instructors  who  wish  to  teach  otherwise  may  one  day  have  to  do  it  in  secret. 
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ABSTRACT 

Based  upon  the  need  for  assessment  of  teachers,  both  pre-service  and  in-service  we  are  in  the  processes  of 
modifying  all  education  courses  to  include  reference  to  and  use  of  Danielson  Framework  linked  to  current 
structure,  which  used  INTASC  Principles.  Adaptation  of  our  standards  for  assessment  to  better  address  the 
assessment  parameters  of  Danielson 's  Framework  is  being  done  through  a visual  presentation.  By  using 
Inspiration  to  design  the  flow  chart,  and  construction  of  a WEB-Folio  template  we  are  better  able  to  show 
how  it  may  serve  as  the  assessment  tool.  Our  timeline  began  with  inclusion  of  INTASC  Principles  during 
our  NCATE  review  and  has  continued  to  be  a primary  tool  for  student  teacher  observation.  As  we  adapt  to 
the  use  of  the  Framework,  in  preparation  for  utilization  in  Fall  2001,  courses  are  being  modified,  students 
being  acquainted  with  the  domains,  and  a phase  in  plan  being  developed  that  allows  for  adjustment  and 
absorption  by  both  faculty  and  students.  This  presentation  will  deal  with  our  adaptation,  the  basic 
frameworks  domains,  and  how  we  designed  the  final  Web-Folio  to  work  for  us  in  this  assessment. 

Background  of  the  Study 

Throughout  the  United  States  today,  many  teacher  education  programs  and  school  systems  are  addressing 
the  growing  need  of  ways  for  assessing  pre -service  teachers  academic  performance.  Many  colleges  have  adopted 
some  form  of  statewide  and  or  national  educational  standards  for  assessment.  The  Commission,  National 
Commission  on  Teaching  and  Americans  Future  (NCTAF)  prefers  to  use  the  word  because  the  word  test 

evokes  an  image  of  simple  paper-and -pencil  examinations.  The  word  assessment  encompasses  paper-and-pencil 
tests,  portfolios  containing  examples  of  work,  videotapes,  and  observations.”  (Spring,  29-30)  For  the  past  six  years, 
Columbus  State  University  has  utilized  many  forms  of  assessment  models  to  measure  academic  performance, 
teaching  practice  and  the  teaching  knowledge  of  pre  -service  teachers.  These  assessment  models  include:  (1). 
Interstate  New  Teacher  Assessment  and  Support  Consortium  (INTASC)  and  (2)  the  Educational  Testing  Service’s 
PRAXIS  1 & 11  exams.  Following  this  a framework  was  "developed  by  Charlotte  Danielson,  who  worked  at  the 
Educational  Testing  Serv  ice  on  both  the  PRAXIS  III  and  national  board  standards  and  assessments.  Her  framework 
comprises  22  different  teaching  components  organized  into  four  teaching  domains,  and  thus  provides  a rich, 
comprehensive  description  of  good  teaching  practice.  In  addition,  the  ETS  has  developed  a set  of  instruments  that 
districts  can  use  to  assess  an  individual  teacher's  practice  to  four  different  levels  of  performance.” 
http://www.edweek.0rg/ew/l  999/41  odden.hl  8 

One  of  the  earlier  efforts  was  the  production  of  a template  utilizing  Digital  Chisel  because  of  its  availability 
in  both  Macintosh  and  PC  formats.  With  the  introduction  of  HyperStudio  for  PC,  the  template  was  redesigned  for 
that  software.  Students  were  instructed  in  the  use  of  the  technology  to  modify  these  templates.  The  equally  acquired 
documentation  of  skills  and  knowledge  gained  in  the  program  as  they  met  the  INTASC  Principles  and  state 
standards.  As  a continued  effort  in  this  process,  the  College  of  Education  at  Columbus  State  University  conducted 
some  research  to  identify  the  most  valid,  reliable  and  salient  research  findings  on  assessment  models  for  measuring 
student’s  academic  performance.  Realizing  that  tangible  products  representing  the  student  work  would  be  an  asset 
in  evaluating  the  mastery  of  professional  skills  in  teaching,  the  use  of  portfolios  was  investigated  and  determined  to 
be  one  means  of  better  incorporating  examples  into  the  process. 

The  term  portfolio  was  not  new,  and  certainly  is  a common  form  of  assessment.  The  major  distinction 
would  be  the  nature  of  the  portfolio.  As  part  of  this  ongoing  assessment  development,  research  was  conducted  on 
the  differences  and  similarities  of  paper,  electronic,  and  web-based.  From  a basic  understanding,  it  became  quite 
evident  that  the  advantage  of  not  having  to  carry  boxes  of  print  materials,  store  disks,  and  maintain  files,  that  web- 
based  is  the  most  attractive  alternative.  This  research  will  be  implemented  in  both  revised  pre -service  and  in-service 
program. 

We  are  told,  "Assessment  is  education's  new  apple  pie  issue.  Everyone  supports  efforts  to  improve 
education;  and  everyone  seems  to  believe  more  assessment  will  help  improve  education.  Green  & Smyser  (1996) 
believe  that  "it's  Just  grand  that  many  people  in  so  many  elected  and  administrative  offices  support  assessment.” 
They  went  further  to  discuss  the  complexity  of  assessment.  They  emphasized  that  "there  is,  however,  one  little 
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problem:  getting  all  these  individuals  to  agree  on  how  and  what  to  assess  and  how  to  use  the  data.  They  all  agree 
about  the  need  for  more  assessment,  Unfortunately,  the  devil  is  in  the  details."  (Green,  K.  p 62) 

Implementation  process  of  MAP  at  Columbus  State  University 

The  structure  of  the  ten  INTASC  Principles  served  as  a well-rounded  guide  to  the  development  of 
professional  educators,  but  did  not  provide  distinctive  components  upon  which  to  assess  mastery  of  the  skills  of 
teaching.  Thus,  another  barrage  of  search  was  undertaken  to  identify  a tool  to  link  our  existing  evaluation  of 
INTASC  Principles  to  deeper,  more  easily  measurable  elements  of  the  pedagogical  process.  As  a result  of  the 
research  findings  a group  of  faculty  members  were  sent  to  Vermont  to  attend  a workshop  on  Danielson  Framework. 
These  colleagues  became  known  as  the  “Vermont  Five”.  Following  the  guidelines  in  the  framework,  they  stressed 
the  “view  of  the  characteristics  that  underlie  all  effective  teaching  is  the  “Framework  for  Teaching,”  developed  as 
part  of  the  Praxis  Series:  Professional  Assessments  for  Beginning  Teachers.  According  to  the  Praxis  framework,  a 
teacher  must  be  proficient  in  four  domains:  planning  and  preparation,  structuring  classroom  environment, 
instruction,  and  professional  responsibilities.”  (Parkay,  357)  Four  months  later,  all  faculty  members  in  the 
Department  of  Curriculum  Education  and  student  teachers/field  experience  supervisors  attended  a two -day 
workshop  on  Danielson’s  Framework.  The  Educator  Preparation  Faculty  also  in  2000-2001  made  some  adaptations 
to  Danielson’s  Framework  to  reflect  Columbus  State  University,  College  of  Education  (COE)  Conceptual 
Framework.  The  adaptation,  under  the  leadership  of  Dr.  Virginia  Causey,  was  dubbed  the  Model  of  Appropriate 
Practice  (MAP).  Thus  the  MAP  puts  into  practice  the  principles  of  the  COE  Conceptual  Framework. 

Orientation  of  faculty  to  Danielson  Framework 

Following  the  adaptation  of  the  Framework  to  become  the  Columbus  State  University,  College  of 
Education  Model  of  Appropriate  Practice  (MAP),  an  orientation  was  planned.  This  consisted  of  an  overview,  two- 
day  workshop,  and  evaluation  workshop  for  use  of  appropriate  software.  The  software,  Pathwise,  provided  a 
method  to  help  evaluate  student  teachers  and  teachers  in  the  field  “within  these  domains  while  taking  into  account 
individual,  developmental,  and  cultural  differences  among  students  and  differences  among  subjects.”  (Parkay,  357) 
Then  MAP  was  addressed  again  as  part  of  the  back  to  school  Teacher  Planning  Week  in  August  2001 . The  faculty 
also  recommended  that  the  MAP  should  be  infused  in  all  pre-service  teacher  education  courses  starting  with  EDUF 
2215  - Introduction  of  Education  - The  American  Educational  Experience.  As  MAP  began  its  implementation, 
faculty  members  were  assigned  the  task  to  develop  an  understanding  of  Danielson's  Framework,  its  relationship  to 
INTASC,  and  a way  that  these  might  be  evaluated  within  a web  based  tool  for  assessment. 

Development  of  instructional  model  and  student  model 

The  need  to  provide  a presentation,  for  the  purpose  of  instruction  in  the  WEB-Folio,  was  indicated  by  the 
faculty  members.  The  decision  to  use  PowerPoint  for  initial  presentation  and  conversion  to  html  (web -based 
materials)  was  made  based  upon  (1)  availability,  (2)  familiarity  to  students  (3)  adaptability.  We  did  research  on 
available  software  according  to  cost,  mastery  of  skills,  and  general  availability.  Having  used  Digital  Chisel, 
HyperStudio,  PowerPoint,  Macromedia,  Astound,  it  was  our  decision  that  the  best,  most  time  and  cost  efficient 
approach  would  be  to  keep  it  simple  (KISS).  This  way,  as  our  students'  access  and  skills  improved,  it  would  be  easy 
to  upgrade  the  folios,  and  of  course,  any  student  will  be  free  to  use  WEB-CT,  FrontPage,  etc.  as  they  so  choose.  We 
only  require  that  they  (1)  provide  us  access,  and  (2)  address  the  Domain  elements  as  proposed  in  the  Guide  and 
MAP  publications.  Once  the  template  design  had  been  developed,  it  was  easy  to  see  that  taking  the  student 
template  and  filling  in  the  blanks  was  the  easiest  method  of  providing  for  instructional  needs.  As  the  utilization  of 
standards  is  a good  way  to  provide  a framework  for  education,  they  contribute  to  ensuring  accountability  and 
verifiable  results.  Students  could  have  a professor  elaborate  on  each  Domain,  the  specific  expectations  for  collection 
of  “artifacts”  and  methods  of  inclusion,  and  retain  a copy  for  use  as  a guide  that  supported  the  rubric  provided  for 
grading.  The  blank  template,  which  included  a frame  for  each  Domain  and  for  each  subheading  was  designed. 

Upon  completion  it  is  to  be  mounted  to  a web  site  for  downloading,  copied  to  disks,  and  revision.  The  requested 
data  to  be  included  set  the  level  for  the  standard  to  be  met.  Thus,  we  set  our  "High  and  clearly  articulated  standards 
for  what  children  should  know  and  learn  (to)  help  teachers  to  become  better  educators  and  help  students  become 
better  learners."  (Paige,  8). 
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Presentation  to  faculty 

. .teachers  must  be  involved,  The  history  of  efforts  to  bring  innovations  into  education  documents  the 
critical  role  of  teacher  involvement  at  each  step  along  the  way.  If  the  assessment  movement  is  to  be  successful,  it 
must  convince  teachers  that  the  intent  of  assessment  is  to  provide  feedback  to  the  student,  to  parents,  and  to  teachers 
about  student  learning.  Assessment  will  fail  if  it  becomes  a punitive  measure  against  teachers.”  (Green,  K.  p.  63) 
During  workshops  held  as  part  of  the  orientation  to  the  College  of  Education  Model  of  Appropriate  Practices 
(MAP),  it  was  announced  that  Drs.  Kuforiji  and  Riggsby  had  been  given  the  assignment  to  develop  a WEB-Folio  for 
use  in  assessment  of  our  students.  As  fall  semester  began,  the  Teacher  Planning  Week  activities  included  a slot  for 
us  to  share  the  process  and  template  we  had  developed  during  four  months  of  work.  At  this  time,  a sanple 
PowerPoint  presentation  was  shared  that  provides  the  faculty  member  with  a means  to  show  the  students  what  is 
expected  with  samples  for  each  area  of  the  Domains,  and  a template,  which  provides  the  student  with  a master  for 
inclusion  of  work  samples.  These  were  uploaded  to  a Website  and  a password  given  to  them  so  that  they  might  have 
access  to  updates,  new  materials,  and  other  things  we  would  like  to  share  about  this  product. 

Orientation  in  class 

Presentation  of  the  model  initially  takes  place  during  EDUF  2215:  The  American  Educational  Experience 
orientation  session.  This  session  the  students  are  given  an  overview  of  the  nature  of  the  assessment,  based  on 
samples  of  work  (artifacts),  and  the  tool  for  exhibiting  these  (WEB -Folio).  There  is  a demonstration  of  the 
PowerPoint  template,  which  has  the  frames  filled  in  with  sample  ideas  of  what  to  include.  The  specific  areas  that 
are  to  be  completed  at  the  end  of  this  class  are  stressed.  In  EDUF  2215,  the  student  must  complete  a basic  resume, 
with  a frame  for  each  of  the  areas,  such  as  Personal  Information,  Education,  Work  History,  Professional  Activities, 
Community  Service,  Honors,  and  References;  and  fifty  percent  of  Domain  1,  which  deals  with  Planning  and 
Preparation  for  Teaching,  A WEB- Folio  Gu  ide  has  been  prepared  to  present  this  information  in  printed  format.  This 
guide  is  available  on  the  class  web  site  for  download. 

Semester  of  field-testing 

Currently  (Fall  semester  2001)  we  are  engaged  in  the  initial  test  of  our  template  and  instructions.  The 
template,  at  this  point,  is  one  in  PowerPoint.  This  was  selected  as  a commonly  recognized  format  that  would  be  less 
threatening  as  an  initial  format,  as  well  as  one  that  is  easily  converted  to  html.  Sophistication  will  follow  as  the 
students  take  advantage  of  the  program  to  study  higher  technology  skills.  At  this  stage,  we  are  more  concerned  with 
ability  to  understand  the  nature  of  the  assessment  program,  what  one  is  trying  to  assess,  and  how  to  identify 
representative  "artifacts”  to  select  for  inclusion.  The  Guide  to  the  WEB -Folio  is  presented  to  the  students  in  teacher 
preparation,  but  we  find  that  comprehension  of  the  written  word  may  be  a challenge.  A rubric  developed  for  grading 
the  product,  the  time  line  for  development,  and  sample  ideas  for  inclusion  are  also  provided  for  faculty  and  student 
guidance.  Basically,  the  very  nature  of  this  being  a NEW  idea  to  students  makes  it  appear  to  be  a difficult 
assignment.  We  are  finding  that  five  minutes  of  direct  question  and  answer  sends  the  student  off  satisfied  with  the 
knowledge  of  what  is  expected. 

Identification  of  Domains  for  each  course  in  Education 

Each  professor,  whether  full-time  or  part  time  is  encouraged  to  identify  the  parts  of  the  Domains  that  are 
applicable  to  the  course  being  taught.  Hopefully,  program  coordinators  will  help  all  those  individuals  teaching  a 
specific  course  to  come  to  an  agreement  of  the  common  elements  from  the  Framework  (MAP).  The  development  of 
a form  for  determining  the  elements  of  each  of  the  four  domains  was  presented  to  all  professors  within  the  College 
of  Education  for  each  course  being  taught.  Upon  completion  of  these,  the  report  will  be  correlated  for  each  major  in 
order  to  provide  a flow  chart  enabling  the  student  to  know  what  is  expected  for  each  course  as  part  of  the  WEB- 
Folio.  Upon  establishment  of  the  template  on  the  Web,  students  and  faculty  may  freely  begin  constructing  a 
personalized  format  of  the  assessment  tool.  Each  faculty  member  will  be  aware  of  the  point  in  time  when  the  class 
must  include  specified  elements  of  the  Four  Domains  of  CSU  MAP.  The  students  will  have  gradually  become 
comfortable  with  updating  entries  and  adding  new  frames  for  their  ever-expanding  catalog  of  “artifacts”  to  support 
the  criteria  for  evaluation.  The  Faculty  will  be  prepared  to  check  the  elements  relative  to  the  specific  course  and 
notations  of  needs  for  inclusion  or  improvement  before  permitting  the  student  to  continue  in  the  program.  As  the 
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process  is  perfected,  files  will  be  maintained  and  student  assessment  becomes  an  incremental  task  shared  by  all 
faculty  members  in  the  program. 
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Abstract 

The  design  of  instructional  text  is  an  important  consideration  when  attempting  to  design  a successful 
learning  environment.  With  the  rising  involvement  in  online  distance  education,  it  is  becoming  even  more 
important  to  adhere  to  good  principles  of  text  design.  The  purpose  of  this  paper  is  to  provide  a framework 
of  possible  instructional  systems  design  strategies  and  their  associated  tactics  to  use  in  an  online 
environment  based  on  research  studies  conducted  on  traditional  paper-based  materials.  We  argue  that  the 
focus  on  content,  and  particularly  the  focus  on  designing  quality  content  should  not  be  discouraged,  but 
rather,  we  must  consider  it  as  a fundamental  foundation  for  the  development  of  other  online  components. 
The  intellectual  property  of  the  faculty  member,  the  content,  the  ‘"text”  as  it  is  called  in  its  fundamental 
sense  should  serve  as  the  basis  by  which  we  ground  our  communication,  our  understanding,  and  our 
interactions  in  an  online  distance  education  course. 


Introduction 

The  design  of  instructional  text  plays  an  important  role  in  the  field  of  Instructional  Systems.  It  is  through 
text  which  we  receive  many  of  our  day-to-day  messages,  including  simple  things  such  as  menus  and  memos,  to 
more  complex  texts  such  as  textbooks  and  articles.  The  increasing  number  of  distance  education  courses  is  creating 
a new  text  medium  through  which  individuals  can  interact.  Many  of  the  information  learner’s  process  come  through 
the  medium  of  text.  In  order  for  text  to  be  in  existence,  authors  must  put  forth  their  words  of  communication  into  a 
form,  such  as  a book,  article,  memo,  and  now  on-line  courses,  and  the  learner  must  interact  with  it.  It  may  appear 
simple  but  much  is  going  on  between  the  time  the  author  puts  their  content  on  paper  and  the  learner  attends  to  it, 
interacts  with  it,  and  processes  it.  What  is  occurring  relates  to  a learner’s  text  processing  strategies  (Duchastel, 

1982).  The  text  should  not  merely  be  a message  from  the  author,  but  rather  the  message  should  be  encompassed  in  a 
textual  design  which  promotes  learning. 

Duchastel  (1982)  identifies  the  processes  of  attention,  comprehension,  and  retention  as  crucial  to  the  textual 
processing  of  a message.  The  learner  mist  first  be  able  to  pay  attention  to  a piece  of  text  and  sustain  interest.  The 
learner  must  then  be  able  to  understand  what  the  author  is  trying  to  convey,  and  for  ultimate  effectiveness  in  the 
communication  of  a message,  the  learner  will  retain  what  they  have  learned.  If  we  agree  that  these  processes  are 
vital  to  the  communication  of  a textual  message,  then  we  can  begin  to  speculate  about  strategies  and  tactics  that 
authors  can  utilize  to  enhance  the  likelihood  that  each  of  the  above  processes  would  occur. 

Where  the  conundrum  occurs  is  in  the  medium  of  on-line  distance  education  courses.  There  has  been  an 
abundance  of  research  studies  addressing  text  design  variables  in  more  traditional  paper-based  materials,  but  few 
studies  have  investigated  the  use  of  design  strategies  and  tactics  in  an  on-line  distance  education  environment.  In 
fact,  some  researchers  argue  that  the  capacity  for  the  new  technology  to  enhance  learning  quality  is  failing  on-line 
courses  (Duchastel,  1997).  The  only  analyses  of  online  courses  have  produced  results  indicating  authors  are  simply 
creating  electronic  versions  of  their  traditional  print-based  materials  (Dehoney  & Reeves,  1998).  This  paper 
provides  an  alternative  way  of  viewing  this  discouraging  environment  of  course  quality  and  use  of  design  strategies. 
We  argue  that  creating  an  on-line  course  must  begin  with  a fundamental  examination  of  the  “text”.  Regardless  of  the 
type  of  interaction  strategies,  communication  environments,  or  activities  planned  in  an  on-line  course,  the  content 
(which  can  be  referred  to  as  the  text  or  the  intellectual  property  of  the  faculty  member  who  authored  it)  provides  the 
essential  foundation  from  which  the  interaction,  communication,  and  collaboration  activities  can  occur.  The  purpose 
of  this  paper  is  to  provide  a framework  of  possible  instructional  systems  design  strategies  and  their  associated  tactics 
based  on  research  studies  conducted  on  traditional  paper-based  materials.  It  is  not  the  purpose  to  provide  an 
exhaustive  review  of  all  possible  design  tactics  that  can  be  utilized  in  an  on-line  environment,  but  rather  to  stimulate 
interest  in  formulating  basic  research  in  the  design  of  content  in  on-line  environments. 
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In  order  to  elicit  optimal  text  processing  as  discussed  by  Duchastel  (1982)  above,  he  has  grouped  the 
various  design  tactics  into  three  broad  strategies:  labeling,  highlighting,  and  illustrating.  Labeling  strategies  identify 
and  summarize  the  facts,  concepts,  rules,  principles,  and  procedures  in  the  text.  Highlighting  strategies  help  the 
learner  to  focus  on  the  material.  The  third  strategy,  illustrating,  assist  the  learner  with  the  functions  of  attention, 
comprehension,  and  retention.  Each  of  these  strategies  and  their  associated  tactics  can  help  the  author  structure  and 
organize  content  in  such  a way  as  to  promote  the  acquisition,  integration,  comprehension,  and  application  of  the 
message  being  communicated.  The  next  section  will  provide  a sampling  of  possible  tactics  that  can  be  utilized  under 
each  of  the  strategies  identified  by  Duchastel  (1982). 

Instructional  Strategies 

This  section  will  describe  the  three  instructional  strategies  of  labeling,  highlighting,  and  illustrating.  Within 
each  of  these  strategies,  specific  tactics  that  would  be  most  easily  implemented  in  an  on-line  distance  education 
medium  will  be  identified.  Although  research  in  the  area  of  instructional  systems  has  primarily  focused  on  the 
design  of  text -based  materials,  the  increasing  rise  of  on-line  distance  education  courses  provides  an  impetus  for 
identifying  which  text -based  variables  could  serve  to  enhance  the  quality  of  on-line  materials.  Once  particular 
variables  have  been  identified,  research  studies  need  to  be  conducted  to  determine  the  effectiveness  and  efficiency  of 
these  text -based  variables  in  an  on-line  medium. 

Labeling 

The  major  design  variables  that  can  be  utilized  under  the  general  category  of  labeling  include  headings, 
marginal  notes,  and  embedded  glossaries.  The  purpose  of  these  tactics  is  to  identify  and  summarize  the  content 
provided  by  the  author.  The  content  can  be  in  the  form  of  facts,  concepts,  rules,  principles,  and/or  procedures.  These 
tactics  can  serve  to  enhance  learner  attention,  comprehension,  and  retention  of  the  material  at  the  lowest  level  of 
processing.  These  instructional  tactics  also  tend  to  be  fairly  efficient.  Labeling  is  often  associated  with  the  strategy 
of  signaling  which  is  defined  as  orienting  the  learner  to  the  text  through  the  use  of  various  textual  display  tactics. 

The  purpose  of  signaling/labeling  is  to  help  the  learner  identify  the  “structure,  type,  or  function  of  content” 

(Jonassen  & Kirschner,  1982,  p.  124).  Its  purpose  is  to  orient  the  learner  to  the  text  document  and  allow  them  to 
organize  their  attention  and  retrieval  strategies  to  improve  performance.  In  the  following  paragraphs,  the  use  of 
headings,  marginal  notes,  and  embedded  glossaries  will  be  discussed. 

Headings  can  be  defined  as  a typographically  and  spatially  distinct  piece  of  text  signifying  the  theme  or 
content  of  the  section  following  it.  Headings  have  been  shown  to  increase  the  general  improvement  of  text  memory 
(Lorch  & Lorch,  1985).  Even  when  a variety  of  different  measures  are  used,  such  as  cued  recall  summarization,  or 
free  recall,  headings  show  improvement  for  memory  (Dee-Lucas  & DiVesta,  1980;  Hartley  & Trueman,  1985; 
Holley,  Dansereau,  Evans,  Collins,  Brooks,  & Larson,  1981).  Headings  are  a relatively  efficient  way  to  incorporate 
good  instructional  design  in  on-line  courses. 

The  second  labeling  tactic,  marginal  notes,  has  received  little  research  attention.  Marginal  notes  are  defined 
as  a column  alongside  the  content  which  both  outlines  the  content  and  acts  as  a running  summary.  Duchastel  and 
Chen  (1980)  also  find  marginal  notes  act  as  an  “access  structure”  allowing  the  learner  to  identify  the  appropriate 
details  from  the  overall  text.  Although  they  did  not  conduct  an  experimental  student  regarding  marginal  notes, 
Duchastel  and  Chen  (1980)  surveyed  sixteen  students  and  found  that  fifteen  of  them  agreed  that  marginal  notes  were 
useful.  Marginal  notes,  if  used  appropriately,  can  be  an  excellent  addition  to  an  on-line  course.  The  only  dilemma  to 
is  the  necessity  to  avoid  the  use  of  frames  in  creating  the  marginal  note  column.  Web  sites  using  frames  create 
extreme  difficulties  for  individuals  with  disabilities  who  use  screen  readers. 

The  third  labeling  tactic  is  the  use  of  an  embedded  glossary.  Acting  as  a terminology  marker,  an  embedded 
glossary  is  defined  in  this  project  as  a tool  which  identifies  difficult  words  and  defines  them.  This  tactic  is 
fundamentally  different  from  marginal  notes,  though,  because  it  does  not  attempt  to  summarize  any  of  the 
information  learned  up  to  a given  point.  It  merely  identifies  a present  word  on  the  page  and  defines  it  so  the  learner 
is  less  involved  in  the  terminology  and  more  involved  with  the  content.  Although  research  has  not  investigated  the 
effectiveness  of  embedded  glossaries,  the  use  of  an  embedded  glossary  is  a relatively  easy  tactic  to  implement  in  an 
on-line  course.  The  author  of  the  course  can  simply  identify  difficult  words  in  the  text  and  hyperlink  them  to  a page 
containing  the  definition. 

Table  1 provides  a brief  overview  of  each  of  these  tactics  and  additional  ones,  their  purposes,  and  research 
literature  that  has  examined  them. 
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Table  L Labeling  Strate^  Tactics 


Tactic  Purpose  Research  Literature 

Headings 

■ Identify  main  points 

■ Enhance  organization 

■ Increase  retrieval 

■ Provide  structure 

■ Doctorow,  Wittrock,  & Marks,  1978 

■ Dee-Lucas  & DiVesta,  1980 

■ Hartley,  Kenely,  Owen,  & Trueman,  1980 

■ Holley,  Dansereau,  Evans,  Collins,  Brooks,  & 
Larson,  1981 

■ Lorch&  Lorch,  1985,  1996 

■ Hartley  <Sc  Trueman,  1985 

■ Wilhite,  1989 

Marginal 

Notes 

■ Identify  and  summarize  main  points 

■ Focus  attention 

■ Increase  retrieval  and  provide  structure 

■ Duchastel  & Chen,  1980 

Embedded 

Glossary 

■ Identify  and  define  key  concepts 

■ Clarify  text  and  enhance  retrieval 

■ No  literature  found  specifically  addressing 
embedded  glossaries 

Chunking/ 

Spacing 

Text 

■ Enhance  structure  and  organization 

■ Assists  with  organization  and  retrieval 

■ Miller,  1956 

■ Frase  & Schwartz,  1979 

■ Wilson,  Pfister,  & Fleury,  1981 

Highlighting 

The  major  design  tactics  that  can  be  utilized  under  the  instructional  strategy  of  highlighting  include 
typographical  cues,  objectives,  summaries,  questions  and  advanced  organizers.  The  purpose  of  these  techniques  is  to 
highlight  for  the  learner  important  content.  These  techniques  can  also  organize  the  content  for  the  reader,  which  will 
enhance  their  attention,  comprehension,  and  retention  of  the  content  at  a deeper  level  of  processing  than  is  provided 
by  the  labeling  instructional  strategy.  Highlighting  is  related  to  the  strategy  of  controlling  defined  as  the  utilization 
of  textual  display  tactics  to  initiate  different  mental  processing  capabilities.  The  purpose  is  not  only  to  signal  the  text 
but  also  to  control  the  attention  of  the  learner,  and  purposefully  identify  information  that  is  important  and  requires 
the  learner’s  focus.  In  the  following  paragraphs,  the  use  of  typographical  cues,  advanced  organizers,  and  objectives 
will  be  discussed. 

Typographical  cues  are  a very  cost-efficient  method  of  improving  the  structure  of  a text  document. 
Particularly  with  the  use  of  bolding  or  italicizing,  one  can  increase  the  readability  of  the  text  and  increase  the 
likelihood  that  the  learner  will  extract  the  main  points.  Typographical  cues  are  defined  as  a highlighted  text  word  or 
fragment  in  the  body  of  the  text  which  serves  to  draw  a learner’s  attention  to  the  content  surrounding  it. 
Typographical  cues  have  been  studied  extensively,  and  many  studies  have  found  the  use  of  them  to  improve 
memory  for  the  text  word  or  fragment  it  is  highlighting  (Cashen  & Leicht,  1970;  Glynn  & Divesta,  1979).  One 
unresolved  matter  in  experimental  studies  on  typographical  cues  is  the  effect  on  the  content  which  is  not  highlighted. 
Some  have  found  that  memory  for  text  which  was  not  highlighted  to  be  unaffected  (Foster,  1979)  while  others  found 
it  to  inhibit  recall  (Glynn  & Di  vesta,  1979).  The  importance  of  distinguishing  among  facts,  concepts,  and 
rules/principles  may  play  a role  in  this  particular  contradiction.  A possible  explanation  is  typographical  cues  may 
need  to  be  used  on  specific  types  of  information.  For  example,  they  may  be  more  appropriate  to  use  on  facts.  Further 
research  can  help  to  clarify  inconsistent  results.  Although  bolding,  italicizing,  underlining,  all  caps,  and  color  can  be 
used  to  create  typographical  cues,  the  most  cost  efficient  in  developing  an  on-line  course  would  be  the  use  of 
bolding  or  italicizing.  Underlining  and  words  in  color  can  often  be  confused  with  hyperlinked  items. 

The  second  major  highlighting  tactic  to  be  discussed  here  is  the  use  of  advanced  organizers.  Advanced 
organizers  are  defined  as  a brief  verbal  statement  providing  a mechanism  for  the  learner  encoding  process.  This  brief 
verbal  statement  contains  no  reference  to  the  content  in  the  document,  but  rather  sets  up  the  organizational  features 
of  the  document  to  aid  the  learner’s  processing.  As  seen  with  some  of  the  other  mentioned  text  variables  above, 
advance  organizers  have  conflicting  findings.  Ausubel  (1960),  the  pioneer  of  advance  organizers,  found  in  all  of  his 
studies  that  those  groups  with  advanced  organizers  recalled  more  than  that  of  the  control  group.  Hartley  and  Davies 
(1976)  after  summarizing  the  results  of  many  studies  on  advanced  organizers  found  that  the  use  of  them  does 
improve  learning.  Particularly  for  an  on-line  course  where  the  learner  is  at  a distance  from  the  instructor,  it  is 
important  to  provide  a clear  understanding  for  the  learner  regarding  the  types  of  strategies  and  tactics  utilized 
throughout  the  document.  For  example,  if  the  author  of  a course  decides  to  utilize  typographical  cues  to  symbolize 
important  concepts  by  bolding  them,  the  student  should  be  made  aware  that  bolded  words  symbolize  concepts  they 
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need  to  make  sure  they  understand.  Advance  organizers  may  even  be  useful  for  stimulating  motivation  because  they 
serve  to  increase  confidence  (Keller,  1987)  in  what  the  learner  is  likely  to  expect  within  the  content. 

Objectives  are  defined  as  statements  of  what  the  learner  should  know  and  be  able  to  answer  questions  about 
in  the  upcoming  sections.  Objectives  are  another  area  of  contradictory  study.  Some  experimental  studies  have  found 
that  learning  is  improved  by  the  use  of  objectives  (Rothkopf  & Kaplan,  1972),  while  other  studies  have  found  the 
opposite  finding  (Jenkins  & Deno,  1971).  One  possible  explanation  for  the  inconsistency  in  research  results  could  be 
that  the  objectives  in  each  study  may  have  been  written  using  incomparable  models.  Most  instructional  designers  use 
Mager’s  (1975)  model  for  writing  behavioral  objectives.  This  model  stipulates  that  objectives  should  include  the 
actual  skill  or  behavior  that  the  learner  should  be  able  to  perform  upon  completion  of  the  instruction,  conditions 
under  which  the  learning  should  occur,  and  the  criteria  for  successful  performance.  The  use  of  objectives  is  a cost 
efficient  and  if  used  appropriate,  effective  strategy  to  utilize  in  an  on-line  course. 

Table  2 provides  a brief  overview  of  each  of  these  tactics  and  additional  ones,  their  purposes , and  research 
literature  that  has  examined  them. 


Table  2.  Highlighting  Strategy  Tactics 


Tactic  Purpose  Research  Literature 

Typographic 

Cues 

■ Focus  attention  on  material 

■ Facilitate  learning  and  retrieval  of  main  points 

■ Enhance  structure  and  organization 

■ Cashen  & Leicht,  1970 

■ Fowler  & Baker,  1 974 

■ Glynn  & Divesta,  1979 

■ Foster,  1979 

■ Hartley,  Kenely,  Owen,  & Trueman, 
1980 

■ Nist&  Hogrebe,  1987 

Advanced 

Organizers 

■ Focus  attention  on  document 

■ Facilitate  learning  and  retrieval  of  main  points 

■ Enhance  structure  and  organization 

■ Provide  learner  a content-oriented  summary  to 
help  learner  focus  on  important  points  as  they 
read  through  document 

■ Ausubel,  1960,  1962,  1963,  1968, 
1978 

■ Ausubel  & Fitzgerald,  1961,  1962 

■ Barnes  & Clawson,  1975 

■ Hartley  & Davies,  1976 

■ Luiten,  A mes,  & Ackerson,  1 976 

■ Mayer,  1979 

■ Hartley  & Trueman,  1982 

■ Lorch&Lorch,  1985,  1996 

Objectives 

■ Focus  attention  on  material 

■ Facilitate  learning  and  retrieval  of  main  points 

■ Enhance  structure  and  organization 

■ Rothkopf  & Kaplan,  1 972 

■ Duchastel  & Merrill,  1973 

■ Duell,  1974 

■ Hartley  & Davies,  1976 

■ Davies,  1976 

■ Duchastel,  1979 

Summaries 

■ Enhance  focus/comprehension  of  material 

■ Newsom  & Gaite,  1971 

■ Hartley  & Bumhill,  1978 

■ Hartley,  Goldie,  & Steen,  1979 

■ Reder  & Anderson,  1980 

Questions 

■ Enhance  deeper  level  of  processing 

■ Rickards  & Denner,  1978 

Illustrating 

The  strategy  of  illustrating  tends  to  incorporate  the  least  cost-efficient  tactics.  Tactics  such  as  graphics  and 
diagrams  tend  to  cost  additional  money  to  develop  but  can  increase  the  level  of  processing  of  particular  content. 
Table  3 provides  a brief  overview  of  the  tactics  of  graphics  and  diagrams/flowcharts/tables,  their  purposes,  as  well 
as  research  literature  that  has  examined  them. 


Table  3.  Illustrating  Strategy  Tactics 


Tactic 

Purpose 

Research  Literature 

Graphics 

■ Provide  additional  assistance  in 

- Dwyer,  1968,  1972 
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processing  content 
■ Offer  a visualization  of  textual 
description 

■ Hurt,  1987 

■ Levie,  1987 

■ Weidenmann,  1989 

Diagrams/ 

■ Offer  a hierarchical  or  process 

■ Bumhill,  Hartley,  & Young,  1976 

Flowcharts/ 

orientation  for  content  through 

■ Richards  & Johnson,  1980 

Tables 

visual  representation 

■ Larkin  & Simon,  1987 

Conclusion 

The  framework  of  strategies  and  tactics  provided  offers  a chance  to  reassess  the  way  we  view  content  in  the 
online  environment.  Some  authors  believe  that  current  and  forward  thinking  should  encourage  a discouragement  of 
the  emphasis  on  content  in  the  design  process  (Oliver,  1999)  in  favor  of  a focus  on  learning  activities  and  learning 
supports.  We  argue  that  the  focus  on  content,  and  particularly  the  focus  on  designing  quality  content  should  not  be 
discouraged,  but  rather,  we  must  consider  it  as  a fundamental  foundation  for  the  development  of  other  online 
components.  The  intellectual  property  of  the  faculty  member,  the  content,  the  “text”  as  it  is  called  in  its  fundamental 
sense  should  serve  as  the  basis  by  which  we  ground  our  communication,  our  understanding,  and  our  interactions  in 
an  online  distance  education  course. 
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Abstract 

Almost  every  state  is  ''imposing''  academic  standards.  Helping  students  to  meet  those  standards  is  a key 
task  for  teachers,  school  administrations,  as  well  as  instructional  systems  designers.  Thus,  instructional 
designers  in  K-I2  environments  are  facing  the  challenge  of  using  appropriately  and  effectively  academic 
standards  in  their  instructional  systems  design  process.  This  presentation  offers  some  effective  ways  of 
handling  academic  standards  in  the  instructional  systems  design  process  together  with  examples.  It  is 
hoped  that  this  presentation  will  encourage  experts  in  both  academic  standards  and  ISD  to  examine  the 
role  and  influence  of  academic  standards  on  ISD  processes  and  the  subsequent  outcomes. 


1.  Introduction 

Many  states  are  “imposing”  academic  standards  which  need  to  be  addressed  in  instructional  systems  design 
(ISD)  process  and  incorporating  academic  standards  effectively  becomes  a crucial  skill  for  instructional  designers  in 
K-12.  This  presentation  examines  effective  ways  of  incorporating  academic  standards  in  ISD  process.  Examples  of 
how  academic  standards  can  guide  and  help  ISD  process  will  also  be  discussed.  Another  goal  of  this  presentation  is 
to  encourage  experts  in  both  academic  standards  and  ISD  to  examine  the  role  and  influence  of  academic  standards 
on  ISD  processes  and  the  subsequent  outcomes. 

2.  Defining  terms 

Academic  standards  in  this  presentation  will  refer  to  the  official  documents  from  the  national,  state 
department  of  education  or  local  educational  commission  that  define  what  each  student  should  know  and  do  in  a 
core  set  of  subjects.  They  are  academic  targets  for  students,  teachers  and  parents  to  meet.  Proposed  Academic 
Standards  for  Science  and  Technology  by  Pennsylvania  Department  of  Education  is  an  example  of  such  state- 
defined  academic  standards. 

Instructional  Systems  is  better  known  as  Educational  Technology  or  Instructional  Technology.  It  is  defined 
as  “the  theory  and  practice  of  design,  development,  utilization,  management  and  evaluation  of  processes  and 
resources  for  learning”  (Seels  & Richey,  1994,  p.l,  the  AECT  Definitions  and  Terminology  Committee,  2000).  Its 
ultimate  purpose  is  to  improve  effectiveness  and  efficiency  of  human  learning.  Instructional  systems  design  (ISD)  in 
this  article  refers  to  the  systematic  process  of  planning  instructional  systems,  specifically  the  process  of  analysis, 
design,  development,  implementation  and  evaluation  of  instructional  programs. 

3.  Relationships  between  academic  standards  and  instructional  system  design 

Academic  standards  are  constructed  to  give  students  a solid  foundation  in  the  basics  and  to  provide 
consistent  targets  for  students,  teachers  and  parents.  They  allow  schools  to  measure  student  achievement.  They  help 
parents,  teachers,  school  administration  and  school  districts  follow  the  progress  that  students  make  from  year  to 
year.  “Done  right,  aligned  standards  and  assessments  give  us  something  that  standards  and  curriculum  objectives,  by 
themselves,  never  delivered:  the  ability  to  see  how  well  we  are  performing  and  how  much  we  are  improving.” 
(Pennsylvania  Department  of  Education,  2001.  online:  http://www.pde.psu.edu/standard/backgmd.htmn 

The  importance  of  academic  standards  has  long  been  recognized  in  education.  Studies  related  to  academic 
standards  range  from  the  characteristics  of  good  academic  standards  to  their  influence  in  facilitating  educational 
reforms  and  students*  achievements.  Setting  up  good  academic  standards  is  important.  However,  how  to  help 
students  to  meet  those  standards  are  the  real  purposes.  This  is  where  the  academic  standards  and  ISD  meet. 
According  to  the  definition  above,  the  ultimate  purpose  of  ISD  is  to  make  the  learning  maximally  effective  and 
efficient.  Therefore,  in  K-12  environments,  the  ultimate  purpose  of  ISD  is  to  help  the  students  achieve  those 
academic  standards  effectively  and  efficiently. 
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4.  Incorporating  academic  standards  in  instructional  systems  design 

All  the  academic  standards  should  be  measurable,  applicable  and  clearly  written.  When  applying  the 
academic  standards  in  instructional  systems  design,  they  might  not  be  so  either  because  of  standards  themselves  or 
the  limits  of  resources  available  to  apply  them.  The  author  summarizes  the  following  ways  to  incorporate  academic 
standards  appropriately  and  effectively  in  ISD.  They  are  to  be  discussed  in  the  four  aspects  (1)  systemic  approach  in 
incorporating  academic  standards,  (2)  academic  standards  as  ISD  guide,  (3)  flexible  use  of  academic  standards  in 
ISD,  and  (4)  moving  beyond  academic  standards. 

4.1.  Systematic  approach  of  incorporating  academic  standards 

The  systemic  approach  in  ISD  refers  to  the  way  in  which  instructional  systems  designer  takes  every  aspect 
of  ISD  into  consideration  in  order  to  produce  effective  instructional  programs.  It  should  be  differentiated  from 
systematical  approach.  The  systematical  approach  is  linear,  step-by-step  approach  while  the  systemic  is  integrated, 
holistic  approach  in  ISD  efforts  (Banathy,  1996). 

A systemic  approach  in  incorporating  academic  standards  in  ISD  process  should  take  following  into 
considerations  and  balance  between  what  should  be  done  and  what  could  be  done: 

• Which  academic  standards  should  be  included  in  the  ISD  programs  that  can  help  the  students 
most? 

• Which  academic  standards  should  be  included  in  the  ISD  programs  that  are  in  alignment  with  the 
project  rationale  and  goals,  and  student  and  school  needs? 

• Which  academic  standards  should  be  included  in  the  ISD  programs  according  to  the  availability  of 
resources  (personnel  and  financial  resources,  equipment  and  facilities  resources,  technical  and 
content  support,  and  time  constraints,  etc.)? 

• How  much  academic  standards  should  be  included  in  the  ISD  programs  according  to  the 
availability  of  resources  (personnel  and  financial  resources,  equipment  and  facilities  resources, 
technical  and  content  support,  and  time  constraints,  etc.)? 

The  systemic  approach  ensures  the  effectiveness  and  efficiency  of  incorporating  academic  standards  in  ISD. 

4.2.  Academic  standards  as  the  ISD  project  guide 

Academic  standards  can  be  used  as  ISD  project  guide.  Instructional  systems  designers  can  refer  to 
academic  standards  to  establish  ISD  project  rationale  and  project  goals.  This  is  especially  true  in  writing  grant 
proposal  for  ISD  project.  In  analyzing  phase  of  ISD,  academic  standards  can  provide  guidance  and  references  on 
specific  learning  objectives  for  the  ISD  program.  This  ensures  that  contents  of  ISD  programs  designed  and 
developed  are  right  on  the  target  by  defining  what  the  students  are  supposed  to  learn  and  help  them  to  leam  well  to 
meet  the  standards.  In  implementing  and  evaluating  of  ISD  programs,  the  academic  standards  may  also  provide 
instructional  systems  designer  with  what  to  assess  or  basis  for  what  to  assess.  Many  academic  standards  can  provide 
yardsticks  for  both  summative  and  formative  evaluation  in  the  ISD  programs.  The  rubrics  or  criteria  for  assessment 
can  be  directly  developed  from  the  academic  standards.  All  these  help  to  increase  the  validity  of  the  assessments 
involved  in  ISD  programs. 

To  use  academic  standards  as  a guide  appropriately,  it  is  always  advisable  for  instructional  systems 
designers  to  consulting  with  subject  matter  experts  and  the  schoolteachers  who  are  going  to  use  the  ISD  programs 
eventually  in  their  classrooms.  It  is  very  important  to  make  sure  that  you  have  reached  a consensus  in  explaining  and 
understanding  the  specific  academic  standards  that  the  ISD  program  is  to  include.  Otherwise,  there  is  no  way  for  the 
teachers  to  use  the  ISD  program  effectively  if  it  conflicted  with  their  beliefs  and  understanding. 

4.3.  Flexible  use  of  academic  standards  in  ISD 

Academic  standards  are  important  to  instructional  systems  designers  and  how  to  incorporate  them 
appropriately  in  ISD  process  is  crucial  for  those  work  in  K-12  environment.  Academic  standards  provide  general 
guidance  in  terms  of  learning  content  for  ISD  programs.  However,  in  most  of  the  cases,  they  cannot  be  directly  used 
as  learning  objective  for  ISD  programs.  A competent  instructional  designer  should  be  able  to  use  academic  standards 
flexibly. 
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There  are  three  effective  ways  of  using  academic  standards  without  comprising  their  quality  and  quantity. 
They  are  (1 ) deriving  learning  objectives  for  a single  ISD  program  unit  from  multiple  standards,  (2)  deriving 
learning  objectives  for  multiple  ISD  program  units  from  single  standard,  and  (3)  deriving  learning  objectives  for 
multiple  ISD  program  units  from  multiple  standards. 

1.  For  a single  ISD  program  unit,  the  learning  objectives  can  be  derived  from  multiple  standards. 
(Please  see  Example  1:  Multiple  standards  for  a single  ISD  program  unit.) 

2.  The  learning  objectives  for  multiple  ISD  program  units  can  be  derived  from  a single  academic 
standard.  (Please  see  Example  2:  A single  standard  for  multiple  ISD  program  units.) 

3.  The  learning  objectives  for  multiple  ISD  program  units  can  be  derived  from  multiple  academic 
standards.  Although  none  of  the  ISD  program  units  fully  covers  a single  academic  standard,  the 
multiple  units  cover  all  the  targeted  academic  standards  and  may  overlap.  (Please  see  Example  3: 
Multiple  standards  for  multiple  ISD  program  units.) 

4.4.  Moving  beyond  academic  standards 

Academic  standards  should  be  the  guide  and  references  for  ISD  rather  than  the  obstacles  and  constraints.  A 
competent  instructional  systems  designer  should  not  only  be  able  to  use  them  flexibly  but  also  be  able  move  beyond 
academic  standards  in  ISD.  This  can  be  done  mainly  in  two  ways.  First,  ISD  should  not  only  aim  at  accomplishing 
the  target  standards  but  also  aim  at  preparing  the  learners  to  achieve  higher  academic  standards.  For  example,  in 
designing  a lesson  of  biology  for  the  5^^  grade  students,  instructional  system  designer  should  not  only  consider  the 
present  learning  target  but  also  prepare  the  students  to  achieve  those  academic  standards  in  their  future  study.  Maybe 
you  could  find  that  some  of  the  academic  standards  for  the  6^^  grade  students  can  be  partially  incorporated  in  your 
design  for  the  5*^  grade. 

Another  way  of  moving  beyond  the  academic  standards  in  ISD  process  is  to  integrate  academic  standards 
of  different  subjects  into  your  ISD  programs  whenever  possible.  For  example,  in  designing  computer-assisted  lesson 
of  biology,  the  academic  standards  of  science  and  technology  can  be  easily  integrated  together.  In  developing  a 
lesson  of  environment  protection,  the  academic  standards  in  writing  can  be  integrated  together.  For  instance,  in 
Example  1:  Multiple  standards  for  a single  ISD  program  unit,  the  learning  objectives  for  ISD  program  are 
derived  from  academic  standards  for  4*^  grade  students.  The  academic  standards  are  from  different  subjects  of  study 
(Environment  and  Ecology,  and  writing)  and  for  different  grades  (grade  4 and  grade  5).  The  integration  of  academic 
standards  from  different  areas  of  learning  would  foster  the  learners’  development  in  all  fields  of  studies. 

5.  Conclusion 

For  instructional  systems  designers  in  K-12  environments,  academic  standards  mean  a lot.  “The  program 
rationale  must  be  based  on  the  academic  standards,  otherwise,  we  will  never  get  the  grant.”  “We  have  to  cover  these 
standards  as  required  by  the  grant.”  “Derive  program  objectives  according  to  the  academic  standards  and  also 
remember  to  assess  the  learners  according  to  them.  We  need  a report  on  that.”  Like  it  or  not,  these  are  common 
statements  and  actual  practices  of  ISD  in  the  field  of  K-12.  The  practical  ways  mentioned  above  can  help 
instructional  systems  designers  in  K-12  incorporate  academic  standards  appropriately  and  effectively  in  ISD  process 
and  help  more  students  meet  their  academic  standards. 

Also,  it  is  hoped  that  this  article  would  bring  more  experts  both  in  the  fields  of  academic  standards  and 
instructional  systems  design  to  study  the  effective  and  efficient  ways  of  incorporating  academic  standards  in  ISD 
processes  and  their  subsequent  outcomes.  It  is  obvious  that  the  efforts  of  study  academic  standards  in  ISD  process 
will  benefit  a lot  both  students  and  teachers  in  K-12  education. 

Example  1:  Multiple  standards  for  a single  ISD  program  unit 


Multiple  Academic  Standards 


Learning  Objectives  for  a ISD  Program  Unit 
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Academic  Standards  for  Environment  and  Ecology 
(Pennsylvania  Department  of  Education,  2001) 

4.9.4  Grade  4 

A.  Know  that  there  are  laws  and  regulations  for  the 
environment 

• Identify  local  and  state  laws  and  regulations 
regarding  the  environment 

• Explaining  how  the  recycling  law  impacts  the 
school  and  home 

• Identify  and  describe  the  role  of  a local  or 
state  agency  that  deals  with  environmental 
laws  and  regulations 


Academic  Standards  for  Environment  and  Ecology 
(Pennsylvania  Department  of  Education,  2001) 

4.2.4  Grade  4 

/).  Identify  by  products  and  their  use  of  natural 
resources 

• Identify  those  items  that  can  be  recycled  and 
those  that  cannot. 


Academic  Standards  for  Reading,  Writing, 
Speaking,  and  Listening 

1.4.5  Grade  5 

Writing  multi-paragraph  informational  pieces  (e,g, 
essays,  descriptions,  letters,  reports,  instructions,) 

• Include  cause  and  effect. 

• Develop  a problem  and  solution  when 
appropriate  to  the  topic 

• Use  relevant  graphics  (e.g.  maps,  charts, 
graphs,  tables,  illustrations.  Photographs). 


Lesson  Objectives  derived  from  the  academic 

standards 

1.  Given  examples  of  environmental  laws  and 

regulations,  the  student  will  be  able  to  categorize 
them  correctly  (85%)  into  three  main  categories 
(Air,  Water,  and  Pesticides  and  other 

pollutants). 

2.  Student  will  be  able  to  discriminate  correctly 
(95%)  between  recyclable  and  non-recyclable 
items  in  their  daily  life  both  at  school  and  home. 

3.  The  student  will  be  able  to  describe  in  writing 

the  following  roles  of  a local  or  state 
environmental  agency  that  deals  with 

environmental  laws  and  regulations. 

a.  Monitoring  environment 

b.  Environmental  law  enforcing 

c.  Gathering  and  sharing  information  on 
environment 


Example  2:  Single  academic  standard  for  multiple  IDS  program  units 


Single  Academic  Standard 

Learning  Objectives  for  Multiple  ISD  Program 
Units 

Academic  Standards  for  Sciences  and  Technology 
(Pennsylvania  Department  of  Education,  2001) 

3.3.4  Grade  4 

A,  Know  th e similarities  and  differences  of 
living  things 

• Identify  life  processes  of  living  things  (e.g. 
growth,  digestion,  react  to  environment). 

• Know  that  some  organism  have  similar 
external  characteristics  (e.g.  anatomical 
characteristics:  appendages,  types  of 
covering,  body  segments)  and  that  similarities 
and  differences  are  related  to  environmental 
habitat. 

• Describe  basic  needs  of  plants  and  animals. 

Objectives  for  Unit  One 

I . Given  an  example  of  living  animal  or  a plant 
that  student  are  familiar  with,  students  will  be 
able  to  clearly  describe  its  life  process  by  using 
concepts  of  growth,  digestion,  and  its  external 
characteristics. 

Objectives  for  Unit  two 

2.  Students  will  be  able  to  describe  the  basic  needs 
of  plants  and  animals  and  explain  how  living 
things  are  dependent  on  other  things — both 
living  and  nonliving-in  the  environment  for 
survival  by  using  examples  that  they  are  familiar 
with 

Example  3:  Multiple  academic  standards  for  multiple  IDS  program  units 


Multiple  Academic  Standards 

Learning  Objectives  for  Multiple 
ISD  Program  Units 

Academic  Standards  for  Sciences  and 

Objectives  for  Unit  1 

Technology  (Pennsylvania  Department  of 

Given  the  category  type  of  search  engine,  the  students 

Education,  2001) 

will  be  able  to  locate  information  needed  to  solve  the 

3.7.4.  Grade  4 

given  problems  that  are  appropriate  to  their  age  by 

D.  Use  basic  computer  software. 

• Forming  up  appropriate  questions 

• Applying  operating  system  skills  to 

• Identifying  the  key  words  to  location 

perform  basic  computer  tasks. 

information  needed 

• Apply  basic  word  processing  skills. 

• Being  selective  (critical)  towards  information 

• Identify  and  use  simple  graphic  and 

\ 

found  online 

presentation  graphic  materials 

• Using  the  information  founded  to  solve  given 

generated  by  computer. 

problem 

• Apply  specific  instructional  software 

• Using  email  format  to  communicate  solutions 

Academic  Standards  for  Sciences  and 

Objectives  for  Unit  2 

Technology  (Pennsylvania  Department  of 

7.  Given  student  a business  web  site,  the  students  will 

Education,  2001) 

be  able  to  identify  correctly  the  basic  components 

3.7.4.  Grade  4 

web  site  including 

E,  Identify  basic  computer  communication 

y 

• URL 

systems. 

• Site  title  and  site  content  (Navigation  bars, 

• Apply  a web  browser. 

contact  information) 

• Apply  basic  electronic  mail 

• Description  of  the  company  (Name,  address. 

functions. 

company  logo  and  contact  information) 

• Use  online  searches  to  answer  age 

• Products  and  services 

appropriate  questions. 

• Date  of  site  information  updated 

2.  Students  will  be  able  to  use  graphic  designing 

principles  to  evaluate  the  site  design  in  terms  of 

• Easy  to  read 

• Layout  of  website 

• Consistency  in  graphic  presentation 

Objectives  for  Unit  3 

Given  a web  design  template,  student  will  be  able  to 
design  a business  site  for  advertising  company 
products  and  services  by  following  graphic  principles 
in  unit  2. 
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Using  Developmental  Research  to  Study  One’s  Teaching  of  an  Instructional  Design  Course 
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Abstract 

This  five-year  study  of  two  instructors  teaching  a master's  level  instructional  design  course  used 
developmental  research  to  systematically  examine  a reflexive  teaching  approach.  The  reflexive  teaching 
model  is  described.  Eight  data  sources  across  six  deliveries  of  the  course  were  analyzed  in  terms  of  design 
decisions  (i.e.,  the  teaching  model),  model  implementation,  and  model  evaluation.  Methodological  issues  of 
this  developmental  research  study  are  discussed,  including  data  management  of  evolving  data  sources, 
data  analysis  of  teaching  artifacts,  and  the  balancing  of  teaching  goals  and  research  objectives.  Guidelines 
for  using  developmental  research  to  study  one's  teaching  are  summarized. 

Developmental  Research 

We  view  developmental  research  as  co-contributing  to  the  development  of  educational  interventions 
(models  and  other  processes,  courses,  media/technology  artifacts),  as  well  as  knowledge  about  this  development  in 
the  form  of  design  principles  or  frameworks.  Developmental  research  uses  a developmental  cycle,  consisting  of 
design,  development,  implementation,  and  evaluation  activities,  to  formally  study  these  interventions  over  time.  The 
intent  of  developmental  research  is  partly  pragmatic  design,  developing  products  or  processes  that  are  needed  to 
serve  human  needs  (Norman,  1993).  The  intent  is  also  to  learn  from  our  design  efforts,  to  formulate  what  we  learn 
so  that  these  principles  and  frameworks  assist  us  in  developing  other  interventions  in  different  contexts. 

Reeves  (2000)  reminds  us  that  the  aims  of  any  research  depend  on  the  researcher's  epistemological  lens, 
which  in  turn  influences  the  selection  of  one's  research  goals  and  research  framework.  For  example,  development 
goals  differ  from  empirical  goals.  While  empirical  research  is  characterized  by  a researcher  using  research 
hypotheses  and  the  testing  of  these  hypotheses  to  refine  existing  theory,  developmental  research  involves 
collaborators  in  the  analysis  of  practical  problems  with  the  testing  of  designed  solution  in  actual  practice.  The 
outcome  of  developmental  research  is  initially  a greater  understanding  of  the  educational  artifact,  process,  or 
intervention  and  ideally  design  principles  generalizable  at  some  level,  whether  it  be  to  refine  the  artifact,  process,  or 
intervention  under  study  or  transfer  to  other  applications  or  contexts. 

Richey  and  Nelson  (1996)  provide  a comprehensive  review  of  developmental  research  projects,  which  they 
classified  as  either  Type  I or  Type  II  studies.  Type  I studies  involve  the  design  and  development  of  products  or 
processes  within  a developmental  cycle,  while  in  Type  II  studies  researchers  study  what  has  already  been  developed 
with  the  intent  of  abstracting  design  principles  for  re  -use.  Meanwhile,  van  den  Akker  (1999)  labels  Type  I as 
formative  studies  in  which  the  goal  is  optimizing  an  intervention,  while  Type  II  studies  are  labeled  as  reconstructive 
studies  and  formulate  design  principles.  In  addition,  van  den  Akker  characterizes  design  principles  to  include 
substantive  knowledge  on  "What  to  design?"  and  procedural  knowledge,  or  "How  should  it  be  developed?"  To  add 
to  our  knowledge  of  what  to  design  and  how  to  design,  says  van  den  Akker,  design  principles  should  be  justified  by 
theoretical  arguments,  procedural  detail,  empirical  evidence,  and  validated  in  multiple  contexts. 

Informing  developmental  research  is  formative  research  (e.g.,  Newman,  1990),  which  (a)  identifies 
shortcomings  and  providing  suggestions  to  a product  or  a process  under  development;  (b)  uses  triangulation  of 
methods,  instruments,  sources,  and  settings;  and  (c)  evaluates  quality  criteria,  such  as  effectiveness 
appeal/practicality,  and  efficiency.  Complementing  formative  research  are  design  experiments  (Brown,  1992), 
involving  the  design  and  study  of  learning  environments  addressing  complex  learning  problems  in  actual  learning 
settings  with  practitioners.  In  using  design  experiments  Brown  reminds  us  how  mutually  informative  studies  done 
in  classroom  and  laboratory  settings  can  be.  Decisions  facing  the  researcher  in  design  experiments,  according  to 
Brown,  include  participant  size,  involving  individualized  cases  studying  subject  traits  or  the  use  of  many  subjects 
looking  at  a single  variable.  A second  decision  deals  with  changes  over  a chosen  length  of  time,  including  cross- 
sectional  studies  from  different  groups,  longitudinal  data  from  a group  over  time,  or  microgenetic  studies  examining 
data  over  a short  period  of  time.  A third  decision  is  the  appropriate  choice  or  mix  of  quantitative  and  qualitative 
techniques  to  use. 
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This  paper  discusses  the  methodological  issues  of  using  a developmental  research  framework  to 
systematically  design,  implement,  and  evaluate  a process;  in  this  case,  a reflexive  approach  for  teaching  instructional 
design  (ID).  The  next  section  of  the  paper  describes  the  developmental  study,  including  the  reflexive  teaching 
model,  the  instructional  sequence  of  the  ID  course,  the  developmental  research  framework,  and  study  conclusions.  A 
subsequent  section  will  discuss  methodological  issues  of  developmental  research,  including  data  management  and 
analysis,  and  the  challenges  of  teaching  and  studying  one’s  teaching.  A final  section  provides  guidelines  on  teaching 
decisions  and  matching  developmental  research  prompts. 

Developmental  Study  of  ID  Instruction 

Reflexive  Teaching  Model 

The  major  components  of  our  reflexive  approach  for  teaching  instructional  design  include  (a) 
characteristics  and  roles  of  instructor  and  learner,  (b)  co-participation  structures,  and  (c)  dialogue  of  teacher  and 
student  within  each  participation  structure.  Teachers  and  students  are  viewed  as  learners,  each  possessing  unique 
learning  beliefs,  knowledge,  competencies,  experiences,  sensibilities,  and  motivations.  The  instructor  roles  within 
the  model  include  that  of  a learner,  a designer  of  an  instructional  environment,  and  a teacher  responsive  to  learner 
needs  within  this  environment.  The  teacher  assumes  a supportive  role,  not  unlike  that  of  a coach  (Schon,  1987).  In 
terms  of  knowledge  and  competencies,  the  teacher  must  bring  not  only  expertise  in  instructional  design,  but  subject 
matter  knowledge,  pedagogical  knowledge,  pedagogical  content  knowledge,  and  knowledge  of  the  learning 
principles  of  instructional  design  (Shulman,  1986).  Student  roles  in  the  reflexive  model  include  that  of  a learner  with 
a willingness  to  engage  within  the  participation  structures  and  perform  learning  tasks.  By  reflexive  we  mean 
instructor  and  student  learning  of  instructional  design  through  multiple  forms  of  activity. 

Participation  structures  include  the  classroom,  learning  tasks  (outside  of  the  classroom),  individual 
conferences,  electronic  mail,  web  site,  and  texts.  Although  careful  consideration  must  be  given  to  the  design  of  these 
structures,  some  negotiation  of  their  features  by  students  is  also  encouraged.  The  key  is  being  open  to  feedback  and 
periodically  “stepping  outside”  a teacher's  perspective  to  consider  these  suggestions.  Dialogue  between  the 
participants  is  crucial  within  these  structures.  For  example,  in  a group  activity,  dialogue  enables  the  knowledge  of 
instructional  design  and  one’s  views  and  experiences  to  be  shared  in  an  open  and  testable  way,  initiating  a shared 
reflective  process.  Cooperative  learning,  presentations,  and  peer/teacher  evaluations  are  key  strategies.  Within  the 
structures  dialogue  with  oneself  through  reflective  tasks  promotes  ID  understanding  and  understanding  of  one’s  own 
thought  processes.  Weekly  written  project  drafts  and  feedback  on  one’s  performance  help  to  develop  reflective 
activity. 

Instructional  Sequence 

The  first  two  phases  in  the  ID  process  in  this  course.  Learning  Beliefs  and  Design  Tools  (principally  ID 
models),  are  used  to  establish  the  context  for  the  traditional  ID  process  (see  Figure  1).  Design-A-Lesson  and 
Learning  Principles  tasks  help  students  to  reflect  on  how  they  currently  plan  instruction  and  their  views  on  learning. 
Students  draft  a Mission  Statement,  which  is  used  to  assess  how  students'  learning  beliefs  are  being  applied  in  their 
projects.  Students  also  sketch  a visual  of  their  own  ID  model  and  provide  an  explanatory  narrative. 


ID  INSTRUCTION:  Instructional  Design  Phases 
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Figure  I . Course  sequence  and  learning  tasks. 

Students  choose  an  instructional  problem  for  an  ID  project  and  record  their  initial  understanding  of  the 
problem  through  an  Intent  Statement.  The  Needs  Assessment  phase  structures  students'  research  into  the 
instructional  problem  and  options  for  addressing  it.  A personal  conference  provides  individual  assistance  with  their 
needs  assessment  strategy  in  terms  of  what  to  study,  with  whom  to  talk,  what  references  to  consult,  and  how  to 
summarize  their  findings.  Based  on  Needs  Assessment,  students  identify  goals  for  their  project.  Following  Needs 
Assessment,  students  are  introduced  to  design  phases,  which  include  Instructional  Sequence,  Assessment, 
Instructional  Framework,  Instructional  Media/Technology,  and  a Prototype  Lesson.  Instruction  helps  raise  students’ 
awareness  to  the  purposes  of  assessment  and  appropriate  assessment  methods.  During  the  Instructional  Frameworks 
phase  in  which  teaching  options  are  specified,  students  demonstrate  a teaching  strategy  in  their  Prototype  Lesson. 
Instructional  Media  and  Technology  is  addressed  throughout  the  course  beginning  with  Needs  Assessment.  We  urge 
students  to  be  open  to  a range  of  media  and  technology  possibilities  and  to  make  a case  for  how  their  choices 
support  their  goals.  A second  personal  conference  addresses  Program  Evaluation  and  individual  project  issues.  The 
final  week  of  the  course  has  students  submit  revised  personal  ID  models  and  a written  self-evaluation  of  the  course 
and  their  learning. 

Developmental  Research  Methodology 

To  study  the  reflexive  teaching  approach,  the  design  and  development  cycle  (Richey  & Nelson,  1996)  was 
adopted  as  a research  framework  for  six  deliveries  of  an  ID  course  from  1994-1998.  In  developmental  research, 
objectives,  rather  than  questions,  characterize  the  inquiry.  In  this  study  we  were  interested  in  how  the  reflexive 
approach  developed  over  time,  and  our  research  objectives  for  each  delivery  of  the  course  included  the  following: 

• Describe  the  design  decisions. 

• Describe  the  implementation  of  the  model,  or  what  occurred  during  each  delivery. 

• Describe  the  evaluation  of  the  model,  in  terms  of  student  learning  and  student  perceptions  of  their 
learning  and  teaching. 

The  case  study,  defined  as  each  class  delivery,  was  chosen  as  the  unit  of  analysis  to  describe  the  course 
development  (Yin,  1994).  Case  1 was  a 5 -week  summer  course  with  nine  contact  hours  per  week.  Cases  2-5  were 
15 -week  semesters,  which  met  for  three  hours  per  week.  Case  6 involved  K-12  teachers  from  a school 
district/university-sponsored  master’s  program,  during  a 15-week  spring  semester,  which  met  off-campus  for  3 
hours  once  per  week. 

Participants  included  113  students  and  two  instructors  in  a master’s  level  instructional  design  course  from  a 
university’s  instructional  technology  graduate  education  program.  Of  the  113  students,  73  had  teaching  experience. 
Educational  levels  of  interest  included  18  elementary  school,  15  middle  school,  26  high  school,  6 with  an  overall  K- 
12  interest,  29  college,  and  19  training.  The  largest  content  area  focus  of  the  participants  included  science  and 
technology  (19),  followed  by  language  (17),  computing  (14),  and  special  education  (12). 

Eight  different  data  sources  were  used  (see  Figure  2),  providing  us  with  a means  of  data  triangulation  in 
which  different  sources  of  information  informed  the  three  research  objectives  (Yin,  1994).  Data  sources  for  design 
decisions  included  working  logs,  e-mail,  and  syllabi.  Working  logs  documented  our  thinking  and  involvement  in  the 
ID  course,  including  class  presentations,  learning  tasks,  student  guide  content,  and  teaching  model  representations. 
A syllabus  recorded  major  design  decisions  for  each  case,  including  course  purpose,  instructional  materials, 
assessment,  and  course  sequence. 
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Data  sources  for  implementation  of  the  reflexive  approach  included  working  logs,  e-mail,  draft  ID  projects, 
and  personal  conference  notes.  Whenever  possible  during  class,  student  comments  and  observations  of  class 
activities  and  student/instructor  performance  were  recorded  in  working  logs.  Outside  of  class  working  logs  recorded 
our  perceptions  of  what  occurred  in  class,  summaries  of  weekly  submissions  of  student  work,  and  notes  from  weekly 
instructor  meetings.  E-mail  was  a source  of  instructor  dialogue  on  weekly  shifts  in  instruction  and  student  needs. 
Draft  ID  projects  provided  evidence  on  how  students  transferred  their  ID  process  understanding  into  design 
decisions.  The  first  of  two  personal  conferences  between  instructors  and  student  met  early  in  the  semester  to  discuss 
a student’s  project  choice  and  needs  assessment  strategy. 

Data  sources  for  evaluation  of  the  model  included  students’  completed  projects,  course  evaluations,  student 
evaluations  of  their  learning  and  written  notes  from  a second  personal  conference.  Required  project  components 
included  a Mission  Statement  of  beliefs  about  leaming/leamers/instruction.  Project  Intent  Statement,  Needs 
Assessment,  Sequence,  Assessment  Plan,  Instructional  Framework,  Instructional  Media/Technology  Plan,  a 
Prototype  Lesson,  and  a Program  Evaluation  Plan  (formative  and  summative).  Course  evaluations  included  Likert- 
scale  questions  to  record  student  perceptions  of  instruction,  instructors  and  materials.  Students  also  completed  open- 
ended  questions,  which  asked  them  to  rate  and/or  comment  on  learning  tasks  and  instructional  materials.  The  self- 
evaluation  task  varied  over  each  course  delivery  but  typically  asked  students  to  summarize  what  they  learned  in  the 
course,  what  they  would  like  to  learn  next,  and  comment  on  their  experiences  with  the  course.  Finally,  a second 
student-teacher  conference  was  conducted  at  the  end  of  the  course  to  address  student  questions  on  their  projects. 

For  each  case  the  design  decisions  were  reported  by  describing  (a)  participants,  (b)  learning  tasks,  (c) 
course  sequence,  (d)  assessment,  and  (e)  instructional  materials.  Analysis  of  the  implementation  of  the  model  for 
each  case  was  reported  by  describing  student  performance  and  responses  to  instruction  and  instructor’s  assistance 
during  (a)  ID  context  activities,  (b)  ID  process  instruction,  and  (c)  draft  ID  projects.  Evaluation  of  the  teaching 
model  was  reported  on  the  basis  of  summarizing  (a)  student  performance  on  the  ID  project,  (b)  students’  self- 
perceptions of  their  learning,  and  (c)  instructor  responsivity  to  student  needs.  Details  of  the  data  collection  and 
analysis  procedures  can  found  in  Shambaugh  and  Magliaro  (2001). 

Developmental  Study  Conclusions 
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What  we  learned  from  this  developmental  study  fall  into  two  categories  that  also  match  our  view  of 
developmental  research:  a greater  understanding  of  (a)  a process;  in  this  case,  a reflexive  teaching  approach  for  ID 
instruction,  and  (b)  the  use  of  developmental  research  to  study  ID  instruction. 

Reflexive  model  understanding.  A summary  of  what  we  learned  about  the  teaching  model  is  elaborated 

below. 

• Articulate  the  theoretical  basis  for  a reflexive  approach : learning  is  constructed  by  the  individual 
(Bruner,  1990),  that  there  is  a developmental  interplay  between  one's  thinking  and  the  social  world 
(Brown,  Collins  & Duguid,  1989;  Fosnot,  1996;  Lave  & Wenger,  1991),  and  that  teaching 
supports  learning  construction  through  multiple  forms  of  assistance  (Tharp  & Gallimore,  1988). 
Teaching  is  not  viewed  as  content  delivery  or  communicating  knowledge  but  the  development  of 
learning  environments  to  support  individual  knowledge  construction  (Duffy  & Cunningham, 

1996). 

• The  social  system  in  which  our  reflexive  teaching  was  embedded:  a reflexive  stance  regards 
teacher  and  student  as  mutual  learners  with  different  roles,  both  involved  in  a critical,  self- 
appraisal of  their  activity  within  the  learning  environments  (i.e.,  the  participation  structures). 

• Classroom  syntax:  Out  of  this  study  the  classroom  participation  structure  was  characterized  by 
three  phases,  including  (a)  setting  the  stage,  (b)  representing  understanding  by  instructor  and 
student,  and  (c)  debriefing.  Our  assistance  to  student  learning  was  also  depicted  in  a responsivity 
cycle  involving  (a)  design  and  implementation;  (b)  mutual  engagement,  performance,  and 
reflection;  and  (c)  submit  learning  tasks,  query  instructors  on  issues,  and  respond  to  student 
concerns. 

• Principles  of  student  reaction : How  students  reacted  to  the  teaching  are  summarized  by  individual 
ID  phases, 

• Support  system  requirements:  In  this  model  students  were  new  to  learning  beliefs  examination, 
were  sometimes  uncomfortable  with  submitting  work- in-progress  versus  finished  work,  were  new 
to  detailed  feedback  from  instructors,  and  required  some  time  to  take  responsibility  for  design 
decisions  responsive  to  learners  versus  what  instructors  wanted. 

• Requirements  for  co-participation : Joint  student-instructor  learning  required  a (a)  willingness  to 
share  control  and  responsibility  for  learning,  (b)  a readiness  for  dialogic  education,  and  (c)  a 
genuine  desire  to  be  reflexive  in  one's  teaching  and  learning. 

Developmental  research  understanding.  Our  understanding  of  what  it  meant  to  study  our  teaching  using  a 
development  research  framework  included  the  following. 

• Model  representation : our  description  of  the  model  provides  a basis  for  subsequent  development 
of  the  model  for  purposes  of  exploration,  prediction,  and  planning. 

• Model  development:  a theoretical  basis  for  a model  must  be  articulated  and  one  must  be  clear  as  to 
the  purposes  of  the  developmental  study  to  study  the  model;  in  particular,  what  knowledge, 
understanding,  or  predicting  is  being  developed. 

• Data  management:  maintaining  systematic  data  management  procedures  are  crucial  to  track  the 
evolution  of  the  model  prototype  and  to  make  any  generalizations  across  time. 

• Individual  and  collaborative  requirements:  a reflexive  disposition  is  needed  to  study  one's 
teaching. 

Developmental  Research  Issues 

This  section  provides  more  details  on  methodology  issues  involved  in  the  above  developmental  research 

study. 

Data  Management  and  Analysis 

Data  management  in  developmental  research  involves  the  procedures  for  a systematic,  coherent  process  of 
data  collection,  storage  and  retrieval  for  the  purpose  of  high  quality,  accessible  data,  the  documentation  of  analysis, 
and  retention  of  data  (Huberman  & Miles,  1998).  In  this  study  data  were  analyzed  using  the  qualitative  techniques 
of  Miles  and  Huberman  (1994),  which  consisted  of  data  reduction  from  original  data  sources  using  categorical 
analysis  (Spradley,  1980)  and  display  of  this  reduced  data  in  “frames”  or  tables  that  enabled  conclusions  to  be 
drawn.  The  data  analysis  sequence  included  collecting  data,  reducing  the  data  into  frames,  and  reporting  the 
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reduction  in  an  analysis  section.  The  data  reduction  documents  for  each  case  were  kept  in  3 -ring  notebooks  and  each 
notebook  was  divided  by  data  sources,  a strategy  that  served  to  separate  the  data  from  the  report  and  provided  a 
means  to  organize  the  data  and  track  the  analysis  sequence  from  data  source  to  data  reduction  to  data  reporting. 

Describing  design  decisions  was  straightforward.  Syllabi  concisely  recorded  these  as  well  as  what  was 
written  in  working  logs,  which  recorded  dialogue  between  instructors  over  time  as  to  what  ID  content  means  and 
how  to  teach  this  content.  Implementation  of  these  design  decisions  in  an  actual  course  was  also  relatively  clear  to 
record  in  Working  Logs  in  terms  of  what  we  as  instructors  implemented  and  how  students  reacted  to  these  decisions 
from  what  they  said  and  what  they  designed  during  the  semester.  Weekly  submissions  of  draft  ID  project 
components  were  evaluated  in  terms  of  performance  criteria.  Criteria  for  each  project  component  were 
communicated  to  students  using  a task  sheet. 

Evaluation  of  student  learning  was  more  complicated.  The  ID  projects  were  the  principal  source  to  indicate 
student  performance  of  ID  understanding  and  were  analyzed  for  (a)  completeness  (i.e.,  Were  all  of  the  project 
components  in  place?),  (b)  consistency  of  learning  beliefs  across  design  components,  and  (c)  coherence  of  design 
components.  For  consistency,  important  ideas  (e.g.,  assisting  learners,  skill  proficiency,  working  together,  problem 
solving,  multiple  instructional  approaches)  were  identified  from  a Project  Intent  Statement.  The  ID  project  was 
reviewed  to  note  whether  or  not  these  ideas  were  explicitly  addressed  in  the  projects.  For  example,  if  a student  wrote 
about  the  importance  of  students  working  together,  we  looked  for  this  feature  in  the  instructional  approach, 
activities,  or  prototype  lesson.  A Mission  Statement  recorded  what  students  believed  were  important  teaching  and 
learning  principles.  The  ID  project  was  reviewed  to  note  whether  or  not  the  features  of  the  Mission  Statement  were 
addressed  in  the  projects.  For  coherence,  notations  were  recorded  in  terms  of  how  each  design  component  was 
represented.  A judgment  was  made  whether  or  not  these  design  components  appropriately  supported  each  other, 
such  as  a match  between  assessment  methods  and  teaching. 

Each  case  was  a description  of  “what  happened  and  how  the  course  proceeded”  using  design  decisions, 
model  implementation,  and  evaluation  of  the  model  as  a way  to  describe  the  use  and  results  of  the  model  (between- 
case  analysis).  Data  displays,  structured  summaries,  and  tables  allowed  a condensed  view  of  the  data  sources  and 
revealed  that  some  further  analysis  was  needed,  such  as  coding  of  structured  summaries  to  reveal  themes  as  well  as 
to  identify  exceptions  and  differences. 

The  summary  of  an  across-case  analysis  of  the  six  cases  (across-case  analysis)  reported  the  changes  in 
design  decisions,  implementation,  and  evaluation  of  the  model  (see  Shambaugh  & Magliaro,  2001).  As  Huberman 
and  Miles  (1998)  have  commented,  “each  case  has  a specific  history-which  we  discard  at  our  peril-but  it  is  a history 
contained  within  the  general  principles  that  influence  its  development”  (p.  194).  This  summary  attempted  to 
preserve  the  uniqueness  of  each  case,  yet  also  make  comparisons  along  the  developmental  cycle  based  on  repeat 
deliveries  of  the  course.  In  an  effort  to  extend  external  validity,  what  participants’  “did,  said,  or  designed,”  were 
examined  in  multiple  settings.  The  description  of  the  reflexive  model,  based  on  what  was  found  fromthis  analysis, 
provided  a set  of  generalizations  on  how  the  model  was  implemented,  as  well  as  conditions  necessary  for  its  use. 

The  danger  to  this  generalization  was  that  “multiple  cases  will  be  analyzed  at  high  levels  of  inference,  aggregating 
out  the  local  webs  of  causality  and  ending  with  a smoothed  set  of  generalizations  that  may  not  apply  to  any  single 
case”  (Huberman  & Miles,  p.  194).  We  did  not  average,  for  example,  course  evaluation  results  (i.e.,  from  Likert 
scales)  to  avoid  misinterpretation  and  superficiality  and  to  preserve  case  uniqueness.  The  goal  was  to  better 
understand  the  overall  processes  at  work  across  the  cases,  including  teacher  and  student  thinking,  participation,  and 
teacher  responsivity. 

In  traditional  instances  of  qualitative  data  collection  and  analysis,  the  research  “shifts  between  cycles  of 
inductive  data  collection  and  analysis  to  deductive  cycles  of  testing  and  verification”  (Huberman  & Miles,  1998,  p. 
198).  In  this  study,  sources  of  data  were  already  in  place  prior  to  conceptualizing  a study  framework.  However,  the 
details  of  the  framework  and  the  subsequent  data  analysis  of  the  six  cases  cycled  back  and  forth  to  realize  more 
appropriate  matches  of  methodology  and  method  to  existing  data  sources  and  research  objectives.  The  analytic  cycle 
for  this  study  could  be  better  described  as  one  that  moved  between  conceptual  framework,  case  analysis,  and  study 
purpose.  Although  being  clear  as  to  the  purpose  of  a study  is  preferable  before  constructing  a methodology,  such 
clarity  is  not  always  possible  due  to  the  complexity  of  processes  to  be  studied,  data,  and  personal  involvement  over 
time.  This  reality  requires  teachers-researchers  who  feel  comfortable  about  this  dynamic  movement  and  emergence 
of  understanding. 

One  possible  source  for  bias  in  this  study  is  the  large  amount  of  data,  which  may  have  led  to  missing 
important  information  or  overweighing  some  findings  due  to  focusing  on  a particular  and  large  set  of  data.  Personal 
involvement  with  the  course  also  increased  the  possibility  that  recorded  observations  in  working  logs  highlighted 
particular  incidents  while  ignoring  others.  The  working  logs,  however,  served  as  a “reflexivity  journal”  (Carney, 
1990)  and  recorded  observations  or  design  decisions  that  would  have  been  lost  to  our  collective  memories  over  the 
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five  years  of  involvement.  Personal  involvement  as  co -instructors  also  implied  a danger  in  being  selective  and 
overconfident  with  some  data.  Another  shortcoming  was  not  checking  descriptions  with  each  case  of  students  and 
additional  peer  review  outside  of  the  co-instructor. 

To  address  these  shortcomings,  we  used  multiple  data  sources  for  triangulation  to  achieve  an  agreement  of 
one  data  source  with  another.  Multiple  sources  of  data,  such  as  working  logs,  e-mail,  and  syllabi,  also  provided 
different  strengths  and  complemented  each  other.  Syllabi,  for  example,  compactly  recorded  design  decisions,  while 
working  logs  and  email  documented  our  thinking  that  influenced  these  decisions.  The  data  sources  were  a mix  of 
student-generated  (i.e.,  conferences,  ID  projects,  course  and  self-evaluations)  and  instructor-generated  (i.e.,  working 
logs,  e-mail,  syllabi)  data. 

During  the  analysis  of  these  data  sources  we  looked  for  contrasts,  comparisons,  and  exemplars  and  reported 
these  during  the  data  reduction  so  as  not  to  filter  out  outliers  and  extreme  instances.  Replication  of  the  conceptual 
framework  across  multiple  cases  helped  to  provide  evidence  that  what  was  described  in  each  case  was  based  upon 
the  details  of  the  instructional  approach  and  uniqueness  of  the  setting  and  participants.  We  were  conscious  to  remain 
“descriptive”  in  the  writing  during  the  analysis  of  each  case. 

Another  means  of  addressing  verification  of  methodology,  findings,  and  conclusions  was  an  “auditing”  by 
the  co-instructor.  Through  periodic  reviews  of  methodology  and  analysis,  inconsistencies  in  design  decisions  were 
identified  and  prompted  for  clarification.  Such  feedback  characterized  another  aspect  of  our  reflexive  stance,  the 
need  to  assume  regular,  ongoing,  and  self-conscious  documentation  of  teaching. 

Balancing  Design-Teaching  and  Developmental  Research 

The  design  decisions,  implementation,  and  evaluation  of  the  ID  course  were  event-driven,  meaning  that 
they  served  our  instructional  needs  to  watch,  ask,  and  examine  (Wolcott,  1992).  These  observations,  interviews,  and 
documents  were  in  place  prior  to  the  conceptual  framework  of  the  study.  As  a result,  the  data  sources  were  not  as 
complete,  tightly  defined,  or  structured  across  the  six  cases  if  they  had  been  researcher-driven.  Some  data  sources, 
such  as  syllabi,  course  evaluations,  and  self-evaluations,  evolved  to  suit  the  learning  needs  of  the  students.  However, 
because  we  had  presented  preliminary  findings  at  research  conferences  (e.g.,  Shambaugh  & Magliaro,  1995, 1996), 
we  had  collected  and  stored  data  for  each  case,  as  well  as  conducted  analysis  with  most  of  the  data  sources,  although 
using  different  methodologies.  These  research  efforts  can  be  regarded  as  interim  analyses  in  which  we  became 
familiar  with  studying  teaching  and  learning  products,  developing  procedures  in  recording  observations  and  personal 
conferences,  as  well  as  retaining  and  analyzing  documents.  Over  the  six  cases,  we  came  to  better  understand  the 
instructional  setting,  being  sensitive  to  research  opportunities  and  becoming  more  systematic  in  our  data  collection 
and  management  efforts,  but  also  retaining  instruction  and  responsiveness  to  learners  as  our  top  priority. 

Developmental  Research  Guidelines 

From  this  study  we  have  become  more  aware  of  how  designing/teaching  can  be  informed  by  adherence  to 
research  tenets,  as  well  as  more  trustworthy  results  that  can  be  obtained  by  structuring  teaching  decisions  and 
learning  artifacts  as  data  sources.  The  following  guidelines  for  conducting  developmental  research  are  organized 
along  the  developmental  research  cycle,  consisting  of  design,  implementation,  and  evaluation  components  (see 
Figure  3)  and  are  listed  first  by  teaching  decisions  and  in  the  second  column  by  developmental  research  prompts. 

The  guidelines  address  both  designing  and  teaching,  activities  customarily  viewed  as  separate,  but  are  viewed  by  us 
as  complementary.  Design  requires  the  involvement  of  practitioners  who  bring  insight  into  practical  implementation 
problems,  and  our  belief  that  designer  and  teacher  can  be  one  and  the  same. 

Design  Guidelines 

A major  teaching  decision  is  determining  the  purpose  of  the  instructional  intervention,  through  the  use  of 
course  goals.  A complementary  developmental  research  prompt  is  being  clear  as  to  the  purpose  of  the  research.  By 
answering  the  question,  "What  is  this  study  about?"  an  appropriate  methodology  to  study  the  research  question(s) 
can  be  formulated.  Developmental  research  objectives  help  to  understand  the  complex  phenomena  at  play  in  any 
educational  intervention,  so  as  to  provide  initial  descriptive  data  for  subsequent  research  questions. 

Design  and  teacher  thinking  can  be  made  explicit  by  recording  design  conversations  and  decisions  from 
notes,  lesson  plans,  unit  plans,  curriculum  guides,  or  syllabi.  A more  comprehensive  representation  of  one's  teaching 
can  be  documented  in  an  instructional  framework,  using  the  conceptual  approach  of  Joyce,  Weil,  and  Calhoun 
(2000).  The  instructional  framework  records  the  theoretical  learning  foundation  of  the  different  teaching  approaches. 
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describes  the  social  and  support  systems  of  the  approaches,  specifies  syntax  or  procedural  guidelines  of  to 
implement  the  approaches,  and  the  instructional  and  nurturant  effects  of  the  teaching. 

Another  set  of  design/teaching  decisions  is  to  think  through  the  activity  roles  of  the  teacher  and  the  student 
and  to  identify  the  purpose  of  the  activities,  determine  if  the  activities  are  developmentally  appropriate,  and 
sequenced  appropriately.  Thus,  one  can  examine  the  alignment  between  desred  learning  outcomes,  teaching 
options,  and  assessment.  Developmental  research  prompts  here  would  retain  copies  of  learning  activities  and  tasks 
so  as  to  provide  evidence  of  how  these  data  sources  evolved  over  time  to  support  teaching  adjustments. 

Implementation  Guidelines 

Implementation  guidelines  for  teachers  involve  ongoing  evaluation  of  one's  design/teaching  decisions  and 
dynamic  adjustments  needed  during  the  intervention  to  address  the  desired  learning  outcomes.  From  a 
developmental  research  point  of  view,  these  design  and/or  teaching  adjustments  need  to  be  recorded  so  as  to  provide 
a clear  description  of  what  occurred.  Also,  any  changes  in  activities  or  tasks  need  to  be  documented  to  provide  a 
description  of  how  these  data  sources  evolve  over  time. 

It  is  possible  during  implementation  that  other  forms  of  learning  may  be  occurring.  A research  prompt  can 
be  useful  to  continue  looking  for  phenomena  that  may  be  occurring  but  not  addressed  through  a methodology  or  that 
emerges  as  a result  of  what  occurs  from  the  intervention  or  from  "looking  at  classrooms",  in  new  methodological 
approaches.  Each  school  year  or  course  delivery  provides  a unique  set  of  learners  and  learning  characteristics  that 
must  be  analyzed  in  developmental  research  as  a unique  case. 

During  implementation  teachers  remain  conscious  of  student  perceptions  to  their  teaching  and  the  tasks  and 
activities  provided  for  them,  as  well  as  ongoing  assessment  of  the  ways  in  which  students  are  learning  or  not 
learning.  Are  the  assessment  methods  providing  this  information?  Developmental  research  prompts  here  include 
habitual  analysis  of  ongoing  assessment,  whether  observations,  interviews,  or  learning  artifacts.  In  addition,  the 
development  of  instruments  to  periodically  assess  student  perceptions  provides  a new  data  source  for 
implementation  analysis  and  teaching  adjustments. 

Evaluation  Guidelines 

Evaluation  guidelines  primarily  involve  the  examination  of  student  performance  on  learning  tasks  and 
feedback  on  teaching  efforts.  Developmental  research  requires  data  sources  that  reveal  student  learning  in  terms  of 
learning  outcomes,  as  well  as  student  or  peer  perceptions  on  teaching.  Multiple  forms  of  data  from  student  activities 
(process  and  product  forms)  provide  triangulating  evidence  of  student  learning.  A challenge  here  is  to  maintain 
systematic  data  collection  and  management  procedures  in  light  of  busy  school  schedules. 

A secondary  set  of  evaluation  guidelines  encourages  designers/teachers  to  solicit  feedback  and  advice  from 
peers  and  other  sources.  Developmental  research  prompts  teachers  to  collaborate  on  teacher  studies  and  to 
disseminate  findings. 


Designing/Teaching  Decisions 

Developmental  Research  Prompts 

DESIGN 

What  is  to  be  learned  in  this  intervention? 

What  is  the  purpose  of  the  study?  Be  explicit  about  how 
teaching  will  be  studied  through  research  objectives 

Make  explicit  teacher  thinking.  Determine  if  there  is  an 
alignment  between  learning  outcomes,  teaching  strategy, 
and  assessment. 

Record  and  store  design  decisions  from  conversations, 
notes,  or  lesson  plans,  syllabi. 

Document  instructional  framework,  including  theoretical 
foundation  of  teaching  (Joyce,  Weil  & Calhoun,  2000) 

Identify  and  describe  teacher  and  student  activity. 

Examine  teacher  and  student  activity  as  data  through 
reflective  and  performance  artifacts.  Retain  copies  of 
learning  tasks. 

IMPLEMENTATION 

Continually  evaluate  appropriateness  of  design  decisions 
to  learning  outcomes.  Document  teaching  activities  and 
adjustments. 

Document  changes  in  data  sources  as  they  evolve  over 
time  to  address  research  objectives. 

What  other  forms  of  learning  are  occurring  or  not 
occurring?  (incidental  learning) 

Document  direct  (learning  goals)  and  indirect  (nurturant) 
effects  of  instruction  (Jovce,  Weil  & Calhoun,  2000). 

305 


Determine  learning  characteristics  of  students. 

Each  course  delivery  analyzed  as  a unique  case. 

Examine  assessment  criteria. 

Are  assessment  criteria  analyzable? 

How  are  students  learning  and  perceiving  your 
instruction? 

Develop  instruments  to  obtain  student  learning  and 
student  perceptions. 

EVALUATION 

Keep  records  on  student  performance. 

Identify  data  sources  that  reveal  student  learning. 
Maintain  systematic  data  management  procedures. 

Solicit  feedback  from  students  and  listen  to  what  they 
say. 

Identify  data  sources  for  student  perceptions  of  teaching. 
Maintain  systematic  data  management  procedures. 

Seek  out  feedback  from  peers,  develop  teaching 
repertoire. 

Collaborate  on  research  into  one’s  teaching,  disseminate 
findings. 

Figure  3 . Designing/teaching  and  developmental  research  guidelines. 


Implications 

The  downsides  of  research  for  designers-teachers  have  been  characterized  by  van  den  Akker  (1999)  as 
answers  that  are  frequently  too  narrow,  too  superficial,  or  too  late  to  do  any  good.  Developmental  research  aims  to 
address  these  issues  of  meaningfulness,  generalizability,  and  usefulness.  Developmental  research  can  be  used  to 
assist  designers  and  teachers  in  the  development  of  educational  interventions,  while  providing  a systematic  means  to 
study  their  implementation.  As  all  interventions  are  influenced  by  multiple  stakeholders  and  are  contextually -rich  in 
nature,  developmental  research  provides  a dynamic  vantage  point  for  collaborators  to  talk  about  their  roles,  whether 
these  roles  be  pragmatic  (design,  teaching)  or  knowledge-building  (research).  Developmental  research  also  provides 
a framework  to  study  instructional  problems  and  responses,  particularly  as  changes  in  the  methodological 
framework  may  be  required  to  adapt  to  evolving  response  prototypes. 

Reeves  (2000)  provides  several  heuristics  for  developmental  research  activities,  including  the  need  to  focus 
on  difficult  learning  problems;  align  designs  with  learning  outcomes,  teaching,  and  assessment;  collaborate  and 
share  with  others;  and  the  hard  work  needed  for  any  developmental  research  project.  Another  critical  heuristic 
suggested  by  Reeves  involves  identifying  the  theoretical  and  practical  design  principles  that  underlie  a prototype  and 
conduct  rigorous  studies  of  these  principles  in  real  settings.  Carroll  (2000)  reminds  us  that  viewing  design  as  inquiry 
"raises  the  question  of  what  abstractions  can  support  the  development  and  sense  of  knowledge  in  design"  (p.  65).  In 
particular,  we  raise  to  ourselves  and  others  two  challenges;  namely,  (a)  the  difficulty  of  abstracting  principles  or 
frameworks  to  help  us  in  designing  contextually -rich  learning  environments,  and  (b)  acknowledging  in 
developmental  research  the  naturalistic  nature  of  teacher  knowledge.  Both  of  these  challenges  resonate  with  the  twin 
purposes  of  developmental  research  raised  at  the  beginning  of  this  paper:  (a)  formulating  what  we  learn  from  our 
design  efforts  and  (b)  developing  responsive  educational  interventions. 
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Abstract 

Instructional  designers  are  well  equipped  to  handle  the  leadership  of  faculty  development  in  higher 
education.  Faculty  development  is  part  of  the  process  of  lifelong  learning  for  the  college  or  university 
instructor  and  a key  component  of  the  transformational  changes  taking  place  in  higher  education.  The  need 
for  faculty  to  appropriately  integrate  technology  into  their  curriculum  and  utilize  innovative  instructional 
methodologies  is  driven  by  five  factors:  students,  faculty,  administration,  society,  and  technology.  The  role 
of  instructional  designers  and  instructional  systems  design  methodologies  are  critical  to  the  success  of 
faculty  development  programs  and  can  successfully  facilitate  the  dynamic  change  process  currently 
underway  in  colleges  and  universities. 

Faculty  development  in  higher  education  is  a part  of  the  process  of  lifelong  learning  for  the  college  or 
university  instructor  and  a key  component  of  the  transformational  changes  taking  place  in  higher 
education.  Five  factors  are  driving  the  need  for  faculty  to  appropriately  integrate  technology  and 
instructional  systems  approaches  into  the  design  and  development  of  their  courses:  students,  faculty, 
administration,  society,  and  technology.  A skilled  instructional  designer  is  a well-trained  professional  for 
assisting  faculty  members  and  serving  faculty  development  programs  to  better  utilize  innovative 
instructional  methodologies,  strategies,  and  techniques.  The  anticipated  outcome  of  the  current 
transformation  in  higher  education  is  improvement  in  teaching  facilitated  by  faculty  development  initiatives 
under  the  guidance  of  the  instructional  designer. 

The  Transformation  of  Higher  Education 

Change  is  happening  within  many  sectors  that  have  direct  influence  on  colleges  and  universities  and  the 
effects  are  certain  to  alter  the  way  in  which  higher  education  operates  in  the  future.  Managing  the  transformation  is 
the  key  to  survival  for  colleges  and  universities.  Understanding  the  forces  of  change  requires  leadership  that  is 
skilled  in  evaluating  and  synthesizing  the  inter-relatedness  of  the  variables  involved.  Technology,  as  an  innovation, 
consistently  creates  changes  in  the  way  people  and  organizations  function,  access  information,  and  communicate. 
The  transformation  of  higher  education  can  be  understood  in  terms  of  the  forces  that  are  driving  the  need  for  faculty 
development.  The  changing  student  character  is  creating  the  need  for  faculty  to  adopt  new  teaching  strategies.  The 
need  of  faculty  and  administration  to  accomplish  their  respective  functions  drives  the  need  for  and  enables  the 
establishment  of  faculty  development  programs.  The  changes  in  the  demographics,  culture,  and  nature  of  work  in 
society  are  reflected  in  the  expectations  of  graduates.  The  application  of  technology  to  educational  objectives  creates 
an  evolving,  dynamic  environment  for  learning,  and  subsequently  requires  an  improved,  dynamic  methodology  of 
teaching.  Based  on  their  knowledge  of  systems  theory  and  change  management,  instructional  designers  can  serve  as 
change  agents  within  faculty  development  programs. 

Role  of  Faculty  Development  in  the  Transformation  of  Higher  Education 

Faculty  development  is  a process  of  professional  training  (and  retraining)  undertaken  by  instructors  in 
higher  education.  Like  its  corporate  counterpart,  faculty  training  and  development  is  important  for  maintaining  or 
improving  the  quality  of  services  and  products  offered  by  the  organization.  Highly  skilled  faculty  is  the  core  of  a top 
quality  academic  institution  and  is  the  primary  producers  of  critical  higher  education  products:  prized  research  and 
educated  graduates. 

A working  definition  of  faculty  development  will  help  clarify  this  topic  and  show  the  far-reaching 
boundaries  that  support  its  role  as  a catalyst  for  transforming  higher  education.  Faculty  in  colleges  and  universities 
are  undertaking  fundamental  remodeling  of  their  teaching  approaches.  Duderstadt  (1999)  believes  that  faculty  in 
higher  education  will  require  new  instructional  methods,  models,  and  techniques  for  serving  the  learning  needs  of 
the  future  generation  and  that  faculty  development  initiatives  are  instrumental  in  guiding  the  transformation. 
Typically,  the  literature  addresses  faculty  development  only  in  terms  of  integrating  technology  into  the  teaching  and 
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learning  experience.  While  technology  should  play  an  important  role,  it  remains  a mere  tool  to  support 
fundamentally  good  educational  practice. 

Faculty  development  can  also  be  viewed  as  a process  of  careful  identification  of  past  teaching  successes 
and  the  generation  of  ideas,  beliefs,  and  convictions  about  teaching  and  learning.  From  these  beliefs  it  is  then 
possible  to  choose  among  the  many  new  tools,  technologies,  and  instructional  strategies  available.  Faculty 
development  programs  are  focused  on  the  integration  of  educational  technology  tools,  such  as,  the  Web,  hardware, 
software,  and  the  appropriate  use  of  audio-visual  equipment,  into  a faculty  member’s  established  teaching  practices. 
Instructional  strategies  and  methodologies  include  the  instructional  systems  theories,  models,  and  strategies  for 
analyzing,  designing,  developing,  implementing,  and  evaluating  educational  experiences  and  outcomes. 

Faculty  development  is  for  the  purpose  of  improving  teaching  and  learning  at  undergraduate  and  graduate 
levels.  American  colleges  and  universities  have  been  making  numerous  efforts  to  improve  teaching  and  learning  on 
their  campuses  since  the  1 980s.  As  a result,  some  changes  have  occurred  and  the  repertoire  of  teaching  practices  has 
expanded,  such  as  greater  student  involvement  through  collaborative  and  cooperative  learning;  technology -based 
learning;  learning  communities  to  bring  faculty  and  students  closer  together;  and  teaching  centers  to  improve 
practice.  Despite  the  various  pedagogical  innovations  there  has  not  been  enough  deeper  reform.  There  is  little 
evidence  that  the  changes  amount  to  a real  systemic  reconsideration  of  how  and  why  students  learn  or  of  how 
institutions,  not  just  faculty,  can  revise  their  approaches  to  teaching  (Lazerson,  Wagener,  & Shumanis,  2000). 

Rationale  for  Faculty  Development 

Why  is  there  a need  for  faculty  development?  A few  reasons  for  undertaking  faculty  development 
initiatives  emerged  from  the  literature.  The  primary  reason  is  to  help  faculty  to  move  their  teaching,  research,  and 
service  forward,  which  are  the  three  parts  of  the  higher  education  mission.  The  goal  of  improving  skills  and 
techniques  is  equivalent  to  innovating,  which  is  required  for  continuing  to  meet  the  needs  of  the  stakeholders  in 
higher  education,  namely  students  who  enroll  for  the  purpose  of  learning.  In  most  cases,  faculty  in  higher  education 
are  paid  for  teaching  but  rewarded  for  scholarship.  Further,  most  faculty  are'not  trained  in  instructional  design  and 
methodologies,  as  are  graduates  of  teacher  education  programs  (Noone  & Swenson,  2001).  They  learned  how  to 
teach  by  the  example  set  for  them  by  their  instructors  and  then  perhaps  modified  those  practices  based  on  actual 
classroom  experience.  As  research  on  effective  teaching  and  learning  methodologies  moves  forward,  adoption  of  the 
best  practices  by  faculty  must  move  with  it.  Instructional  technology  practitioners  help  the  internal  processes  of 
faculty  development  by  providing  the  necessary  training  and  support  for  both  pedagogical  and  technological  issues. 

Faculty  development  initiatives  have  cross-interaction  effects  with  five  key  areas  of  the  college  and 
university:  student  acceptance,  administrative  policy,  faculty  adoption,  technology  integration,  and  the  societal 
context  in  which  the  institution  functions.  Addressing  the  needs  for  faculty  development  has  a significant  holistic 
impact  on  the  institution  and  can  act  as  an  energizing  catalyst  for  the  systemic  change  and  transformation  of  higher 
education.  The  instructional  designer  operating  within  higher  education  should  develop  an  understanding  of  this 
dynamic  interaction  for  the  improvement  of  faculty  development  initiatives. 

Driving  Factor:  Students 

Today’s  students  are  more  wired,  technologically  savvy,  and  connected  than  any  previous  generation  their 
character  has  been  shaped  by  a fast-food  style,  digital  revolution  of  media  and  near  instant  access  to  information,  has 
little  desire  for  the  traditional  modes  of  teacher-centered  classroom  instruction.  Their  desires  for  technology- 
enhanced  experiences  are  not  limited  to  personal  use.  Students’  expectations  for  technology-enhanced,  practical, 
collaborative,  real-world  learning  environments  contrast  with  the  majority  of  faculty  who  still  depend  on  lectures  as 
their  prime  teaching  method  (Hansen  & Stephens,  2000;  Noone  & Swenson,  2001).  Students  are  the  consumers  of 
the  higher  education  institution’s  products.  If  teachers  continue  to  teach  in  the  same  way  that  they  have  always 
taught,  they  will  lose  the  interest  of  this  digital  generation  and  miss  the  mark  of  helping  to  educate,  and  thereby 
transform,  today’s  diverse  students.  Frand  (2000)  suggests  higher  education  needs  to  account  for  the  new  attitudes 
and  beliefs  of  students  and  transform  the  educational  experience  so  that  it  is  meaningful  to  the  information-age 
learner.  Given  an  increasing  awareness  of  duty  to  undergraduate  students,  colleges  and  universities,  particularly 
research  universities,  are  engaging  in  lively  debate  on  how  much  attention  should  be  paid  to  undergraduate 
education  (Kennedy,  1998). 

Driving  Factor:  Faculty 
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Forces  of  change  are  most  productive  when  they  originate  from  within  the  entity  that  needs  changing. 
Faculty  is  a key  subset  of  an  institution’s  administrative  body,  and  may  be  considered  the  most  significant  driving 
factor  affecting  faculty  development.  The  opportunity  for  facilitating  the  change  through  faculty  development 
programs  is  one  that  faculty  have  begun  to  take  advantage  of,  but  not  in  such  a manner  as  to  have  the  intended 
effect.  Lazerson  et  al.  (2000)  report  “...efforts  to  improve  teaching  and  learning  have  been  supported  only  in  part  by 
faculty  and  institutions  as  a whole,  with  results  that  are  neither  significant  nor  pervasive.”  When  faculty  perceive 
there  is  a need  to  change  and  they  understand  the  true  benefit  of  change  to  their  professional  development,  there  will 
be  a tremendous  shift  towards  faculty  development.  According  to  Brown  (2000),  the  shift  is  occurring  now.  Scholars 
worldwide  are  creating  a storm  of  educational  technology  experiments.  As  they  assemble  in  conferences,  hallways, 
and  special  panels,  they  are  seeking  to  engage  in  the  issues  and  opportunities  arising  from  technology-enhanced 
learning.  Faculty  seem  to  be  undertaking  fundamental  remodeling  of  their  teaching  approaches  and  giving  a 
thoughtful  consideration  of  pedagogy.  Even  though  professional  development  for  faculty  is  important,  it  is  not 
enough  to  ensure  support  for  and  adoption  of  technology  for  teaching.  It  is  a mistake  to  frame  the  issue  as  one  of 
training  faculty,  which  tends  to  put  the  "blame"  on  faculty  members  and  implies  that  they  are  the  problem  that  needs 
to  be  fixed.  Professional  development  is  the  last  stage  in  a broader,  holistic  change  process  (Bates,  1999).  Given  the 
proper  conditions  of  creative  energy  and  institutional  loyalty,  the  faculty  is  willing  to  experiment  and  to  engage 
actively  with  the  needs  of  students  (Kennedy,  1998).  In  response,  faculty  members  are  asking  for  help  from  their 
administrations  and  one  another. 

Driving  Factor:  Administration 

The  holistic  change  process  is  in  large  part,  the  purview  of  the  administration.  The  administrative  function 
of  colleges  and  universities  is  a driving  force  on  faculty  development  by  virtue  of  their  responsibility  for  setting 
policy,  managing  financial  and  capital  resources,  and  ensuring  the  ongoing  vitality  of  the  institution.  The 
administration  knows  that  faculty  is  the  key  source  of  a healthy  environment,  but  they  must  also  take  action  for 
altering  the  current  atmosphere  to  invite  change.  Transformational  change  powered  by  the  technological  revolution 
is  constrained  when  mistakenly  held  within  the  context  of  the  old  organizational  structures.  This  is  the  “mirage  of 
continuity”  that  denies  the  need  for  reorganization  of  financial  and  management  systems.  Outmoded  administrative 
units  falsely  believe  the  historic  tradition  of  knowledge  creation  and  transmission  can  be  transformed  by  the  simple 
substitution  of  digital  for  analog  technology.  A new  conception  of  the  university  is  needed  (Battin  & Hawkins, 

1998). 

Policy  should  precede  and  guide  action.  Kennedy  (1998)  suggests  that  in  order  for  the  transformation  of  a 
college  or  university  to  occur,  institutions  require  new  methods  of  making  faculty  members  feel  responsible  for  the 
institution  and  for  its  students.  One  suggested  way  is  to  develop  a more  centralized  sense  of  direction,  while  at  the 
same  time,  maintaining  a shared  governance  structure  in  which  faculty  members  feel  more  like  stakeholders.  To 
help  this  process,  institutions  must  be  more  flexible  and  responsive  to  new  needs,  trends,  and  opportunities,  by 
setting  aside  funds  for  new  initiatives  and  perhaps  most  importantly,  by  cultivating  the  spirit  of  innovation. 

Action  follows  and  supports  policy.  The  adoption  of  academic  technologies  is  a strategic  imperative  for 
higher  education.  The  first  step  in  the  process  of  reinventing  instructional  technology  is  to  convert  it  into  a strategic 
tool  tightly  incorporated  in  well-defined  and  well-researched  institutional  objectives.  Most  every  college  and 
university  mission  statement  lists  "quality  teaching  and  learning"  as  one  of  its  key  strategic  objectives,  but  have  not 
adequately  defined  the  meaning  of  "quality  learning”  with  respect  to  new  workplace  skills  and  individual  student 
needs  or  associated  the  criteria  to  particular  instructional  technology  strategies  that  can  be  used  to  achieve  them 
(Privateer,  1999). 

Driving  Factor:  Society 

Faculty  development,  as  the  primary  catalyst  for  change  in  two  core  areas  of  the  university — teaching  and 
learning — is  positioned  to  lead  the  transformation  of  the  university  to  meet  the  needs  of  the  21st  century  society 
dominated  by  electronic  technology  (Battin  & Hawkins,  1998).  Society  provides  the  context  in  which  higher 
education  institutions  exist  and  ultimately  serves.  The  relationship  is  symbiotic — society  produces  the  students  who 
matriculate  and  then  graduate  with  some  increased  capacity  to  productively  serve  society.  Kennedy  (1998)  observes 
that  society  is  paying  attention  to  higher  education  as  evident  by  media  reports  of  academic  scandal,  research 
misconduct,  and  athletic  scholarship  violations,  as  well  as  more  thoughtful  and  private  criticism  of  employers, 
government  leaders,  and  parents.  It  can  be  said  that  some  attention,  even  negative,  is  a sign  that  Americans  care 
about  colleges  and  universities. 
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Society  perceives  higher  education  as  the  archivists  of  cultural  heritage  and  conservators  of  its  history  for 
the  purpose  of  passing  both  on  to  subsequent  generations  of  students.  It  is  in  the  best  interest  of  society  that  colleges 
and  universities  are  effectively  fulfilling  these  responsibilities  (Kennedy,  1998).  These  expectations  of  a changing 
society  on  higher  education  have  implications  for  how  teaching  and  learning  is  carried  out.  Employers  are  a primary 
stakeholder  in  society  and  are  looking  for  graduates  who  are  problem-solvers,  which  require  higher  order  thinking 
skills  and  good  collaboration  skills.  In  addition,  the  changing  nature  of  society — characterized  by  eras  of  economic 
shifts  from  industrial  to  information  to  knowledge — places  pressure  on  colleges  and  universities  to  improve  the 
information  intelligence  of  its  graduates.  The  establishment  of  information  science  departments  within  universities 
in  recent  years  evidences  this  trend.  Faculty  will  require  new  skills  for  delivering,  monitoring,  and  assessing  the 
types  of  instruction  that  encourage  the  maturation  of  higher  cognitive  functions  and  better  collaboration  skills  in 
students. 

Driving  Factor:  Technology 

Technology  advancements  both  drive  and  support  faculty  development  initiatives.  Technology  is  in 
constant  change.  Each  advancement  or  application  to  education  opens  new  possibilities  for  its  adoption  and 
diffusion  in  the  teaching  and  learning  enterprise.  Faculty  should  seek  professional  development  to  better  understand 
and  possibly  integrate  technology  into  their  practice.  Like  a high-speed  train,  technology  is  a rapid  transportation 
vehicle  to  new  levels  of  learner  knowledge  construction.  Faculty  must  choose  to  ride  the  train,  step  off,  or  at  least, 
move  out  of  the  way.  Given  the  risk  and  potential  reward  of  integrating  technology  into  an  existing  academic 
paradigm,  much  more  time  and  research  will  be  needed  before  a set  of  “best  practices”  for  the  use  of  new 
technologies  in  higher  education  can  be  determined.  Meanwhile,  change  is  happening  at  a rate  not  seen  in  higher 
education  for  a long  time.  In  most  colleges  and  universities,  innovation  has  historically  been  descriptive  of  research 
and  scholarship,  not  teaching  methods.  The  new  digital  technologies  now  make  bold  and  creative  educational 
experimentation  possible  (Farrington,  1999).  With  each  new  telecommunication  innovation,  the  basic  nature  of 
learning  and  teaching  is  changing  and  creating  new  ways  to  process  and  disseminate  information.  Instructional 
technology  leaders  must  be  a part  of  the  decision  making  process  when  telecommunications  and  computing 
technologies  are  determined  (Withrow,  1994). 

The  Role  of  Instructional  Technology  and  Instructional  Designers 

By  its  innovative  nature,  instructional  technology  creates  a dynamic  for  change  wherever  it  is  properly 
used.  In  particular,  computer  and  telecommunication  technologies  forces  institutions  and  individuals  to  adapt  to  the 
revolutionary  ways  in  which  data  and  information  are  stored,  retrieved,  and  communicated.  The  traditional  tasks  of 
editorial  criticism  and  evaluation  of  course  assignments  can  now  take  place  electronically  and,  in  the  best 
circumstances,  link  professor  and  students  more  closely  for  more  of  the  work  than  ever  before.  Instructional 
technology  facilitates  the  effective  design  process  of  innovative  learning  environments  through  the  use  of  efficient 
systematic  methodologies  and  strategies. 

It  is  a positive  note  for  the  instructional  technology  field  that  instructional  designers  are  increasingly 
appearing  on  the  payrolls  of  universities,  namely  in  faculty  development  and  support  programs.  Surry  (1996)  reports 
that  instructional  designers  are  steadily  being  hired  in  higher  education  and  in  a more  recent  study,  Surry  and 
Robinson  (2001)  categorized  hundreds  of  educational  technology  job  postings.  The  instructional  design  leaders  who 
fill  these  positions  will  need  to  have  supplemental  skills,  such  as  project  management  and  facilitating  change  to 
complement  their  ISD  skills.  Their  backgrounds  and  experiences,  more  than  any  other  professional  field,  qualify 
them  to  handle  the  dynamic  nature  of  change  in  educational  technology  and  its  application  to  learning  processes  and 
teaching  strategies. 

As  the  transformation  progresses,  faculty  will  continue  to  need  training  and  refreshers  in  the  skills  that  are 
essential  for  teaching  and  learning  with  technology;  support  during  the  development  process;  and  advice  for  the 
effective  integration  of  media  and  information  technologies.  Instructional  technology  practitioners  need  to  be 
prepared  for  these  challenges.  Duderstadt  (1999)  said  so  well,  “The  real  question  is  not  whether  higher  education 
will  be  transformed  but  rather  how  and  by  whom''  Instructional  designers  are  the  professionals  prepared  to  be 
involved  in  the  transformation  and  should  seek  leadership  positions  in  order  to  positively  affect  organizational 
change  during  the  transformation.  In  addition,  it  will  become  incumbent  upon  them  to  make  contributions  to  the 
instructional  technology  knowledge  base  regarding  research,  instruction,  process,  and  outcomes  of  faculty 
development  initiatives. 
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Conclusion 


To  secure  future  viability  and  fulfill  its  tripartite  mission  of  teaching,  research,  and  service,  higher 
education  must  choose  a better  strategic  path.  If  they  want  to  reinvent  themselves,  they  have  to  take  a long  and  hard 
strategic  look  into  how  their  delivery  of  instruction  conflicts  with  the  cognitive  potentials  of  contemporary 
information  technologies.  Instructional  designers  are  uniquely  qualified  to  take  on  significant  leadership  roles  within 
higher  education  to  manage  faculty  development  programs.  Faculty  development  is  a component  of  the  process  of 
lifelong  learning  for  professors  and  educators  in  higher  education  and  a key  component  of  managing  the 
transformational  changes  taking  place  in  higher  education  over  the  next  decade.  The  key  concept  of  faculty 
development  as  a transforming  agent  of  colleges  and  universities  is  accepting,  understanding,  and  managing  the 
dynamic  changes  brought  about  by  the  five  external  and  internal  factors:  students,  faculty,  administration,  society, 
and  technology.  These  factors  drive  the  need  for  faculty  to  integrate  technology  into  their  curriculum  and  utilize  new 
instructional  methodologies,  strategies,  and  techniques. 

Students’  expectations  for  technology-enhanced,  practical,  collaborative  learning  environments  contrast 
with  the  majority  of  faculty  who  still  depend  on  lectures  as  their  prime  teaching  method  (Hansen  & Stephens,  2000). 
If  teachers  continue  to  teach  in  the  same  way  that  they  have  always  taught,  they  will  miss  the  mark  of  helping  to 
educate  (transform)  today’s  diverse  students  and  make  the  educational  experience  meaningful  to  the  information-age 
learner  (Frand,  2000). 

The  opportunity  for  facilitating  the  change  through  faculty  development  programs  is  one  that  faculties  have 
begun  to  take,  but  with  modest  results  that  are  neither  significant  nor  pervasive  (Lazerson,  et  al.,  2000).  When 
faculty  perceives  there  is  a need  to  change  and  they  understand  the  true  benefit  of  change  to  their  professional 
development,  there  will  be  a tremendous  shift  towards  faculty  development.  Even  though  professional  development 
for  faculty  is  important,  it  is  not  enough  to  ensure  support  for  and  adoption  of  technology  for  teaching.  A holistic 
change  is  needed  to  support  faculty  adoption. 

The  holistic  change  process  is  primarily  the  leadership  responsibility  of  the  institution’s  administration.  The 
administrative  function  of  colleges  and  universities  is  a top-down,  driving  force  on  faculty  development 
characterized  by  setting  policy,  managing  financial  and  capital  resources,  and  ensuring  the  ongoing  vitality  of  the 
institution.  The  administration  knows  that  faculty  is  the  key  source  of  a healthy  environment,  but  they  must  also 
recognize  the  need  to  alter  the  current  atmosphere  to  invite  change.  Transformational  change  should  not  be 
constrained  within  the  context  of  the  old  organizational  structures.  Battin  and  Hawkins  (1998)  refer  to  this  as  the 
“mirage  of  continuity”  that  denies  the  need  for  reorganization  of  financial  and  management  systems.  Historic 
tradition  of  knowledge  creation  and  transmission  must  be  replaced  with  a new  conception  of  the  university. 

The  expectations  of  a changing  society  on  higher  education  have  implications  for  how  teaching  and 
learning  is  carried  out.  Employers  are  looking  for  graduates  who  are  problem-solvers,  which  is  challenging  to  teach 
in  every  discipline.  Nevertheless,  the  changing  economic  nature  of  society  places  pressure  on  colleges  and 
universities  to  improve  the  information  intelligence  of  its  graduates.  Faculty  will  require  new  skills  for  delivering 
the  types  of  instruction  that  encourage  the  maturation  of  higher  cognitive  functions  and  better  collaboration  skills  in 
students. 

Given  the  risk  and  potential  reward  of  integrating  technology  into  an  existing  academic  paradigm,  more 
research  on  the  best  uses  of  the  new  technologies  is  needed.  New  digital  technologies  allow  for  bold  and  creative 
educational  experimentation.  Instructional  technology,  by  its  innovative  nature,  facilitates  change  wherever  it  is 
appropriately  used.  To  secure  future  viability  and  fulfill  its  mission  of  teaching,  research,  and  service,  higher 
education  must  choose  a better  strategic  path.  If  they  want  to  reinvent  themselves,  they  have  to  take  a long  and  hard 
strategic  look  into  how  their  delivery  of  instruction  conflicts  with  the  cognitive  potentials  of  contemp  orary 
information  technologies.  The  key  concept  of  faculty  development  as  a transforming  agent  of  colleges  and 
universities  is  accepting,  understanding,  and  managing  the  dynamic  changes  brought  about  by  the  five  factors: 
students,  faculty,  administration,  society,  and  technology. 

The  instructional  designer  is  one  of  the  best  prepared  education  professionals  to  provide  training  in  the 
skills  that  are  essential  for  teaching  and  learning  with  technology,  to  provide  support  during  the  instructional 
development  process,  and  to  offer  pedagogically  sound  guidance  for  the  effective  integration  of  media  and 
information  technologies.  Instructional  technology  practitioners  should  seek  leadership  positions  in  faculty 
development  programs.  The  result  is  likely  to  positively  affect  implementation  of  ISD  practices,  theories,  and 
strategies  into  faculty  development.  The  instructional  designer  is  a versatile  education  professional  that  can  offer 
valuable  skills  and  facilitate  appropriate  use  of  instructional  systems  design  for  improving  teaching  and  learning 
methodologies  in  faculty  development  programs.  In  this  capacity,  instructional  designers  can  play  a key  leadership 
role  in  the  transformation  of  higher  education. 
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Co-Inquiry  Approach  to  Learning  and  Using  Hypermedia 


Aaron  Doering 
Richard  Beach 
University  of  Minnesota 

Abstract: 

This  article  analyzes  the  uses  of  various  technologies  to  enhance  Uteracy  practices  using  a multi-genre 
writing  project  with  pre-service  teachers  and  middle  school  students.  Twenty-seven  English  pre-service 
teachers,  simultaneously  enrolled  in  a methods  and  a technology  course,  collaborated  with  middle  school 
students  using  asynchronous  web  discussion  to  develop  hypermedia  projects  that  fostered  and  promoted  the 
use  of  technology  as  a tool.  These  tools  mediated  the  uses  of  various  literacy  practices  within  the  larger 
activity  system  of  teacher  education,  whose  object  is  to  assist  teachers  to  acquire  those  practices  involved 
in  working  effectively  with  students.  Qualitative  data  were  collected  through  analyzing  preservice 
teachers  ’ development  of  Story  space  hypermedia  projects,  the  use  of  asynchronous  discussion  with  their 
middle  school  students,  and  participation  on  a WebCT  bulletin-board  discussion.  The  hypermedia 
productions  with  middle  school  students  helped  the  preservice  teachers  learn  how  to  model  the  literacy 
practices  of  making  intertextual  or  hypertextual  links.  The  web-based  communication  with  students  helped 
preservice  teachers  develop  relationships  with  students  in  the  absence  of  face-to-face  interaction.  And, 
through  participation  in  the  WebCT  bulletin  board,  preservice  teachers  employed  different  literary 
practices  ranging  from  the  display  of  spontaneous  thinking  to  engaging  in  word/role  play. 

Literacy  Practices  and  Technology  Tools 

A preservice  teacher  and  a middle -school  student  are  exchanging  messages  on  a web-based  bulletin  board 
about  a biography  project  they  are  working  on  together  on  the  topic  of  Princess  Diana.  The  student  posted  the 
following  message: 

Last  night  1 went  on  the  Internet  and  found  a lot  of  stuff  like  her  will,  and  her  divorce  papers  and 
some  poems  some  people  wrote  about  her.  I also  fond  some  pictures  of  when  she  was  younger. 

The  preservice  teacher  responded: 

Last  night  1 bought  a couple  of  books  about  Princess  Diana  that  were  on  sale  at  the  bookstore.  One 
contains  a bunch  of  short  little  memories  of  her  written  by  all  sorts  of  people  that  knew  her  in  her 
lifetime.  I will  also  print  at  least  2 articles  from  the  Internet  that  will  be  helpful  (not  too  long)  for 
us  to  think  about  what  we  want  to  write  about. 

See  you  Wednesday. 

This  on-line  exchange  was  part  of  a project  involving  preservice  English  teachers  working  in  a semester- 
long  practicum  experience  with  a group  of  middle  -school  students,  a project  that  involved  extensive  uses  of 
technology.  Their  on-line  exchange  entails  uses  of  literacy  practices  such  as  sharing  information  and  planning 
activities,  practices  central  to  a co-inquiry  writing  project.  This  project  represents  the  increasing  use  of  technology 
as  a tool  for  linking  adults  with  students  in  schools,  an  approach  that  is  highly  relevant  to  teacher  education. 

In  many  teacher  education  programs,  in  addition  to  their  student  teaching,  preservice  teachers  are  required 
to  complete  practicum  experiences  that  involve  minimal  face-to-face  interaction  with  students.  Technology  can 
enhance  preservice  teachers’  interaction  with  students,  as  well  as  providing  students  with  positive  learning 
experiences  through  technology.  For  example,  in  the  “Fifth  Dimension”  after-school  computer-mediated  program 
operated  by  the  University  of  California,  San  Diego,  participation  in  an  elaborate  set  of  computer  games  and 
activities  resulted  in  increased  student  engagement,  participation,  and  learning  within  a community  (Cole,  1999).  In 
this  program.  University  undergraduates  serve  as  “Wizards”  who  guide  students  through  a “maze”  of  activities 
based  on  the  students’  zone  of  proximal  development. 

Educators  are  also  employing  web-based  tools  to  foster  on-line  discussions  between  teachers  regarding 
issues  faced  in  their  programs  or  in  the  classroom.  The  Inquiry  Page  housed  at  the  University  of  Illinois 
<http://inquiry.uiuc.edu7>  is  designed  to  help  teachers  share  teaching  successes  and  collective  expertise  (Bruce  & 
Davidson,  1996;  Bruce  & Easley,  2000).  Teachers  engage  in  mutual  inquiry  through  their  access  to  resources  on 
teaching  and  learning,  articles,  project  links,  curriculum  units,  and  content  resources.  Users  of  the  site  are 
themselves  the  developers  who  reconstruct  the  tool  as  they  use  it.  Participants  may  also  share  video,  photos. 
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graphics,  texts  showing  people  engaged  in  inquiry  in  different  settings  and  access  resources  involving  a dynamic 
incorporation  (using  Digital  Windmill)  of  the  Open  Directory  category  on  Inquiry  Based  Learning. 

This  site  represents  new  generation  of  web  design  that  serves  the  social  needs  of  teachers  to  mutually 
engage  them  in  co- inquiry  about  problems,  issues,  or  dilemmas.  Research  on  uses  of  these  sites  indicates  the 
importance  of  quality  of  the  social  interaction  in  this  on-line  co-inquiry.  For  example,  Barah  and  Schatz  (2001) 
analyzed  the  development  of  a web-based  learning  site  designed  to  foster  sharing  of  inquiry -instruction  ideas  by 
Indiana  math  and  science  teachers  in  terms  of  the  components  of  evolving  activity  systems.  This  web  site  was 
initially  designed  as  a tool  by  University  educators  to  achieve  the  object  of  more  discussion/sharing  about  inquiry 
instruction  with  the  outcome  being  improved  understanding  of  inquiry -based  instruction.  However,  given  the  lack 
of  participation,  the  University  educators,  along  with  teacher  participants,  shifted  the  focus  of  the  web  site  to 
emphasize  participants’  mutual  collaboration  at  the  site  around  inquiry -based  math/science  instruction. 

In  this  report,  we  examine  the  various  literacy  practices  that  were  fostered  through  the  uses  of  technology 
tools  that  included  web-based  bulletin-board  exchanges  and  hypermedia  productions.  We  hope  to  demonstrate  that 
technology  tools  can  serve  to  mediate  and  foster  the  development  of  a range  of  different  literacy  practices  within  a 
teacher  education  program. 

Technology  Tools  as  Mediating  Literacy  Practices 

Social-cultural  activity  theory  of  learning  (Cole,  1996;  Engestrom,  1987;  Wertsch,  1998)  posits  that 
learning  occurs  through  social  uses  of  various  tools  -language,  signs,  images,  texts,  as  well  as  technology  tools. 
Activity  theorists  believe  that  people  learn  the  uses  of  these  tools  by  learning  how  they  are  linked  to  the  objects  or 
outcomes  driving  a specific  activity  within  an  “activity  system.”  Russell  (1997)  defined  an  activity  system  as:  “any 
ongoing,  object-directed,  historically  conditioned,  dialectically  structured,  tool-mediated  human  interaction.  Some 
examples  are  a family,  a religious  organization,  a school,  a discipline,  a research  laboratory,  and  a profession”  (p, 
510). 

Central  to  activity  theory  of  learning  is  the  idea  that  these  tools  function  to  mediate  learning  of  literacy 
practices  (Bruce  & Levin,  1997).  Students  learn  to  use  a range  of  tools  to  engage  in  these  literacy  practices.  Work 
in  the  field  of  “distributed  cognition”  (Hutchins,  1993)  posits  that  certain  practices  associated  with  an  activity 
become  embodied  or  “distributed”  in  tools.  For  example,  navigational  instruments  are  used  to  capture  what  is 
known  about  navigating  the  seas.  They  then  serve  as  tools  that  guide  a ship  based  on  human  knowledge  about 
navigation.  Similarly,  expert  computer  systems  are  built  on  experts’  knowledge  about  a certain  phenomena  such  as 
diagnosing  a particular  disease.  Tools  are  therefore  used  within  an  activity  to  function  as  extensions  of  certain 
practices  involved  in  an  activity  (Vygotsky,  1978),  We  turn  now  to  discuss  examples  of  how  learning  the  following 
literacy  practices  are  mediated  through  technology  tools: 

Defining  intertextual  connections.  One  basic  literacy  practice  involves  defining  intertextual  links  between 
texts.  In  defining  intertextual  links,  people  define  connections  between  texts  in  terms  of  similar  images,  characters, 
topics,  or  themes.  Roland  Barthes  (1970)  argues  that  “Every  text,  being  itself  the  intertext  of  another  text,  belongs 
to  the  intertextual. ..the  quotations  from  which  a text  is  constructed  are  anonymous,  irrecoverable,  as  yti  already 
react"  (p.  443). 

Students  are  engaged  in  making  intertextual  links  in  through  multi-genre  writing  about  a topic,  an  approach 
currently  popular  in  secondary  writing  instruction  (Romano,  2000).  Multi-genre  writing  involves  using  a range  of 
different  types  of  genres — reports,  poems,  letters,  diaries,  stories,  advertisements,  field  notes,  photos,  drawings,  etc. 
to  e^lore  different  aspects  of  and  perspectives  on  a topic.  Connecting  these  disparate  genre  types  requires  the 
ability  to  determine  how  different  types  of  texts  yield  different  perspectives  on  the  same  topic  or  phenomenon. 

One  technology  tool  that  mediates  the  practice  of  making  intertextual  links  is  hypermedia.  Hypermedia 
functions  as  a tool  by  combining  hypertext  (texts  linked  together  by  multi-linear  nodes)  and  multimedia  (photos, 
video,  art,  audio,  text,  etc.)  to  produce  an  interactive  media  experience  for  participants  (Jonassen,  2000;  Landow, 
1997).  Because  hypertext  allows  participants  to  choose  optional  paths  through  multimedia,  participants  can  both 
construct  and  respond  to  hypermedia  interactively.  Students  often  respond  positively  to  hypermedia  texts  because  it 
is  consistent  with  their  everyday  experiences  with  multi-modal  environments  that  combine  images,  animation, 
video,  music,  and  texts  (Myers  & Beach,  2001). 

In  an  essay  about  the  pedagogical  implications  of  this  shift  towards  hypermedia.  Jay  Bolter  (1998)  argues 
that  hypermedia  challenges  the  traditional  emphasis  in  literacy  instruction  on  understanding  or  producing  unified, 
coherent  texts  based  on  a definitive,  single  perspective.  He  calls  for  an  alternative  focus  on  teaching  a "rhetoric  of 
expectations  and  arrivals"  (p.  10)  that  help  students  understand  where  certain  links  may  take  them  and  how  they 
should  respond  to  where  they  arrive.  And,  given  the  important  role  of  graphic  representations  in  hypermedia,  he 
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posits  the  need  for  often-marginalized  art  and  video-production  instruction  to  help  students  respond  critically  to 
images. 

Producing  hypermedia  texts  using  tools  such  as  Storyspace  HyperStudio™,  HyperCard™,  and  various 
web  authoring  programs,  involves  defining  interte?tual  links  between  a range  of  different  types  or  genres  of  texts 
(McKillop  & Myers,  1999;  Myers,  Hammett  & McKillop,  1998;  2000).  For  example,  high  school  students 
represented  their  experiences  with  peers  through  combining  photos,  music,  video  clips,  and  texts  to  interpret  short 
stories  (Beach  & Myers,  2001;  http://www.ed.psu.edu/k-12/socialworlds/). 

Hypermedia  can  also  assist  in  organizing  links  around  central  themes  or  topics  in  writing  instruction. 
Analysis  of  first-year  college  writing  class  students’  construction  of  hypertexts  indicated  that  students  structured 
information  around  central  ideas  and  illustrated  that  idea  through  links  to  other  texts  or  graphics  (Duguay,  1999). 
Using  the  hypertext  as  a tool,  helped  students  define  links  between  diverse  parts  of  their  hypertext  because  the  links 
made  it  visually  easier  to  connect  the  ideas. 

Researchers  have  also  examined  the  nature  and  types  of  links  constructed  in  hypermedia  production,  as 
well  as  the  social  motivation  to  construct  these  links  within  the  classroom  as  an  activity  system.  In  one  study,  16 
seventh  graders  1 8 preservice  teachers  used  StorySpaceTM  to  combine  original  poems,  images,  and  QuickTime 
movies  to  explain  the  various  literacy  devices  used  in  poetry  (McKillop  & Myers,  1999).  The  types  of  links 
employed  in  the  hypermedia  productions  were  analyzed  in  terms  of  their  functions — an  “iconic  function”  was  used 
to  illustrate  another  text,  an  “indexical  function”  was  used  to  extend  a text  to  show  shared  meaning,  and  a “symbolic 
function”  was  used  to  question  the  meaning  of  a text  which  resulted  in  a greater  understanding  of  or  a critical 
analysis  of  a text.  Most  of  the  seventh  graders’  links  served  as  iconic  illustrations  of  ideas  in  poems.  There  were 
far  fewer  instances  of  links  reflecting  critical  analysis,  for  example,  when  students  juxtaposed  texts  to  generate 
contested  meanings.  The  undergraduates  were  more  likely  to  employ  links  serving  a “symbolic  function”  that 
involved  critical  analysis  of  texts.  This  study  suggests  that  users  employ  links  for  different  purposes  representing 
different  levels  of  critical  thinking. 

Ryan  (1999)  examined  in  college  students’  construction  of  hypermedia  links  using  HyperCardTM  to  write 
a “Literary  Journal”  biography  of  an  American  author  based  on  a range  of  different  sources  and  information  about 
that  author’s  life,  as  well  as  comments  on  other  students’  work  and  supplementary  material.  In  contrast  to  the  essay 
format  that  often  constrains  exploration  of  alternative,  conflicting  perspectives,  the  hypertext  format  fostered 
exploration  of  alternative,  conflicted  perspectives  about  an  author’s  life  that  resisted  closure. 

Nancy  Patterson's  (Patterson,  2000)  middle  school  students  at  Portland  Middle  School,  Portland,  Michigan, 
used  Storyspace™  to  construct  hypertexts  based  on  research  on  American  history  and  culture 
(http://angelfire.com/mi/patter/america.html).  Students  created  hypertext  narratives  with  links  to  information  about 
slavery.  As  Patterson  notes  (http ://www.npatterson.net/mid.html ).  working  with  Storyspace™  shifted  students  away 
from  simply  rehashing  information  about  persons  to  understanding  people  and  events  as  shaped  by  historical  and 
cultural  forces. 

Posing  questions.  Another  literacy  practice  involves  posing  questions  related  to  exploration  of  issues, 
topics,  concerns,  or  dilemmas  (Beach  & Myers,  2001;  Short  & Harste,  1996;  Smithson  & Dias,  1996).  In 
teacher/students  journal  dialogue  exchange,  teachers  pose  questions  designed  to  encourage  students  to  elaborate  on 
their  answers  or  explore  other  perspectives,  modeling  heuristics  for  exploring  topics.  Overtime,  students 
internalize  these  questions  and  employ  them  in  their  own  writing,  resulting  in  increased  elaboration  in  their  writing 
(Peyton  & Staton,  1993).  Computer-mediated  written  communication  between  teacher  and  student  can  serve  as  a 
tool  for  teachers  to  engage  in  similar  dialogue-journal  writing  modeling  of  question-asking  (Beach  & Lundell, 

1998). 

Adopting  multiple  voices  and  perspectives.  Another  basic  liteiacy  practice  involves  adopting  multiple 
voices  and  perspectives  through  making  “double -voice”  intertextual  references  or  evoking  or  mimicking  the 
languages  or  styles  from  other  texts  or  worlds  (Bakhtin,  1981;  Knoeller,  1998).  Speakers  and  writers  employ  these 
intertextual  references  to  establish  social  relationships  and  identities  (Bloome  & Egan-Robertson,  1993).  Through 
interaction  with  others,  participants  construct  identities  by  performing  in  ways  that  position  them  in  relation  to 
others'  positions~"it  is  in  the  connection  to  another's  response  that  a performance  takes  shape"  (McNamee,  1996,  p. 
150).  As  Bakhtin  (1981)  argued  in  his  concept  of  "answerability,"  people's  utterances  reflect  their  relationships 
with  others’  potential,  anticipated  reactions  to  their  utterances.  In  participating  with  a range  of  diverse  perspectives 
and  voices  in  a computer-mediated  context,  students  learn  to  consider  alternative  perspectives  different  from  their 
own  (Taylor,  T992).  The  more  open  students  are  to  experimenting  with  alternative  ways  of  being  and  knowing,  the 
more  open  they  are  to  entertaining  alternative  values,  as  opposed  to  a rigid,  monologic  perspective  on  the  world 
(Lewis  & Fabos,  1999). 
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Adopting  a collaborative,  inquiry  stance.  In  conducting  discussions  with  students,  teachers  attempt  to 
adopt  a collaborative,  exploratory  stance  that  serves  to  invite  mutual  exploration  with  students.  Adopting  this 
stance  requires  teachers  to  balance  their  status  as  authority  figure  with  the  need  to  establish  a relationship  with 
students.  As  Deborah  Tannen  (1984)  notes,  in  this  negotiation,  participants  may  use  conversation  as  “ symmetrica  1”- 
-to  maintain  equal  status,  or,  as  “asymmetrical”  -to  establish  a dominant/subordinate  relationship.  On-line 
discussions  s erve  to  minimize  some  of  the  nonverbal  aspects  creating  “asymmetrical”  status  differences  in  face-to- 
face  interactions  (Walther,  1996).  Differences  in  uses  of  “asymmetrical”  practices  may  also  be  related  to  gender 
stances.  Analysis  of  college  classroom  discussions  indicated  that  females  were  more  likely  to  employ  “task- 
continuative”  practices  comprised  of  questions  and  answers,  validation  of  others’  comments,  back-channel 
comments,  repetition,  extension,  supportive  laughter,  extended  development  or  talk  than  were  males  (Bergvall  & 
Remlinger,  1996). 

The  ability  to  adopt  collaborative,  exploratory  stance  depends  on  participants’  willingness  to  be  open  to 
entertaining  others’  beliefs  as  valid  and  rational,  something  that  what  Donald  Davidson  (1984)  refers  to  as  the 
“principle  of  charity”  (p.  126).  As  Porter  (2001)  notes,  “because  communicators  cannot  assume  shared 
meanings...,  they  must  assume  a shared  world;  if  they  assume  that  they  share  neither  a language  nor  a world,  there 
would  be  no  possibility  for  communication”  (p.  586).  It  also  requires  the  ability  to  frame  statements  of  beliefs  or 
opinions  as  tentative  hunches  or  hypotheses — what  Davidson  (1984)  refers  to  as  “passing  theories”  (p.  45).  The 
concept  of  “passing  theories”  refers  to  the  idea  that  participants  are  willing  to  modify  their  established  “prior 
theories”  to  be  open  to  entertaining  and  integrating  others’  beliefs  into  one’s  own  beliefs  (Dasenbrook,  2000).  In 
classroom  discussions  of  literature,  when  students  framed  a new  topic  in  a tentative,  exploratory  manner,  other 
students  were  more  likely  to  follow  up  on  that  topic  than  when  the  topic  was  framed  in  a definitive  manner  (Beach 
& Phinney,  1998).  Synchronous  computer-mediated  classroom  interaction  in  a seventh-grade  classroom  served  to 
foster  students’  mutual  exploration  of  tentative  ideas  and  perspectives  because  they  were  simultaneously 
brainstorming  together  in  the  same  chat  site;  adopting  a hard-line  stance  was  socially  unacceptable  in  this  exchange 
(Beach  & Lundell,  1998). 

This  research  indicates  that  a range  of  different  literacy  practices  can  be  fostered  through  uses  of 
technology  tools.  This  raises  the  question  as  to  whether  technology  tools  can  be  used  in  a teacher  education  activity 
system  whose  object  is  to  foster  preservice  teachers’  ability  to  acquire  and  teacher  these  literacy  practices. 

Preservice  English  Teachers’  Participation  in  a Co-Inquiry  Multi-Genre  Writing  Project 

This  research  project  examined  the  question  as  to  how  one  group  of  preservice  teachers  used  technology 
tools  to  acquire  various  literacy  practices  involved  in  working  with  middle -school  students  in  a multi-genre  writing 
project. 

The  participants  in  this  project  were  27  preservice  English  teachers  enrolled  in  a composition-methods 
course  taught  by  Beach  and  an  instructional  technology  course  taught  by  Doering  in  the  Fall  Semester,  2000  at  the 
University  of  Minnesota.  Preservice  teachers  [hereafter  “teachers”]  in  the  composition  methods  course  learned 
various  strategies  for  engaging  in  inquiry -projects  and  for  teaching  multi-genre  writing.  The  purpose  of  the 
instructional  technology  course  was  to  help  teachers  acquire  a set  of  technology  tools  they  could  employ  in  teaching 
English. 

In  conjunction  with  these  courses,  participants  were  engaged  in  a semester-long  practicum  experience  in  a 
magnate  middle  school  that  draws  students  from  a wide  range  of  both  urban  and  suburban  districts  in  the  St.  Paul, 
Minnesota  area.  The  school  curriculum  is  organized  around  interdisciplinary  inquiry  projects  in  which  students  are 
engaged  in  constructivist  exploration  of  topics  across  different  subjects.  The  students  represented  a wide  range  of 
socio-economic  backgrounds  and  ability  levels,  with  many  students  testing  at  a relatively  low  reading  level.  The 
teachers  each  worked  during  weekly  visits  with  one  or  two  students  in  each  of  two  different  class  periods. 

A multi-genre  writing  project.  The  teachers  and  middle -school  students  worked  together  on  a multi-genre 
project  involving  writing  a biographical  sketch,  a newspaper  report,  and  a narrative  about  famous  people  ranging 
from  Martin  Luther  King  Jr.  to  Princess  Diana.  They  conducted  research  about  their  person  using  the  Web  and  other 
sources  based  on  questions  posed  about  the  person,  generating  information  they  used  to  write  a biographical  sketch. 
Students  then  wrote  a newspaper  article  about  some  aspect  of  or  even  in  the  person’s  life  employing  ClarisWorks  to 
create  a news  article  format.  The  project  concluded  with  students  writing  a fictional  narrative  about  their  person  in 
which  they  adopted  that  person’s  or  another  person’s  first-person  point  of  view  to  describe  some  event  in  the 
person’s  life.  This  required  students  to  imagine  the  person’s  subjective  experience  in  an  event,  along  with 
descriptions  of  dialogue;  setting;  and  the  person’s  feelings,  attitudes,  and  beliefs  about  the  event. 
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For  the  final  presentations  of  their  multi-genre  projects,  the  students  shared  the  results  of  their  work  in  short 
ten-minute  presentations  in  small  groups.  Students  employed  a range  of  multi»modal  presentations  acting  out  a 
scene  from  their  lives,  a skit,  interview  the  person,  a piece  of  art  in  the  person’s  form;  an  overhead,  slide 
presentation,  news  report/sportscast,  and  dramatic  reading. 

Hypermedia  production.  As  part  of  their  instructional  technology  class,  the  teachers  created  their  own 
hypermedia  production  based  on  their  students’  multi-genre  writing.  They  used  Storyspace^  (Bolter,  Smith,  & 
Joyce,  1990)  as  a tool  to  develop  and  link  multimedia  material  within  windows  that  can  include  or  be  embedded  in 
other  hierarchical  windows.  (Given  the  lack  of  access  to  computers  in  the  middle  school,  and  the  expense  of  the 
Storyspace™  software,  the  teachers,  in  discussion  with  their  middle -school  students,  developed  the  hypermedia 
versions  of  the  multi-genre  writing  at  their  University  site.  In  an  ideal  situation,  the  teachers  and  the  students  would 
have  developed  the  hypermedia  at  the  middle  -school  site.) 

Constructing  the  hypermedia  production  to  share  with  their  students  involved  a shift  in  role  for  the  teachers 
from  purveying  knowledge  to  demonstrating  their  “knowledge  by  design”  (Perkins,  1986).  This  change  in  learning 
when  using  hypermedia  sometimes  causes  problems  as  learners  struggle  to  integrate  the  information  they  are 
learning  into  a hypertext  document  (Jonassen,  2000).  To  explore  their  knowledge  as  related  to  their  audience,  they 
initially  developed  concept  maps  using  Inspiration  ™ to  represent  their  knowledge  prior  to  creating  the  hypermedia 
production.  These  concept  maps  were  used  as  guides  to  help  the  teachers  choose  what  links  they  believed  were 
important  as  well  as  what  of  media  they  may  want  to  employ  (graphics,  video,  sounds)  to  represent  their 

knowledge  in  StorySpace  ^ 

The  hypermedia  productions  were  analyzed  by  the  investigators  in  terms  of  the  types  of  texts — images, 
written  texts,  sounds,  etc.,  teachers  included  in  their  productions,  as  well  as  the  types  of  links  they  employed  in 
connecting  these  texts. 

Web-based  teacher /student  communication . As  part  of  a federally  funded  technology-development 
program,  an  asynchronous  Web-based  teacher/student  communication  site  was  created  to  foster  communication 
between  the  teachers  and  students  during  the  time  when  they  were  not  working  with  each  other  in  school.  To 
address  potential  security  and  privacy  issues,  pupils  would  click  on  the  name  of  their  assigned  student  and  then 
engage  in  conversation  about  their  projects  or  personal  matters.  Only  the  pupils  assigned  to  the  teachers  could 
access  those  particular  teachers.  Because  the  communications  were  asynchronous,  teachers  and  the  middle  school 
students  could  post  and  respond  to  questions  relating  to  their  cooperation  on  the  project  at  any  time. 

Transcripts  of  the  web-based  communications  were  analyzed  in  terms  of  the  amount  of  participation  as 
determined  by  the  number  of  comments  employed,  defined  in  terms  of  a complete  thought  unit,  a procedure 
employed  by  Diane  Schallert  in  her  research  on  on-line  communication  (Schallert  et  al,  2001).  Each  “thought-unit” 
was  also  analyzed  using  a constant  comparative  method  (Glaser  & Strauss,  1967)  to  guide  the  development  of  the 
significant  categories  and  patterns  in  the  data  in  terms  of  the  types  of  topics  discussed  and  the  literacy  practices 
enp  loyed.  The  types  of  topics  and  practices  were  then  crosschecked  with  an  experienced  English  teacher  for  further 
verification  (Merriam,  1998). 

WebCT  bulletin-board  discussion . The  teachers  also  participated  in  an  asynchronous  discussion  on  the 
course  WebCT  site.  For  this  site,  teachers  were  asked  by  the  course  instructor  to  make  at  least  one  posting  a week; 
they  were  told  that  they  could  respond  to  topics  or  issues  in  the  course  discussions,  readings,  or  practicum 
experiences,  as  well  as  other  topics  outside  the  course.  The  instructor  hoped  that  through  participation  in  this 
bulletin  board  exchange,  students  would  gain  some  experience  with  uses  of  a bulletin  board  as  a learning  tool  for  use 
in  their  own  future  teaching.  The  instructor  also  hoped  that  the  students  would  acquire  an  understanding  of  how 
writing  is  driven  by  social  purposes  or  needs  related  to  participating  in  a community  constituted  through  a bulletin 
board  exchange.  Transcripts  of  the  WebCT  discussion  were  analyzed  in  terms  of  the  types  of  literacy  practices 
employed  in  the  exchanges  using  the  same  analysis  methods  employed  with  analysis  of  the  teacher/student 
interactions. 

Results 

Hypermedia  Productions 

Development  of  the  initial  and  following  nodes.  Analysis  of  the  hypermedia  productions  based  on  the 
students  multi-genre  writing  projects  indicated  that  80  percent  of  the  teachers  began  their  multimedia  development 
with  a picture  of  the  person  with  links  to  the  “major  nodes”  or  events  of  the  person’s  life.  It  was  these  major  events 
that  lent  themselves  to  links  where  the  students  explained  the  person  in  more  detail  using  various  medias.  For 
example,  one  student  studying  Martin  Luther  King,  Jr.  began  their  multimedia  development  with  a picture  of  Martin 
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Luther  King,  Jr.  with  four  links  underneath  the  picture  to  take  them  to  nodes  about  “Enemies  and  Resistance,” 
“Awards  and  Supporters,”  “Biographical  Information,”  and  “Civil  Rights  Efforts.”  Each  one  of  these  four  major 
nodes  had  a short  written  description  that  explained  Martin  Luther  Kings  Jr’s  relationship  to  each  node.  In  the 
“Civil  Rights  Efforts”  node,  the  teacher  developed  five  sub-nodes  that  described  Martin  Luther  King  Jr.’s  efforts. 
These  nodes  included  “Sit-in  Demonstrations,”  “Passive  Resistance,”  “Montgomery  Bus  Boycott,”  “Writings,”  and 
“Marches  and  Speeches.”  Within  each  of  these  nodes,  the  teacher  used  images,  texts,  or  clips  to  represent  the  civil 
rights  theme.  Within  the  “Writings”  node,  the  teacher  listed  and  included  writings  from  Martin  Luther  King  Jr’s 
books.  These  writings  were  obtained  through  searching  the  Internet  and  incorporated  within  a separate  “exploding” 
Storyspace  ^ node.  To  represent  the  “Montgomery  Bus  Boycott,”  the  teacher  decided  to  use  a video  clip  she  also 
obtained  from  the  Internet  and  to  represent  the  “Passive  Resistance”  theme;  she  scanned  in  pictures  that  were 
obtained  through  a family  trip.  Teachers  integrated  a wide  range  of  media  texts  into  their  productions,  frequently 
selecting  texts  most  readily  available  from  the  World  Wide  Web. 

Other  teachers  chose  to  limit  their  biography  to  detailed  portrayals  of  a specific  period  in  person’s  life 
because  information  about  that  period  was  more  available  and  they  preferred  to  develop  a specific  aspect  of  a 
person’s  life.  As  one  teacher  indicated  in  her  learning  log,  she  would  rather  research  the  person’s  life  “using  depth, 
rather  than  breadth,  and  develop  an  understanding  that  was  more  meaningful.” 

Analysis  of  the  links  employed.  The  27  projects  indicated  that  the  most  common  approach  to  linking  was 
directly  from  a picture  or  words  that  described  themes  for  analysis  placed  under  a picture.  For  example,  when 
placing  a picture  of  the  “Montgomery  Bus  Boycott”  in  a node,  a reader  would  click  on  the  picture  to  move  to  an 
explanation  of  the  boycott  and  then  link  back  to  another  node  with  another  theme  when  finished.  Sixty-five  percent 
of  the  teachers  used  this  approach  of  simply  linking  images  and  texts  without  use  of  hypertext  links  from  individual 
words. 

The  other  thirty-five  percent  of  the  teachers  used  hypertext  links  in  which  certain  words  were  linked  to 
other  words  or  texts.  One  teacher  described  the  life  of  John  F.  Kennedy  and  made  links  to  words  that  they  found 
most  difficult  for  a reader  to  outside  nodes  that  either  described  the  word  through  text,  a graphic,  or  both.  The 
words  that  were  linked  were  words  that  the  teachers  believed  would  improve  the  students’  reading  experience  or  that 
they  found  most  interesting.  Of  the  35  percent  that  used  hypertext  links  for  development,  over  80  percent  of  them 
had  five  or  more  links  within  each  biographical  description.  The  words  that  were  most  commonly  linked  were  those 
that  the  teacher  believed  would  provide  background  knowledge  for  readers  assumed  to  have  no  previous  knowledge 
of  the  person.  An  example  is  the  links  in  the  nodes  on  John  F.  Kennedy,  which  included  the  “Cuban  Missile 
Crisis,”  “Bay  of  Pigs,”  “Marilyn  Monroe,”  “Fidel  Castro,”  and  “Camelot.”  All  of  these  words  were  linked  to 
additional  nodes  that  explained  John  F.  Kennedy’s  relationship  to  each  of  these  nodes. 

Analysis  of  the  media  employed.  All  teachers  used  digital  pictures  copied  from  the  Internet  or  scanned  from 
a book.  Thirty  percent  of  the  students  also  used  QuickTime  movies  obtained  from  the  Internet  that  showed  the  event 
in  detail.  As  they  indicated  in  their  learning  logs,  teachers  believed  that  these  video  clips  effectively  conveyed  ideas 
they  wanted  to  portray  about  their  person.  In  addition  to  pictures  and  movies,  20  percent  of  the  teachers  used  sound 
clips  that  they  prerecorded  using  SoundEdit  Pro  ™ or  that  they  captured  from  the  Internet  to  add  narration  to  their 
project. 

Through  these  hypermedia  productions  based  on  the  students’  writing,  the  teachers  were  using  multimedia 
links  to  model  uses  of  technology  for  their  students  as  a tool  for  portraying  a range  of  different  biographical 
elements  of  their  subjects’  lives 

Web-based  Communications  between  Teachers  and  Students 

Building  personal  relationships.  Analysis  of  the  web-based  communication  between  teachers  and  students 
indicated  that  the  teachers  initiated  all  the  comments  on  the  asynchronous  discussion  board.  The  initial 
conversations  during  the  first  two  weeks  of  the  semester  typically  began  with  three-to-five  sentence  personal 
anecdotes  that  served  to  help  establish  a personal  relationship  between  the  teacher  and  students.  The  interaction  and 
writing  style  during  these  initial  exchanges  was  relatively  formal. 

Many  of  the  middle  school  students  described  how  they  enjoyed  the  ability  to  communicate  on-line  to  build 
a better  relationship  with  the  teacher  before  they  started  the  co-inquiry  multimedia  project.  One  student  said, 
"because  we're  able  to  communicate  online,  it  was  easier  to  get  to  know  the  practicum  teacher  because  it  gave  me 
more  time  to  think  about  what  I would  want  to  know  from  them  and  how  I might  want  to  answer  their  questions." 
Another  student  said,  "I  was  always  excited  to  check  the  discussion  area  when  I got  home  so  I could  see  if  my 
practicum  teacher  had  sent  me  a message  back."  The  middle -school  students  expressed  some  disappointment  to 
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their  teachers  when  the  teacher  did  not  respond  immediately  to  their  posting,  an  indication  of  their  interest  in  hearing 
from  their  teacher. 

Planning  and  development.  As  illustrated  by  the  initial  example  of  work  on  the  Princess  Diana  project,  as 
the  semester  progressed,  the  conversations  focused  more  on  planning  and  developing  the  multi-genre  writing 
project.  While  the  students  normally  posed  a topic  that  was  directly  related  to  the  media  and  the  popular  culture, 
many  of  the  teachers  encouraged  students  to  select  topics  that  they  found,  as  one  teacher  noted,  “would  be  more 
meaningful  and  easier  to  obtain  quality  information.”  During  these  exchanges,  the  sentences  became  much  shorter 
than  during  the  initial  exchanges,  with  incomplete  one  to  two  sentence  responses.  The  interaction  and  writing  style 
also  became  more  informal. 

The  discussion  board  served  to  support  the  teachers  and  students  in  sharing  ideas  about  the  content  of  their 
multi-genre  writing  project,  sharing  involving  literacy  practices  such  as  posing  questions.  In  the  exchanges, 
teachers  frequently  posed  questions  to  students  regarding  further  elaboration  about  their  projects,  questions  that  they 
may  then  have  internalized  to  think  about  different  aspects  of  their  projects. 

The  assignment  of  working  with  two  to  four  students,  each  of  who  was  creating  a different  project  was  a bit 
overwhelming  for  the  pre-service  teachers.  The  discussion  board  helped  the  teachers  monitor  the  students*  progress 
on  the  project  to  insure  that  they  completed  it  on  time.  Some  of  the  teachers  commented  on  the  convenience  of 
being  able  to  send  multiple  messages  to  the  middle  school  students  and  determine  their  progress  through  their 
responses.  As  one  teacher  noted,  " I am  able  to  keep  in  constant  communication  with  them  up  to  the  days  I meet 
with  them.  We  are  then  able  to  get  much  more  accomplished  as  we  have  been  communicating  and  know  what  the 
plan  is  when  we  will  see  each  other." 

Frequency  of  exchange.  In  the  exchanges  on  the  multi-genre  project,  the  teachers  were  more  likely  to 
dominate  the  discussion.  Seventy  percent  of  the  conversation  focused  on  direction  and  control  comments  where  the 
teachers  were  guiding  the  students  in  their  research  asking  them  about  the  progress  they  were  making  on  research  or 
reminding  them  what  was  due  the  next  time  they  were  able  to  meet.  When  responses  were  elicited  on  research 
progress,  85  percent  of  sharing  included  Internet  addresses  where  students  had  found  information  they  believed 
could  contribute  to  the  final  project. 

Analysis  of  the  exchange  based  on  gender  differences  indicated  that  male  and  female  students  who 
collaborated  with  female  teachers  had  a 35  percent  greater  quantity  of  discussions  overall  than  with  male  teachers. 
Students  were  also  52  percent  more  likely  to  employ  what  was  categorized  as  “personal”  topics  with  female  teachers 
than  with  male  teachers.  There  was  also  a difference  within  the  student  group;  female  students  communicated  mo  re 
frequently  and  also  contributed  more  project-related  information  than  their  male  counterpart. 

Analyzing  all  of  the  asynchronous  discussions,  teachers  employed  73  percent  of  “thought  units,”  while 
students  contributed  only  27  percent.  Overall,  the  focus  of  the  discussions  moved  from  initial  personal 
conversations  to  project-related  conversation  during  the  middle  of  the  semester  to  personal  conversation  at  the  end 
of  the  semester. 

Given  the  infrequency  of  face-to-face  meetings  during  the  practicum,  this  web-based  communication 
served  to  enhance  the  quality  of  teacher/student  relationships  and  provide  for  frequent  collaboration  on  the  project. 
Through  this  experience,  both  teachers  and  students  learned  to  perceive  the  value  of  web-based  communication  as  a 
tool  for  engaging  in  collaborative  co-inquiry. 

Teachers’  WebCT  Bulletin  Board  Communication 

Analysis  of  the  topics  addressed  in  the  WebCT  class  bulletin  board  exchanges  indicated  that  teachers  used 
the  exchanges  to  discuss  a range  of  different  issues,  particularly  those  associated  with  education:  teachers  as  role 
models,  vouchers,  censorship,  testing,  etc.  And,  teachers  shared  their  experiences  with  working  in  the  middle  - 
school  practicum,  as  well  as  personal  experiences.  In  doing  so,  they  employed  a number  of  literacy  practices  that 
served  to  foster  productive  exchanges: 

Display  of  spontaneous  thinking.  Teachers  used  the  postings  to  openly  think  through  a topic  or  issue, 
creating  a written  record  of  their  unfolding  thought.  Rather  than  formulate  their  ideas  prior  to  writing  and  then 
write  an  organized  statement,  teachers  were  spontaneously  writing  out  their  thoughts  in  a free -writing  mode.  They 
would  then  entertain  alternative,  even  contradictory  perspectives  as  they  formulated  their  thoughts  in  a posting.  For 
example,  in  discussing  the  issue  of  teaching  expository  versus  narrative  forms  to  middle  -school  students,  one 
teacher,  responding  to  another  teacher’s  belief  in  the  value  of  narrative  writing,  noted: 

As  we  discussed  in  class  eaiiier,  there  is  clearly  something  going  on  with  my  middle -school  student  that 
makes  the  narrative  form  a richer  expressive  medium  for  him.  I will,  of  course,  take  a look  at  your  link.  Also,  I 
would  like  to  see  more  of  the  research  on  this.  The  stuff  we  *ve  gotten  in  the  program  points  specifically  to  class- 
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differentiated  processing.  But  your  post  suggests  that  there  is  also  research  pointing  to  a broader  conclusion.  But 
before  I do  I wanted  to  affirm  your  idea  about  narrative  processes  superseding  linear  logical  processes  in  decision- 
making. I know  for  myself  that  the  work  that  I do  with  I am  reflecting  on  a difficult  problem  often  resembles  a 
conversation  more  than  a reasoned,  bulleted  list.  I wonder  where  conversational  dialogue  fits  in  this  paradigm? 

It’s  not  really  narrative,  but  God  knows,  it  ain’t  logical!  Anyway,  I shouldn’t  say  more  ‘til  I’ve  read  some.  More 
later. 

The  spontaneous  nature  of  his  thinking  is  evident  in  the  fact  that  he  poses  questions  to  himself  (“I  wonder 
where  conversational  dialogue  fits  in  this  paradigm?”  which  then  stimulate  him  to  further  thinking  about  the  issue. 
He  also  openly  reports  on  the  fact  that  “I  shouldn’t  say  more  ‘til  I’ve  read  some,”  implying  that  he  will  continue  to 
think  further  about  the  topic. 

The  fact  that  these  teachers  explicitly  shared /low  they  are  grappling  with  an  issue  provided  other 
participants  with  a window  on  the  reasoning  employed,  allowing  others  to  react  to  that  reasoning. 

Inviting  others'  participation.  The  teachers  also  commended  each  other  for  their  comments  and  invited 
others  to  participate  or  to  respond  to  their  postings.  The  positive  comments  and  invitations  implied  that  they  valued 
the  need  for  others’  perspectives  as  useful,  rational  beliefs  about  a topic,  an  enactment  of  Davidson’s  (1984) 
principle  of  charity.  For  example,  in  discussing  the  topic  of  future  employment  in  the  job  market,  one  student 
reacted  to  another  student’s  description  of  an  interview  with  a school  administrator  about  her  hiring  practices: 

I liked  what  you  said  here....  First  of  all,  way  to  ask  a relevant  questions.  Along  with  finding 
hope  in  her  answer.  I’d  like  to  pose  an  equally  practical  question.  When  and  how  should  we  be 
going  about  searching  out  opportunities  for  our  own  future  employment?  I am  lucky  to  have  a 
few  friends  in  high  places  when  it  comes  to  the  job-search  issue,  but  I think  it  would  be  wonderful 
to  be  getting  some  direction  on  this  subject  in  class.  Anyone  else  have  any  insights  or 
information  for  me??? 

And,  by  framing  their  postings  in  a tentative,  exploratory  manner — as  “passing  theories”  (Davidson  1984), 
the  teachers  were  inviting  or  implying  the  need  for  further  verification-agreements  or  disagreements -from  their 
peers.  For  example,  in  discussing  the  topic  of  grading  writing,  one  teacher  formulated  his  position  on  the  need  to 
provide  feedback  during  the  entire  composing  process: 

So,  my  two  cents:  I kind  of  see  grading  as  a process  that  begins  when  the  paper  is  assigned  and 
ends  when  we  hand  back  that  last  draft.  Plus,  it  bears  great  weight  (some  insist  that  grading 
should  be  done  away  with  in  comp  classes)  in  terms  of  the  whole  process,  their  process,  of 
addressing  and  completing  a writing  assignment.  Does  this  make  sense  to  anyone? 

In  his  positing,  he  hedges  his  comments  with  words  such  as  “my  two  cents”  and  “kind  of  see.”  He  also 
notes  that  others  hold  different  perspectives  on  grading.  And,  his  final  invitation,  “Does  this  make  sense  to 
anyone?”  implies  that  he  himself  is  trying  to  “make  sense”  out  of  his  own  ideas  about  evaluating  writing.  His 
invitation  evoked  a number  of  reactions  in  which  teachers  mutually  explored  the  issue  of  evaluating  writing: 

Engaging  in  word/role  play.  The  teachers  also  frequently  engaged  in  “double -voiced”  word  play  (Bakhtin, 
1981),  mimicking  or  parodying  persons  or  discourses.  Within  the  course,  the  teachers  had  also  participated  in  a 
large-group  role-play  based  on  the  2000  Presidential  election  in  which  they  adopted  various  roles  and  exchanged 
written  memos  with  each  other.  They  compared  their  WebCT  exchange  with  this  role-play  session  in  terms  of 
using  written  texts  to  engage  in  verbal  play  through  writing.  As  one  teacher  noted: 

The  experiences  with  WebCT  has  really  opened  up  my  ideas  on  communication  and  possibilities 
therein...!  think  both  WebCT  and  the  role  play  offer  something  priceless  to  learning,  i.e.,  play. 

It’s  learning  of  and  appreciation  for  multiplicity.  There  were  so  many  contexts  overlapping  in 
that  classroom  that  multiple  uses  and  abuses  are  inevitable,  and,  I think,  productive.  The  same  is 
true  for  the  WebCT. 

This  dialogic  word-play  included  intertextual  references  to  stances  and  discourses  operating  in  the  group 
and  the  teacher  education  program.  By  mimicking  or  parodying  the  language  of  these  stances  or  discourses, 
student  were  formulating  oppositional  stances  reflecting  their  own  beliefs  and  ideas  about  teaching  and  learning. 

Self-reflection  on  the  process.  Teachers  also  explicitly  reflected  on  or  described  their  stances  or  attitudes 
adopted  in  their  postings.  In  some  cases,  they  apologized  for  repeating  themselves,  making  overly  assertive 
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statements,  or  sharing  complaints.  For  example,  one  student  noted:  “Whoops,  I just  browsed  back  up  the  thread  and 
realized  Tm  repeating  myself!”  After  posting  a long  message,  one  student  commented,  “Sorry  to  drop  such  a wide 
load  here  on  the  CT,  but  it  was  cathartic.”  They  also  valued  the  fact  that  they  could  openly  express  their  opinions 
within  their  group  without  necessarily  being  concerned  about  offending  others.  One  student  noted,  “could  you  just 
imagine  if  we  were  afraid  of  speaking  our  souls  for  fear  of  offending  someone.  Our  class  would  be  pretty  damn 
quiet  if  that  were  the  case.” 

A Korean  student  noted  that  the  site  served  to  foster  development  of  open  expression,  something  she  finds 
lacking  in  her  Korean  student  peers:  “we  are  too  concerned  about  hurting  others  people’s  feelings  to  think  out 
loud... I think  Korean  students  have  to  learn  to  be  more  assertive  in  order  to  exchange  their  thoughts.” 

Teachers  also  noted  some  of  the  difficulties  specific  to  participating  on  a bulletin  board  discussion.  One 
teacher  commented  on  the  difficulty  of  conveying  her  attitudes:  “I  don’t  know  exactly  how  or  why,  but  threaded 
discussions  transform  words.  Unless  the  writer  is  incredibly  skilled,  the  tone  is  hard  (if  not  impossible)  to 
communicate.  Perhaps  it  is  the  instantaneous  nature  of  it  that  is  its  main  draw  and  downfall...?  Her  comment 
suggests  that  some  participants  had  difficulty  communicating  their  attitudes  in  the  exchanges. 

The  teachers  therefore  used  their  exchanges  on  the  WebCT  bulletin  board  discussion  as  a tool  for  mutually 
formulating  strategies  for  coping  with  various  issues  associated  with  teaching.  Through  their  participation  in  these 
exchanges,  they  were  learning  to  employ  computer-mediated  communication  as  a tool  for  the  literacy  practices  of 
displaying  spontaneous  thinking,  inviting  others’  participation,  adopting  an  exploratory  stance,  engaging  in 
word/role  play,  and  reflecting  on  the  process. 

Summary 

The  results  of  this  study  indicated  that  the  teachers  and  middle  -school  students  were  employing  the 
technology  tools  of  hypermedia  production  and  web-based  communication  to  engage  in  literacy  practices  involved 
in  their  multi-genre  writing  project  and  in  communicating  with  each  other.  These  tools  served  to  mediate  the  uses 
of  various  literacy  practices  within  the  larger  activity  system  of  teacher  education,  whose  object  is  to  assist  teachers 
to  acquire  those  practices  involved  in  working  effectively  with  students. 

The  teachers  used  the  Inspiration™  and  Story  space™  tools  to  define  intertextual  and  hypertextual 
connections  between  the  texts  included  in  their  multi-genre  writing  project.  These  hypermedia  tools  allowed 
teachers  to  combine  written  texts,  images,  sounds,  and  video  to  portray  the  characteristics  of  a person  in  a web-based 
production  for  sharing  with  others,  including  their  students.  Creating  these  hypermedia  productions  in  a co-inquiry 
project  with  their  students  also  helped  teachers  learn  how  to  model  the  literacy  practices  of  making  intertextual  or 
hypertextual  connections  for  students,  an  important  teaching  strategy. 

With  the  increased  focus  on  multi-media  and  hypertext  communication  in  literacy  education. 
English/language  arts  teachers  need  to  acquire  an  ability  to  use  this  tool  as  part  of  their  literacy  instruction. 

Teachers  also  need  to  be  able  to  employ  links  that  go  beyond  just  illustration  to  engage  in  critical  interrogation 
(McKillop  & Myers,  1999).  One  limitation  of  this  project  remains  the  relatively  high  cost  of  the  Storyspace™ 
software  for  large-scale  use  in  schools.  At  the  same  time,  other,  less  expensive  hypermedia  software  such  as 
HyperStudio™  or  HyperCard™  can  be  used  as  an  alternative. 

The  teachers  and  students  used  the  web-based  communication  site  as  a tool  for  establishing  social 
relationships  and  for  planning  their  multi-genre  writing  projects.  This  site  provided  teachers  with  continuous,  on- 
going interaction  with  their  students,  something  often  lacking  in  practicum  experiences  with  infrequent  school  visits. 
The  students  expressed  a high  level  of  engagement  with  this  site,  expressing  disappointment  when  they  did  not 
receive  responses  from  their  teachers.  The  written  exchange  allowed  teachers  to  model  a range  of  literacy 
practices,  particularly  self-disclosure  about  their  own  lives  and  posing  questions  about  the  project,  practices  students 
then  demonstrated  in  their  own  responses. 

One  problematic  aspect  of  the  exchange  was  the  fact  that  the  teachers  dominated  the  interactions  by  a ratio 
of  four  to  one.  One  possible  explanation  of  this  disparity  was  that  the  students  had  minimal  access  to  computers  in 
their  school  and  simply  did  not  have  the  time  to  write  extensive  answers.  While  students  could  also  access  the  site 
from  their  homes,  many  students  did  not  have  computer  access  in  their  homes.  Another  factor  may  have  been  that 
some  students  had  minimal  writing  skills;  limiting  the  amount  they  were  able  to  write.  The  fact  that  the  students 
wrote  longer  entries  when  they  were  discussing  their  own  lives  and  shorter  entries  when  they  were  discussing  their 
projects  suggests  that  teachers  employing  this  tool  need  to  include  a focus  on  autobiographical  topics,  as  well  as 
topics  related  to  tasks.  There  were  also  marked  gender  differences,  with  female  teachers  eliciting  more 
participation  from  students  than  male  teachers  through  uses  of  “personal”  connections,  suggesting  the  need  for 
teachers  of  both  genders  to  employ  such  connections. 
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Through  their  participation  on  the  WebCT  bulletin  board,  teachers  were  recognizing  how  participating  in 
an  active,  on-line  community  helped  them  explore  issues  and  concerns  related  to  education.  Given  this  experience, 
they  may  then  be  more  likely  to  participant  in  similar  web  sites  or  employ  such  sites  in  their  own  teaching.  And, 
through  that  participation,  they  were  employing  a number  of  literacy  practices  that  they  could  model  as  participants 
in  teacher/student  web-communication. 
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A Themed  and  Collaborative  Approach  to  Teaching  Computers  and  the  Internet 


Teshia  Young  Roby 
Georgia  Institute  of  Technology 


This  research  series  is  a practical  and  effective  approach  to  exposing  middle  - and  high-  school  students  to 
fun  and  useful  elements  of  the  Internet,  current  technology  issues  and  to  Microsoft  Word,  Excel  and  PowerPoint. 

The  theme  for  the  learning  series  is  one  in  which  the  students  are  owners  of  music  recording  companies.  Youths  are 
fascinated  by  careers  in  the  recording  industry!  The  focus  of  this  theme  is  not  on  the  musical  talents,  but  on  the 
professionals  who  run  these  organizations. 

The  Program  Community 

The  6-month  bi-weekly  Saturday  program  was  sponsored  by  the  100  Black  Men  of  Atlanta  under  their 
Project  Success  initiative.  Project  Success  is  a program  that  allows  inner-city  middle  - and  high-school  students  to 
be  mentored  by  members  of  100  MBA  and  volunteering  college  students.  The  Project  Success  students  and  their 
parents  have  to  be  actively  committed  to  the  entire  program,  which  includes  participation  in  workshops,  training, 
and  cultural  events  that  are  designed  to  teach  the  students  to  be  productive,  progressive,  members  of  society.  For 
each  student  that  stays  committed  throughout  his  or  her  middle-  and  high-school  career,  the  100  BMA  pledges  to 
pay  the  full  tuition  for  any  college  to  which  the  student  is  accepted.  The  program  is  a long-term  commitment  for 
everyone  involved,  including  the  students’  guardians.  The  100  BMA  are  currently  into  their  second  round  of 
students  to  reach  college  age. 

The  program  community  consisted  of  one  instructional  designer  (me).  One  instructor  (me),  approximately  5 
undergraduate  student  volunteers  and  two  groups  of  16  - 20  students.  The  volunteers,  each  whom  had  technical 
experience,  floated  around  the  classroom  offering  help  to  students  who  needed  it  and  helped  to  keep  the  noise  down 
when  the  students  got  a little  excited. 

The  students  were  broken  into  two  groups.  For  the  first  two  hours  of  the  half-day  program,  half  of  the 
entire  group  would  work  with  me  and  the  other  half  would  participate  in  another  part  of  the  program.  At  the  end  of 
the  two  hours,  the  two  groups  would  switch  activities.  The  groups  were  assigned  arbitrarily. 

The  Program  Theme 

During  my  volunteer  time  with  Southeastern  Consortium  of  Mathematicians  and  Engineers  (SECME),  I 
found  that  many  - if  not  most  - of  the  students  that  we  visited  wanted  to  pursue  a career  in  acting,  modeling  or  the 
music  industry  and  the  heaviest  concentration  was  in  the  latter.  I decided  from  the  start  to  use  a theme  throughout 
the  program,  to  put  the  students  in  the  center  of  the  learning,  and  to  bring  about  reinforcement.  I decided  the  theme 
would  be  that  of  the  recording  industry  to  provide  interest  and  familiarity  for  the  students.  Though  the  theme 
encompassed  the  idea  of  the  recording  company,  the  focus  of  the  theme  was  not  on  the  musical  talents,  but  on  the 
professionals  who  run  the  companies.  I wanted  the  students  to  be  in  positions  of  authority. 

In  each  scenario  within  the  learning  series,  the  students  were  cast  as  recording  company  executives  who 
needed  to  accomplish  a particular  task  for  the  business.  A list  of  objectives  or  tools  needed  to  accomplish  the  task 
was  also  included.  The  instructions  that  followed  the  scenario  and  objectives,  which  required  the  use  of  one  of  the 
applications  addressed  in  the  program,  helped  them  to  accomplish  those  tasks.  I included  actual  screen  captures  in 
the  notes,  which  allowed  the  students  to  know  exactly  what  they  should  see  on  their  screens  and  it  enabled  them  to 
work  ahead  if  they  liked. 

Curriculum  format 

The  objective  of  the  Saturday  technology  sessions  was  to  expose  the  students  to  fun  and  useful  elements  of 
the  Internet,  current  technology  issues,  and  Microsoft  Word,  Excel,  and  PowerPoint. 

Module  1 - Program  Introduction 
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The  first  module  began  with  student,  volunteer  and  instructor  introductions.  To  bring  cohesiveness, 
consistency  and  relevance  to  the  program  activities  for  the  students,  a general  theme  for  the  activities  was 
introduced.  Two  members  of  local  recording  companies  were  invited  and  were  present  to  give  the  students  an  idea 
of  the  importance  of  computer  and  technical  knowledge  in  the  industry.  They  also  answered  a barrage  of  questions 
about  various  facets  of  the  recording  industry. 

During  the  module,  the  students  also  took  a pre-test  on  the  various  components  of  the  computer.  They 
were  then  shown  actual  internal  components  of  a computer  and  told  of  their  functions.  Finally,  they  were  given  a 
verbal  overview  of  what  the  entire  program  would  entail. 

Module  2 - The  Internet 

Unit  one  of  the  second  module  included  a PowerPoint  presentation  on  the  history  and  current  state  of  the 
Internet  that  included  definitions,  information  search  techniques  and  emailing  instructions.  The  students  were  then 
lead  to  the  computer  rooms  where  they  all  were  assisted  in  setting  up  their  own  email  accounts.  They  also 
participated  in  a World  Wide  Web  scavenger  hunt.  The  scavenger  hunt  consisted  of  questions  that  varied  in 
difficulty.  The  students  were  asked  to  use  any  of  the  various  search  techniques  that  were  introduced  in  the 
presentation  to  find  the  answers.  Once  a student  found  an  answer,  the  student  was  asked  to  show  the  class  how  that 
answer  was  found  and  received  a treat. 

Unit  two  involved  more  in-depth  Internet  searching  by  the  students.  The  students  were  asked  to  find  three 
types  of  websites:  recreational,  reference  and  consumer.  They  were  instructed  to  find  three  websites  for  each 
category  (for  a total  of  nine  websites)  and  to  give  a description  of  each.  The  students  were  then  asked  to  volunteer 
to  share  the  websites  they  found.  The  reports  were  collected  so  that  a list  could  be  compiled  and  distributed  at  a 
later  date.  The  students  were  also  given  a post-test  on  the  components  of  the  computer. 

Module  3 - Microsoft  Word 

The  first  unit  of  the  third  module  focused  on  an  introduction  to  Microsoft  Word.  The  students  took  a pre- 
assessment  regarding  their  knowledge  of  MS  Word  and  received  instruction  on  creating  a flyer  for  their  recording 
company’s  annual  talent  show. 

The  second  unit  included  instruction  on  creating  a research  paper  in  MS  Word.  The  activity  was 
introduced  as  a report  for  the  recording  company  CEO  about  the  Internet  and  web  publishing.  In  the  scenario,  the 
report  occurred  as  a result  of  a suggestion  by  an  executive  (the  student)  about  creating  a website  for  the  company. 
Afterward,  the  students  took  a post-assessment  on  their  knowledge  level  with  MS  Word. 

Module  4-  Current  Events  in  Technology 

The  fourth  module  consisted  of  a discussion  on  the  importance  of  technology.  The  students  were  arranged 
in  groups  and  asked  to  brainstorm  about  benefits  of  having  access  to  a computer  and  the  Internet.  The  students  were 
also  introduced  to  the  highlights  of  a report  called  Fa///«g  Through  the  Net:  Defining  the  Digital  Divide 
(www.ntia.doc.gov/ntiahome/fttn99).  The  report,  based  on  the  December  1998  U.S.  Department  of  Commerce 
Census  Bureau  data,  provides  an  updated  snapshot  of  American  use  of  technology.  The  report  identified  that 
computer  ownership  and  Internet  access  has  increased  for  all  ethnic  groups  in  all  locations.  Unfortunately,  though, 
groups  that  were  already  connected  are  now  far  more  connected,  while  those  with  lower  rates  of  access  have 
increased  less  quickly.  As  a result,  the  gap  between  the  technologically  wealthy  and  the  technologically  poor  is 
growing.  This  gap  is  known  as  the  “digital  divide”.  The  report  also  revealed  that  there  are  many  factors  that  create 
the  huge  difference  between  those  who  have  computers  and  Internet  access  and  those  who  do  not.  The  children 
discussed  in  groups  and  with  the  class  several  ways  to  provide  solutions  to  the  problems. 

Module  5-  Microsoft  Excel 

The  fifth  module  included  a pre -assessment  and  an  activity  to  introduce  the  students  to  Microsoft  Excel. 
The  activity  was  presented  as  scenario  regarding  record  sales  for  the  various  recording  label  artists  and  the  overall 
recording  company.  The  students  created  a spreadsheet  and  accompanying  charts  and  shared  their  end  results  with 
the  class.  A post-assessment  followed  the  activity. 
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Module  6-  Microsoft  PowerPoint 


Unit  one  of  the  sixth  module  began  with  a pre -assessment  on  PowerPoint.  The  students  participated  in  an 
activity  that  allowed  them  to  create  a promotional  PowerPoint  presentation  for  their  company’s  new  artist  event.  In 
the  scenario,  the  company  executive  would  design  the  presentation  to  attract  interest  and  place  it  on  a computer  in 
the  student  center  of  a local  university. 

Unit  two  provided  the  students  with  an  opportunity  to  develop  a presentation  of  their  own.  The  students 
were  asked  create  an  original  presentation  that  included  autobiographical  information  and  a 3-,  5-,  and  10-year  plan 
for  the  future.  The  presentation  was  also  to  include  family  pictures  that  were  scanned  into  the  computer,  digital 
pictures  taken  in  class  and  graphics  imported  from  the  Internet.  The  students  received  instruction  on  visual 
consistency  and  quality  presentation  design. 

The  third  unit  of  the  sixth  module  began  with  presentation  tips  such  as  eye  contact  and  annunciation.  The 
students  were  given  the  opportunity  to  practice,  present  to  the  class  and  receive  feedback.  The  unit  ended  with  a 
post-assessment  on  PowerPoint. 

The  fourth  and  final  unit  consisted  of  presentations  by  the  students.  The  students  were  encouraged  to  invite 
their  parents  to  observe  the  presentations.  Sponsors  and  mentors  of  the  program  were  also  encouraged  to  attend. 

Program  Conclusion  and  Assessment 

The  program  concluded  with  an  opportunity  for  the  students  to  demonstrate  their  ability  to  use  the 
technology  through  a well-attended  voluntary  presentation  at  an  assembly  for  parents  and  program  sponsors. 
Evidence  of  the  effectiveness  of  the  instruction  was  provided  through  a statistical  comparison  of  the  pre  - and  post- 
tests, as  well  as  comments  of  interest,  excitement  and  approval  from  several  of  the  program  students,  their  parents, 
and  Project  Success  volunteers  and  employees. 

Jonassen’s  Seven  Aspects  of  a Technology  Integration  Learning  Environment 

According  to  Jonassen  (1999),  technology  integration  does  not  happen  in  a particular  location,  but  in  a 
particular  learning  environment.  That  environment  includes  a learner-centered-teacher-as-facilitator  atmosphere  and 
seven  additional  aspects  that  make  the  learning  meaningful.  The  learning  environment  is  active  and  requires 
students  participate  in  the  processing  of  information;  it  is  constructive,  so  students  are  encouraged  to  integrated  new 
ideas  into  their  prior  knowledge;  it  is  collaborative,  which  allows  students  to  work  in  learning  communities;  it  is 
conversational,  so  that  students  share  ideas  and  build  upon  each  other’s  knowledge;  it  is  contextualized  or  situated  in 
real  world  task  or  problem-based  activities;  it  is  intentional,  and  students  are  made  aware  of  cognitive  goals  and 
objectives  at  the  onset;  and  it  is  reflective,  where  students  are  encouraged  to  reflect  on  the  process  and  articulate 
what  they  have  learned. 

The  overall  theme  of  the  technology  learning  series  was  that  of  a recording  company  in  which  the  students 
were  corporate  executives  acting  on  behalf  of  the  company.  Each  of  the  seven  aspects  was  present  within  the 
instruction  produced  for  the  program.  For  instance,  with  their  prior  knowledge  of  and  interest  in  recording 
companies,  students  were  able  to  put  the  new  information  regarding  the  software  applications  into  context;  students 
were  assigned  ownership  of  their  recording  companies  and  were  given  positions  of  action  and  authority  within  the 
business.  They  were  also  allowed  to  choose  an  “executive  board”  of  three  to  four  members.  This  allowed  them  to 
learn  in  teams.  The  team  atmosphere  allowed  the  students  to  share  ideas  and  construct  the  knowledge-building 
communities.  The  modules  began  with  a scenario  that  put  the  instruction  into  a real-world  perspective  for  the 
students  and  each  scenario  was  followed  by  a list  of  objectives  for  the  instruction,  so  students  were  made  aware  of 
the  concepts  that  were  to  be  learned.  Many  of  the  modules  also  ended  with  an  opportunity  for  the  students  to  share 
their  projects  with  the  other  teams,  talk  about  their  process,  and  answer  questions  asked  by  the  instructor  and  other 
students. 

Lessons  learned 

There  were  several  valuable  lessons  that  I learned  while  participating  in  this  technology  program,  most  of 
which  had  nothing  to  do  with  the  technology  itself  Most  had  to  do  with  the  relationships  and  dynamics  of  the 
classroom. 

Use  a title  with  vour  name  in  the  beginning 
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First,  I would  have  asked  them  to  call  me  Ms.  Teshia,  I did  not  do  that  at  first  because  most  of  the  adult 
volunteers  were  addressed  by  their  first  names  and  I wanted  to  “fit  if’  but  what  I failed  to  realize  was  that  those  kids 
had  long-term  intimate  relationships  with  those  people  that  made  it  appropriate.  I had  not  established  myself  in  the 
community  and  had  not  earned  the  respect  of  the  students,  so  I was  placing  myself  on  the  peer  level  with  them 
instead  of  earning  my  position.  If  I had  it  to  do  again,  I would  have  asked  them  to  call  me  Ms.  Teshia  and  considered 
allowing  them  to  do  otherwise  once  we  grew  our  relationships.  It  is  very  hard  to  go  the  other  way. 

Include  plenty  of  conversation  and  break  time  for  students 

Break  time  was  the  time  when  I got  to  know  the  kids  of  personal  levels,  when  they  got  to 
know  me,  and  when  they  got  to  fellowship  with  each  other.  I suggest  taking  plenty  of  breaks. 

You  can  get  them  to  stay  on  task  if  they  know  a break  is  coming  up  soon.  Also,  since  the 
program  is  not  compulsory,  it  should  not  feel  like  hard  work  and  school,  but  fun  like  computer 
camp.  This  was  also  a time  when  I got  a chance  to  truly  learn  about  the  likes  and  dislikes  of  the 
kids’  generation,  which  informed  my  design  of  the  activities. 

Allow  the  use  of  home  language  at  times 


Home  language  should  be  permitted  in  the  classroom  in  certain  situations.  I allowed  them  to  talk  amongst 
themselves  in  a way  that  is  comfortable  to  them  but  not  offensive  to  anyone.  They  even  communicated  with  me  in  a 
respectable  manner  in  a form  of  their  home  language.  However,  when  it  came  time  to  present,  they  were  asked  to 
use  Standard  English,  Presentations  were  voluntary,  and  they  knew  by  volunteering,  they  would  have  to  present 
using  Standard  English.  Most  were  more  than  compliant. 

Place  no  labels  or  stereotypes  on  the  students 

Even  though  you  have  to  try  to  find  out  all  you  can  about  the  students  before  the  course  in  order  to  design 
the  instruction,  do  not  associated  the  students  with  labels  as  a result  of  that  inquiry.  It  can  prove  to  be  harmful  the 
their  self-images  and  counter-productive  to  the  instruction. 


Use  small  groups 

The  kids  flourished  in  small  groups!  Students  brainstormed  and  named  their  recording  companies,  and 
many  of  the  groups  even  designed  a logo  and  assigned  rolls  within  the  recording  comp  any. 

Make  certain  the  theme  has  relevance  for  the  learners 


It  was  so  important  to  have  lessons  that  had  relevance  to  the  students  because  they  embraced  the  learning. 
They  saw  immediately  how  they  could  use  that  new  knowledge  and  they  wanted  to  learn. 

Include  group  policy  makine  at  the  start  of  the  program 

It  became  necessary  towards  the  middle  of  the  program  to  establish  some  rules  of  behavior.  The  program 
was  voluntary,  so  there  were  kids  who  came  every  session  and  those  who  came  every  once  in  a while.  There  were 
also  those  who  behaved  very  well  and  those  with  whom  we  had  challenges.  We  has  instance  where  our  group 
disrupted  others  in  the  building  and  times  when  a few  members  the  program  disrupted  other  members.  So  we  had  to 
come  up  with  rules  that  the  kids  understood  and  embraced.  We  decided  to  include  the  students  in  developing  a list 
of  acceptable  and  unacceptable  behavior  and  the  consequences  of  both  and  had  them  sign  it.  It  curtailed  many 
behavioral  problems.  In  retrospect,  I would  have  done  that  much  earlier  in  the  program  and  I would  have  kept  a 
student  signed  copy  and  sent  home  a copy  that  had  the  student’s  signature  for  the  parent  to  sign  and  return,  just  to 
cover  all  bases. 
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Conclusion 


The  themed  and  collaborative  approach  to  teaching  computers  and  the  Internet  worked  well  because  the 
program  was  a community  effort  and  there  was  a consistent  theme  throughout  the  program  that  engaged  the  students 
and  gave  meaning  and  practicality  to  the  content.  Each  of  the  modules  included  screen  captures  and  collaborative 
activities,  which  allowed  the  students  to  work  at  their  own  pace  and  communicate  understanding  of  the  content  with 
group  members.  Jonassen’s  seven  aspects  of  a technology  integrated  learning  environment  certainly  informed  the 
design  and  implementation  of  the  themed  series,  but  group  policy  making,  frequent  breaks,  and  an  absence  of  stereo 
types  of  the  students  were  instrumental  to  the  success  of  the  learners. 

Contact  Information 

Teshia  Young  Roby,  Instructor  and  PhD  Student,  Georgia  State  University,  University  Plaza,  College  of 
Education,  Department  of  Instructional  Technology,  Atlanta,  GA  30303,  MSITI,  404-651-0181, 
Teshia@robyweb.com. 
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Abstract 


This  study  attempted  to  define  what  academic  rigor  means  to  the  faculty  members  and  graduate  assistants 
who  taught  on-campus  and  online  courses  at  a major  Midwestern  state  university.  The  study  sought  to 
discover  the  level  of  importance  these  instructors  gave  to  academic  rigor  and  what  strategies  they  used  to 
ensure  academic  rigor  in  their  courses.  Finally,  the  researchers  asked  instructors  about  their  strategies  to 
ensure  academic  rigor  in  these  two  different  delivery  methods. 

Introduction 

Academic  rigor  is  a topic  that  resides  at  the  very  core  of  the  traditional  conception  of  the  academy.  While 
precise  definitions  (Winston  et  al.,  1994;  Braxton,  1993;  Nicholson,  1996)  may  vary,  the  basic  idea  is  central  to 
postsecondary  education  as  it  has  been  implemented  in  the  past  few  hundred  years.  However,  at  the  dawn  of  this 
new  millennium,  the  popularity  of  new  pedagogical  beliefs  and  instructional  strategies,  such  as  constructivism  and 
problem-based  learning,  and  delivery  methods,  such  as  online  distance  education,  make  it  clear  that  it  is  time  to 
review  our  conceptions  of  academic  learning  and  to  see  if  old  definitions  match  the  postsecondary  educational 
practices  of  the  new  millennium. 

The  amazing  growth  of  online  distance  education,  in  particularly,  heightens  the  need  for  this  type  of  study. 
As  enabling  technologies  such  as  computers  and  the  Internet  have  become  more  ubiquitous  in  the  U.S.,  distance 
education  programs  have  become  increasingly  popular  in  both  formal  academic  settings  as  well  as  corporate 
settings.  Specifically,  over  the  past  five  years  there  has  been  a move  by  many  institutions  of  higher  education  to 
provide  some  kind  of  online  learning  opportunities.  According  to  a recent  MSNBC  report  (McGinn,  2000),  75 
percent  of  all  U.S.  universities  now  offer  online  coursework,  and  5.8  million  students  have  taken  online  college 
courses.  This  rapid  movement  towards  offering  online  courses  has  caused  some  concern,  especially  in  academic 
communities,  regarding  the  wisdom  and  implications  of  this  trend.  One  of  the  concerns  that  is  often  voiced  (Phipps 
& Merisotis,  1999;  Rutmania,  1999)  implies  that  online  courses  are  not  as  academically  rigorous  as  traditional  face- 
to-face  courses.  If  this  is  truly  the  case  then  the  academic  reputation  of  the  institution  could  possibly  be  at  risk. 

This  study  attempted  to  explicitly  define  what  academic  rigor  means  to  the  faculty  members  and  graduate 
assistants  who  taught  on-campus  and  online  courses  at  a major  Midwestern  state  university.  The  study  also  sought  to 
discover  the  level  of  importance  these  instructois  gave  to  academic  rigor  as  they  designed  their  course  offerings,  and 
what  strategies  they  used  to  ensure  academic  rigor  in  their  courses.  Finally,  the  researchers  asked  those  instructors 
who  taught  both  online  and  on-campus  courses  about  their  strategies  to  ensure  academic  rigor  in  these  two  different 
delivery  methods. 

Research  Questions 

This  research  study  attempts  to  answer  three  primary  questions: 

How  do  instructors  define  academic  rigor? 

How  important  is  academic  rigor  to  these  instructors? 

What  strategies  do  these  instructors  use  to  ensure  academic  rigor  in  their  on-campus  courses? 

For  those  instructors  who  teach  both  on -campus  and  online,  what  strategies  do  these  instructors  use  to 
ensure  academic  rigor  in  their  online  courses?  Are  these  strategies  different  than  the  strategies  used  in  their  on- 
campus  courses? 

Our  hypotheses  going  into  the  research  were  that  instructors  would  have  a variety  of  definitions  of 
academic  rigor,  but  that  important  similarities  would  emerge,  and  that  the  differences  in  definitions  would  also  be  of 
interest.  We  assumed  most  instructors  would  view  academic  rigor  to  be  of  great  importance  in  their  course  design. 
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We  thought  that  instructors  would  have  a variety  of  strategies  for  ensuring  academic  rigor  and  that,  again,  we  would 
find  a core  group  of  strategies,  with  interesting  variations  to  report  as  well.  Our  hypothesis  regarding  the  difference 
between  ensuring  rigor  in  online  and  on-campus  courses,  based  on  informal  conversations  with  instructors  of  both 
types  of  courses  prior  to  the  research,  would  be  minimal,  and  that  similar  strategies,  though  obviously  influenced  by 
the  medium  used  to  deliver  the  course,  would  be  employed. 

Significance  of  Research 

There  are  many  reasons  why  this  research  is  significant  to  instructors  and  administrators  at  postsecondary 
institutions,  especially  those  institutions  embarking  upon  distance  education  initiatives.  Three  key  areas  of  impact 
are  described  below. 

Evaluation  and  improvement  of  current  on-campus  and  online  courses 

Most  instmctors,  we  feel  it  is  fair  to  say,  would  agree  that  there  is  a need  for  the  continual  evaluation  and 
improvement  of  their  course  offerings.  If  our  hypothesis  that  academic  rigor  is  important  to  instructors  is  correct, 
then  our  findings  relating  to  the  first  research  question  should  provide  some  clear  criteria  regarding  academic  rigor 
which  then  can  be  used  to  evaluate  the  extent  to  which  it  exists  in  current  course  offerings.  Additionally,  the  findings 
of  the  third  and  fourth  research  questions  will  most  likely  shed  light  on  specific  strategies  relating  to  academic  rigor 
in  on-campus  and  online  courses  that  can  be  focused  on  to  improve  the  academic  rigor  of  both  types  of  courses. 

Promoting  discussion  of  academic  rigor  among  faculty 

We  hope  that  the  results  of  the  research  will  help  to  promote  discussion  of  academic  rigor  and  related 
pedagogical  issues  among  faculty.  By  reading  about  others’  strategies  for  ensuring  academic  rigor,  they  may  be 
moved  to  discuss  these  strategies  and  their  own  with  their  colleagues. 

Providing  an  opportunity  for  distance  educators  to  share  their  beliefs  about,  and  strategies  to  ensure, 
academic  rigor 

Any  study  that  looks  at  the  concept  of  academic  rigor  and  does  not  also  discuss  it  in  the  context  of  online 
distance  education  is  doing  a disservice  to  its  readers,  given  the  prevalence  of  these  courses  today  (McGinn,  2000). 
Distance  educators  have  been  accused  of  providing  less-than-rigorous  courses  voiced  (Phipps  & Merisotis,  1999; 
Rutmania,  1999)  and  we  wish  to  give  them  an  opportunity  to  dispel  these  notions,  if  they  are  indeed  incorrect  in  the 
eyes  of  the  instructors  involved. 

Past  Research 

Our  review  of  the  relevant  research  shows  that  the  topic  of  academic  rigor  is  mentioned  throughout  the 
literature,  but  that  only  in  a few  instances  is  the  meaning  of  the  term  specified.  We  were  only  able  to  find  three 
cases  in  which  the  term  academic  rigor  was  explicitly  defined  in  some  way.  Below  we  include  the  three  different 
definitions.  Being  that  the  concept  is  a complicated  one,  rather  than  trying  to  restate  and  summarize  the  elaborate 
definitions  we  encountered,  and  risk  losing  some  of  the  authors’  meaning,  we  will  present  below  three  of  the  best 
definitions  that  we  found  in  their  entirety. 

An  environment  that  is  intellectually  challenging  and  demanding.  Students  perceive  a norm  of  excellence 
and  responsibility,  which  is  expressed  through  high,  but  realistic,  evaluation  standards.  The  class  is  seen  as  fast- 
paced,  and  there  are  expectations  that  students  will  invest  considerable  energy  and  time  in  completing  assignments 
(Winston  et  al.,  1994) 

Academic  quality  is  manifested  in  such  course-level  academic  processes  as  the  type  of  questions  faculty  ask  students 
during  class,  the  nature  of  term  paper  assignments  or  other  written  exercises.  (...)  [AR  is]  the  level  of 
understanding  of  course  content  to  be  demonstrated  by  students  while  engaging  in  these  course-level  processes. 
(Braxton,  1993) 

Rigor — focused  and  critical  work — arises  from  a sense  of  the  importance  of  subject  matter  and  the 
opportunity  presented  for  its  mastery  and  refinement  through  study.  (...)  In  a rigorous  academic  environment,  the 
purposes,  principles  and  methodologies  of  scholarship  as  a means  of  establishing  the  connectedness  of  things  is 
understood.  (Nicholson,  1996) 


All  three  of  these  definitions  have  different  foci.  While  Winston’s  definition  focuses  on  student  output  and 
perception  of  a course,  Braxton’s  definition  focuses  on  the  instructor’s  efforts  as  well  as  the  learning  outcomes  of 
the  course  and  Nicholson’s  definition  centers  around  a more  traditional  awareness  and  use  of  methodologies. 

Although  most  of  the  articles  that  referred  to  academic  rigor  did  not  explicitly  define  the  term  they  often 
had  embedded  within  the  text  phrases  that  have  helped  us  to  gain  a better  feel  for  what  they  meant  by  academic 
rigor.  Table  1 has  a list  of  key  phrases  relating  to  academic  rigor  from  some  of  the  articles  that  were  reviewed.  It  is 
important  to  note  that  despite  the  fact  that  academic  rigor  is  used  in  many  different  ways,  it  is  almost  always  referred 
to  as  being  a positive  attribute  that  a program  or  course  should  have  under  ideal  circumstances. 


Reference  Terms 


Ridley  et  al.,  1998 

“course  achievement  as  measured  by  grades” 

“concern  that  instructors  have  applied  comparable  standards  in  assigning  grades  to  their 
online  or  traditional  students” 

Snell  et  al.,  1999 

“Online  or  distance  learning  is  harder  than  offline  or  traditional  classroom  learning” 
“it  appeared  that  online  courses  were  more  rigorous” 

“less  withdrawals  and  failures  among  the  traditional  offline  group” 

Thomas,  1998 

“academically  challenging” 

“number  of  homework  assignments” 

Kerr,  1990 

“Depth  of  study  in  areas  such  as  foreign  languages  and  mathematics” 

“Quality:  Four  or  five  academic  subjects  each  year” 

“Balance:  Evidence  that  the  student  took  a broad  curriculum” 

“Trends:  Evidence  as  to  whether  the  student’s  grades  are  gradually  improving  each  year” 

Rossman  et  al.,  1996 

“increased  requirements” 

“more  rigorous  content  standards” 

“students  maintain  a specific  grade  point  average” 
“more  challenging  program  of  study” 

“taking  more  courses” 

“taking  more  challenging  courses” 

“taking  more  course  breadth” 

Taylor  et  al.,  1991 

“critical  thinking” 

“mastery  of  a body  of  facts” 

“development  of  writing  skills” 

“acquisition  of  library  and  research  skills” 

“substantial  reading  and  writing  assignments” 

“analysis  and  interpretations  of  primary  sources” 

“assignments  that  foster  critical  thinking  and  interpretative  skills” 

Niles  et  al.,  1990 

“liberal  education” 

“equip  individuals  with  the  skills  and  understanding  necessary  to  perform  their  duties” 

Bursuck,  1994 

“educational  rigor” 

“higher  standards” 

“increased  accountability” 

“higher  expectations  for  student  performance” 
“assigning  more  homework  to  students” 

Roundtable,  1997 

“academic  rigor  demanded  by  the  national  curriculum”  - content  coverage? 
Skills  vs.  knowledge  outcomes  in  curriculum 

Hayes,  1997 

“the  ultimate  test  of  the  student’s  academic  preparation  and  intellectual  ability  . . . comes 
at  the  time  of  university  admission  exams” 

“academic  standards” 

“academic  demands” 

“scholarship  and  academic  excellence” 

“academic  achievement” 

“academic  effort” 

Table  I.  Key  phrases  related  to  academic  rigor  from  the  literature. 
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Craig  Nelson,  a Biology  professor  at  Indiana  University,  spoke  at  Indiana  University’s  18^^  Annual  Spring 
Symposium,  Listening  to  Learners:  Creating  Contexts  for  Student  Success.  The  title  of  his  presentation  was  “How 
We  Defeat  Ourselves:  Dysfunctional  Illusions  of  Rigor.”  Although  his  talk  did  not  directly  address  what  academic 
rigor  is,  it  shed  light  on  the  subject  by  addressing  what  he  felt  academic  rigor  is  not.  The  illusions  of  rigor  that  he 
presented  expressed  the  idea  that  rigor  is  not: 

• Hard  courses  that  weed  out  weak  students 

• Avoiding  “pampering”  students  (e  .g.,  giving  them  flexibility  etc.) 

• Covering  more  content  (Nelson,  1989) 

• The  difficulty  of  the  exams 

(Nelson,  2000a) 

In  an  article  (1997)  entitled  “Tools  for  Tampering  with  Teaching’s  Taboos,”  Nelson  prornotes  the 
importance  of  “critical  thinking”  in  the  college  classroom.  He  provides  several  methods  or  tools  for  getting  students 
to  think  critically  about  what  they  are  learning. 

Methodology 

The  goal  of  the  research  was  to  answer  the  research  questions  outlined  above.  The  following  general  steps 
were  taken  in  this  effort. 

The  first  step  of  this  phase  was  to  determine  how  other  individuals  and  institutions  have  defined  academic 
rigor,  by  conducting  an  exhaustive  review  of  the  past  research  in  this  area.  Because  of  the  variations  of  academic 
rigor,  and  the  fact  that  the  concept  was  often  referred  to  under  different  names,  this  research  effort  was  quite 
challenging. 

The  findings  from  the  literature  review  were  used  to  inform  the  questions  that  were  asked  in  an  initial  set  of 
interviews  of  a purposefully  chosen  group  of  instructors  at  the  aforementioned  large  Midwestern  state  university. 
These  instructors  were  chosen  from  throughout  the  university’s  many  schools  and  departments  (see  Table  2).  The 
purpose  of  this  initial  data  gathering  was  to  acquire  a larger  body  of  information  regarding  academic  rigor  from 
which  more  targeted  and  precise  questions  could  be  created  for  the  final  survey. 

In  addition  to  the  demographic  information  presented  in  Table  2,  it  is  important  to  note  that  of  the 
instructors  interviewed  fo  r this  study  (n=8),  2 were  full  professors,  3 were  associate,  visiting  or  adjunct  faculty,  and 
3 were  advanced  graduate  students  serving  as  associate  instructors.  Of  this  group  of  instructors,  3 had  taught  online 
as  well  as  on-campus,  and  thus  our  findings  for  question  4 will  be  based  on  responses  from  these  instructors. 


Department 

Gender 

Years  Teaching  Online 

Years  Teaching  On -campus 

School  of  Nursing 

Female 

2 

12 

Leisure  and  Recreation 

Male 

0 

3 

Biology 

Male 

0 

34 

English  & School  of 
Continuing  Studies 

Female 

0 

3 

Telecommunications 

Female 

0 

5 

Language  Education 

Male 

2 

5 

Curriculum  and 
Instruction 

Male 

0 

25 

Instructional  Systems 
Technology 

Female 

3 

5 

Table  2.  Demographic  information  about  interview  participant 

The  questions  posed  of  our  interview  subjects  were  as  follows: 

• What  is  your  personal  definition  of  academic  rigor? 

• How  important  do  you  feel  academic  rigor  is?  Why? 

• What  do  you  do  to  ensure  academic  rigor  in  your  courses? 

• Do  you  do  different  things  in  your  online  courses  compared  with  your  on-campus  courses  to 
ensure  academic  rigor?  If  so,  what? 
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The  final  step  in  our  research  study  procedure  was  to  review  and  summarize  the  data  collected  through  the 
interviews.  Similarities  and  differences  in  the  perceptions  related  to  academic  rigor  were  noted. 

Findings 


Definition  of  Academic  Rigor 


What  it  IS 

What  it  AS’ A/or 

critical  thinking 

not  grades 

high  standards  and  expectations 

not  memorization 

process  more  than  product 

not  regurgitation 

cognitive  development 

Table  3.  Common  responses  to  the  academic  rigor  definition  question 


What  it  AS’ 

What  it  AS’ /VOr 

norm  referenced 
content  coverage 
gatekeeper  of  academy 
scholarship/peer  review 
student  involvement 
pushing  students  beyond  comfort  level 

not  time  based 
not  giving  lots  of  tests 
not  lots  of  people  who  fail 

Table  4.  Interesting  but  uncommon  responses  to  the  academic  rigor  definition  question. 

Our  analysis  of  the  interview  data  related  to  the  definition  of  academic  rigor  developed  into  a two  separate 
listings:  of  common  responses  to  what  academic  rigor  was  and  was  not  (Table  3),  and  interesting  but  less  common 
responses  to  the  question  (Table  4).  Given  that  the  responses  to  the  question  had  such  interesting  variations,  we  felt 
it  was  necessary  to  report  them,  especially  given  that  survey  data  might  potentially  find  that  the  alternate  definitions 
might  be  held  by  a significant  number  of  respondents. 

The  Importance  of  Academic  Rigor 

Based  on  our  interviews  with  faculty  members,  we  found: 

• As  expected,  all  participants  agreed  that  academic  rigor  is  important 

• Two  instructors  said  that  its  importance  varied  depending  on  the  level  of  the  course  (e.g.,  rigor 
was  more  important  for  higher-level  courses) 

Methods  for  Ensuring  Academic  Rigor  with  On -campus  Courses 

After  analyzing  the  data  from  our  interviews  with  faculty  members,  we  found  a diverse  collection  of 
strategies  that  nonetheless  seemed  to  fit  well  together.  We  divided  these  strategies  into  seven  categories,  with  one  or 
more  strategy  associated  with  each  category.  Unfortunately,  it  is  beyond  the  scope  of  this  study  to  go  into  specific 
details  about  how  each  strategy  should  be  implemented,  but  hopefully  this  list  of  strategies  should  be  of  benefit  to 
online  educators. 

Expectations: 

• Make  expectations  clear 

• Make  expectations  high 

• Make  grading  criteria  explicit 
Selection  of  Readings: 

• Provide  high  quality  readings/texts 

• Provide  materials  a step  above  students’  level 

• Provide  a variety  of  materials 


334  8^3 


Critical  Thinking: 

• Require  and  support  students’  efforts  to  examine  multiple  issues  from  mu  Itiple  perspectives 

• Require  students  to  cite  the  readings 

• Require  student  reflections 

Modeling: 

• Model  good  scholarship 

• Model  rigorous  thinking 

Support: 

• Provide  appropriate  scaffolding  for  learners 

Discourse: 

• Ask  questions  that  encourage  thinking  about  relationships  and  not  memorization 

Assignments: 

• Create  challenging  assignments 

• Create  assignments  that  require  higher  cognitive  processes 

• Design  authentic  and  realistic  projects  and  problems  for  the  students 

Methods  for  Ensuring  Academic  Rigor  with  Online  Courses 

Based  on  our  interview  data,  we  found  that,  in  general,  similar  methods  for  ensuring  academic  rigor  seem 
to  be  used  for  online  as  well  as  on-campus  courses,  which  means  that  the  list  directly  above  should  be  relevant  also 
for  online  courses.  However,  some  instructors  did  note  some  differences  in  their  teaching  methods  related  to 
academic  rigor  in  the  online  delivery  method.  Some  instructors  felt  that,  in  their  online  courses,  they: 

• Put  more  demands  on  students  because  of  the  absence  of  feedback  fro  m face-to-face  contact 

• Structured  online  discussions  more  carefully 

• Made  expectations  even  more  explicit  than  they  did  with  on-campus  students 

• Expended  more  effort  to  get  feedback  from  students 

Interestingly,  one  instructor  went  on  to  say  that  if  content  coverage  was  equated  to  academic  rigor  then: 

• The  same  methods  will  work  online  as  in  the  classroom 

• The  online  course  can  be  more  academically  rigorous  than  the  on -campus  course  because  of  up-to- 
date  resources 

However,  he  stated  that,  in  his  mind,  rigor  was  not  the  same  as  content  coverage,  and  that  other  things,  such 
as  the  quality  of  instructor-student  and  student-student  interaction,  were  as  important,  if  not  more  so,  than  merely 
providing  specified  content,  and  thus  required  different  strategies  for  online  delivery.  He  did  not,  unfortunately, 
specify  these  differing  strategies. 

Rigor,  Difficulty,  and  Support 

One  of  the  faculty  interviewees,  Craig  Nelson,  has  been  nationally  recognized  for  excellence  in  teaching  at 
a post-secondary  level.  Nelson  (2000b)  provided  some  insights  related  to  academic  rigor  that  we  feel  are 
particularly  relevant  to  the  online  delivery  of  courses. 

His  first  observation  was  that  oftentimes  faculty  and  students  have  a false  conception  of  what  academic 
rigor  is.  We  often  confuse  the  rigor  of  a course  with  the  difficulty  of  a course.  He  claimed  that  a more  accurate 
synonym  to  describe  academic  rigor  would  be  the  word  challenge  ” It  was  insightful  for  us  to  think  of  academic 
rigor  or  academic  challenge  as  having  the  following  relationship  with  course  difficulty  and  the  support  provided  to 
the  distance  students.  Nelson  (2000b)  related  these  three  factors  using  the  equation  in  Figure  1 . 


Academic  rigor 


Support 


Figure  1.  Relationship  between  course  difficulty,  academic  rigor  and  student  suppo  rt. 

This  helps  us  to  understand  that  the  difficulty  of  a course  is  inversely  proportional  to  the  support  that 
students  are  given.  Thus  the  difficulty  of  a course  increases  as  the  support  for  the  students  in  the  course  decreases. 
Similarly,  the  difficulty  of  a course  could  be  low  and  the  course  could  still  be  academically  rigorous  if  adequate  and 
appropriate  support  is  provided  to  the  students. 

This  understanding  of  the  relationship  difficulty,  academic  rigor,  diXid  support  is  particularly 

important  in  a distance  learning  environment  because  of  high  dropout  rates.  Students  are  more  likely  to  drop  out  of 
a distance  program  as  the  courses  become  more  difficult  and  exceed  the  students*  ability  or  willingness  to  persevere. 
Instructors  teaching  distance  education  courses,  Nelson  (2000b)  argues,  should  maximize  the  challenge  that  a 
student  experiences  but  not  the  difficulty  that  a student  experiences  trying  to  meet  that  challenge. 

Limitations 

It  must  be  stated  at  this  point,  that,  even  though  our  findings  from  our  interviews  were  quite  interesting  and 
potentially  quite  useful,  they  are  based  on  quite  a small  sample  size  (especially  research  question  4,  which  was 
answerable  by  less  than  half  of  our  already  small  sample,  who  taught  online  as  well  as  face-to-face).  Another 
important  limitation  is  that  the  sample  only  included  faculty  from  a single  institution. 

Conclusions 

The  purpose  of  this  study  was  to  gain  a better  understanding  of  academic  rigor  in  online  and  on-campus 
postsecondary  education.  In  order  to  do  this  accurately,  the  researchers  attempted  to  define  factors  that  instructors 
understood  "academic  rigor"  to  consist  of,  as  well  as  quantify  how  important  academic  rigor  is  to  these  instructors. 
Information  about  strategies  used  to  ensure  academic  rigor  in  online  and  on-campus  courses  was  also  gathered. 

While  the  results  reported  in  this  report  are  based  on  a very  small  sample  size,  the  findings  are  promising, 
and  the  researchers  feel  that  this  study  will  be  of  great  interest  to  both  on -campus  and  online  educators  and 
administrators  interested  in  ensuring  the  academic  rigor  of  their  courses. 
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Problem-based  learning  (PBL)  has  great  potential  for  inspiring  K-12  learning.  KaAMS,  an  example  of 
PBL,  was  designed  to  help  teachers  inspire  middle  school  students  to  learn  science.  The  kids  participate  as 
scientists  investigating  environmental  problems  using  NASA  airborne  remote  sensing  data.  This  paper 
provides  an  overview  of  the  instructional  design  challenges  in  creating  web-based  PBL  teacher  support 
materials,  the  PBL  model  we  selected,  the  activities  embedded  in  the  PBL,  and  initial  results  of  classroom 
trials. 

Introduction 

Kids  can  be  motivated  and  inspired  by  making  direct  contributions  to  solving  real  scientific  issues.  Can 
teachers  be  inspired  too?  Through  a PBL  approach,  KaAMS,  a NASA  funded  project  guides  teachers  to  take  middle 
school  children  on  live  and  past  NASA  airborne  missions  to  collect  data  to  study  two  environmental  issues.  The 
ultimate  mission  of  the  project  was  to  inspire  kids  to  learn  and  develop  a career  interest  in  science,  math,  technology 
and  geography  by  their  participating  as  scientists  in  activities  punctuated  by  “bursts”  of  interactive  events 
culminating  in  the  analysis  of  data  from  NASA  airborne  missions.  The  mission  is  accomplished  by  providing 
resources  to  teachers  to  use  with  middle  school  children.  To  do  this,  we  developed  two  problem  based  learning 
modules  addressing  two  different  environmental  issues — active  lava  flows  and  coral  reefs  in  Hawaii.  The  modules 
consisted  of  new  lesson  plans  that  could  be  used  flexibly  by  many  teachers,  and  that  use  existing  NASA  and  other 
web  resources.  The  goal  was  to  harness  those  resources  that  exist  rather  than  create  totally  new  ones. 

Problem-Based  Learning  Literature — the  basis  for  KaAMS 

The  conceptual  framework  of  problem-based  learning  model  for  KaAMS  is  based  on  the  perspectives  and 
implications  of  problem-based  learning  literature.  Problem-based  learning  as  an  instructional  model  associated  with 
the  new  learning  paradigm  (e.g.,  Reigeluth,  1999)  has  been  implemented  in  diverse  content  domains  such  as  medical 
education,  business  education,  social  education,  and  science  education.  Problem-based  learning,  in  general, 
encourages  students  to  engage  in  learning  activities  to  solve  a real  world  problem  (Duffy  & Cunningham,  1996). 
According  to  problem-based  learning  researchers  (Barrows,  1986,  1992;  Savery  & Duffy,  1995;  Schwartz,  et  al, 
1999),  key  characteristics  of  problem-based  learning  include  the  following; 

• Real-world  problems  with  a motivational  context  to  drive  learning.  A real  world  problem  is  used  as  a 
stimulus  for  authentic  activity. 

• Given  a problem  space,  students  generate  their  own  learning  goals  in  terms  of  what  they  will  attain. 

• Multiple  learning  resources  including  print,  electronic  and  humans  are  provided  for  the  student  to  develop  a 
deep  understanding  about  content  knowledge  related  to  the  problem  and  apply  that  knowledge  into  the 
problem  solving  activities. 

• Students  as  active  problem  solvers  work  with  their  peers,  teachers,  and  experts  to  share  their  different 
perspectives  and  develop  deeper  knowledge  on  a subject  area. 

• By  placing  students  in  learning  by  doing  situations,  students  develop  a disciplinary  knowledge  base, 
problem  solving  skills,  reflective  thinking  skills,  and  collaboration  and  communication  skills. 

• Teachers  play  a role  as  coach  or  facilitator  that  supports  students’  learning  and  problem  solving  activity, 
rather  than  directly  teaching  what  students  should  know  and  how  students  should  solve  a problem. 

In  general  problem  based  learning  (e.g.,  Barrows,  1986,  1992;  Savery  & Duffy,  1995;  Schwartz,  et  al,  1999), 
students  engage  in  the  following  five  stages  of  a learning  process: 

• Present  a problem — Students  start  with  a presentation  of  a real  world  problem. 
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• Generate  what  students  know  and  what  they  need  to  know — Students  actively  define  problems  and  generate 
what  they  know  and  what  they  need  to  know  based  on  their  prior  knowledge  and  experience.  They  are 
encouraged  to  identify  learning  issues  or  knowledge  necessary  to  construct  an  understanding  about  how  to 
solve  problems. 

• List  possible  actions— on  the  previous  activity,  students  discuss  and  come  up  with  strategies  and 
activities  for  solving  the  problem. 

• Collect  and  analyze  information — All  students  engage  in  gathering  information  from  available  learning 
resources  ranging  from  print-based  materials,  electronic  and  human  resources  and  from  the  designated 
facilitator.  After  gathering  the  information,  they  analyze  and  evaluate  information  in  terms  of  what  is  most 
useful  or  what  is  not  useful  to  solve  the  problem.  They  discuss  and  negotiate  their  perspectives  about 
alternative  solutions  with  peers,  their  teacher,  and  experts. 

• Present  and  share  solutions— Ihty  finally  propose  their  solutions,  share  them  with  their  peers  and  experts 
who  might  provide  different  perspectives  to  the  solutions,  and  revise  their  solution  based  on  feedback  from 
their  peers  or  experts. 

The  KaAMS  Model 

From  these  perspectives  and  implications  of  the  problem-based  learning  literature,  we  developed  the 
KaAMS  problem  based  learning  model  with  four  learning  stages  in  which  middle  school  students  engage:  1) 
problem  scenario,  2)  propose  ideas/search  information,  3)  conduct  mission/collect  and  analyze  data,  and  4)  propose 
solution.  Each  learning  stage  of  the  KaAMS  problem  based  learning  model  includes  the  following  key  attributes: 

• Authentic,  ill -structured  problem  situation 

• Assumption  of  roles  by  the  students 

• Reflections  about  what  they  know,  what  they  need  to  know 

• Planning  the  investigation  procedure 

• Access  to  rich  NASA  web  resources 

• Active  investigation 

• Learning  activities  situated  within  real  NASA  missions 

• Reflective  thinking  exercises 

• Peer  and  expert  collaboration 

• Learner  activities/tools  in  interpreting  data  gathered 

• NASA  scientist  support 

• Shared  solutions  with  peers  and  experts 

Conceptual  Framework  for  KaAMS 
Foundation  of  KaAMS 

Conceptually,  the  KaAMS  framework,  as  shown  in  Figure  1,  is  built  upon  the  premise  and  foundation  that 
among  all  NASA  web  resources  from  all  aspects  of  the  agency,  a multitude  of  resources  can  be  used  in  the 
classroom.  These  resources  are  filtered  through  a second-level  premise,  which  is  the  Web-Enhanced  Learning 
Strategies  (WELES)  interface.  This  interface  helps  to  sift  through  the  available  resources  for  elements  and 
composite  sites  that  are  appropriate  for  use  by  middle  school  teachers  and  students.  These  resources  are  then  used  in 
four  parts  of  a lesson  plan — frame/inform/explore/try.  The  third  premise  is  that  teachers  can  use  real  web  resources 
from  real  NASA  missions  in  a problem  based  lesson  format.  Finally,  these  three  levels  of  resources  are  harvested  as 
part  of  the  KaAMS  PBL  lesson  plans. 

Students  are  presented  with  an  environmental  problem  that  is  of  concern  of  to  a NASA  mission.  They  begin 
a series  of  problem-solving  lessons  from  which  they  develop  content  and  applied  knowledge  by  participating  in 
problem  solving  activities.  Through  a series  of  framing  and  informing  activities,  students  search  for  additional 
information  on  the  problem,  develop  an  understanding  of  the  science  of  the  problem,  and  propose  a solutions  for 
conducting  a mission  that  will  provide  remote  sensing  data  to  solve  the  problem.  Students  become  involved  in 
“bursts”  of  activities  to  conduct  a mission,  collect  and  analyze  data.  Finally,  students  summarize  their  findings  in 
several  different  ways  and  go  public  to  share  what  they  have  learned  with  classmate  and/or  other  outside  their 
classroom.  One  important  note  is  that  the  students  participate  in  reflective  activities  throughout  the  entire  process. 

Developmentally  Appropriate  Lessons 

Also  evident  in  the  design  were  the  following  key  characteristics  we  found  about  middle  school  students 
(From:  This  We  Believe). 
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• Moving  from  concrete  to  abstract  thinking 

• Curious  on  a wide  range  of  topics,  few  of  which  are  sustained 

• Prefer  active  over  passive  learning 

• Respond  positively  to  participating  in  real  life  learning  situations 

• Are  inquisitive  and  challenge  adults 

• Desire  recognition 


•From  concrete  to  \ 
abstract  thinking 
•Curious  on  a wide 
range  of  topics,  few 
of  which  are 
sustained 
•Prefer  active  over 
passive  learning 
•Respond  positively) 
to  participating  in 
real  life  learning 
situations 

•Are  inquisitive  and 
challenge  adults 
•Desire  recognition 


Figure  I.  KaA MS  Foundation 
Links  to  National  Standards 

To  maintain  the  link  to  the  National  Standards,  we  have  completed  an  analysis  of  the  NSTA/NRC  standards 
and  the  AAAS  Project  2061  Benchmarks  to  target  in  the  KaAMS  Project.  Each  lesson  plan  links  to  the  specific 
national  education  standards  that  might  be  satisfied  by  completing  the  lesson  activities 
Flexibility 

Since  flexibility  is  very  important  to  maximize  the  usability  of  this  site,  we  have  designed  the  site  for  the 
teacher.  The  framework  is  constructed  so  that  the  teacher  is  in  control  of  how  much  and  what  types  of  the  available 
activities  that  his  or  her  students  actually  see.  He  or  she  can  start  from  Phase  1 and  proceed  to  Phase  4,  or  he  or  she 
can  just  go  the  activities  of  Phase  3,  for  example. 

Mapping  Design  Attributes  of  KaAMs  onto  PBL  and  the  Scientific  Process 

Problem-based  learning  and  the  scientific  process  follow  similar  steps.  This  cross  over  made  explaining 
the  learning  process  to  content  experts  very  easy.  See  Table  1 . Some  examples  of  how  the  design  attributes  were 
built  into  the  KaAMs  is  shown  in  the  last  column. 


Table  1.  Mapping  KaAMS  onto  the  PBL  Process 


PBL  Process 

Scientific  Process 

KaAMS 

Design  Examples 

Problem  Clarification 

Identify  Problem 

Problem  Scenario 

• Two  problems — 
finding  lava  flows  for 
the  Pacific  Disaster 
Center,  and 
determining  if  the 
coral  reef  need 
protection  for  a real 
Congressional 
Executive  Order 

Plan  Development 

Research  ideas  and 

Propose  Ideas/Search  for 

• Activity  Sheets  as 
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Develop  plan  for 
investigation 

Information 

Who,  What,  When, 
Where,  Why  and  How 
questions  to  determine 
what  the  students 
know 

• Students  complete 
reflection  journals 

• Find  information 
using  existing  NASA 
web  resources 

• Participate  in  activity 
bursts  to  explore  new 
concepts 

Collect  Information 

Collect  Data 

Conduct  Mission  and 
Collect  Data 

• Students  think 
critically  about  which 
aircraft  can  run  their 
mission  and  select 
from  several 
possibilities 

• Students  plan  the 
actual  mission  and 
compare  it  to  the 
actual  NASA  mission 

Analyze  Information 

Analyze  Data 

Analyze  Data 

• Students  use  actual 
data  from  the  live 
missions  to  draw 
conclusions 

• Students  use 
guidelines  from 
NASA  scientists  for 
interpretation  of  their 
data 

Present  Solutions 

Report  Findings 

Propose  Solution: 
(Go  Public) 

• Write  a report  to  the 
Pacific  Disaster 
Center, 

• Make 

recommendations  to 
the  President  in 
response  to  his 
executive  order 

o 

ERIC 


Alpha  Testing  Phase 

The  assessment  strategy  for  the  entire  KaAMS  project  was  divided  into  three  major  phases  designed  to 
capture  data  that  would  support  initial  product  development  (alpha  testing),  on-going  resources  development  and 
implementation  planning  (beta  testing),  and  analysis  of  impact  on  the  stakeholders  in  the  learning  environment 
(research-impact  analysis).  See  Figure  2.  The  diagram  below  illustrates  the  flow  of  alpha,  beta,  and  research 
processes  used  for  each  of  the  two  major  products  being  developed;  (1)  volcano  mission  and  (2)  coral  reef  mission. 
This  report  summarizes  overall  data  collection  methods  and  procedures  as  well  as  the  findings  from  the  alpha 
development  cycle  for  Products  1 and  2 - Mission:  Studying  active  lava  flows. 

ALPHA  TEST  BETA  TEST  (new  classrooms) 

» — > 


IMPLEMENT 
(new  classrooms) 


Figure  2.  Overall  Assessment  and  Research  Strategy 
Methods 


During  the  Alpha  testing  formative  and  summative  evaluation  data  were  collected.  With  project 
enhancement  in  mind,  data  collected  from  key  stakeholders  during  the  alpha  (initial  formative  development)  testing 
phase  included  five  levels  of  assessment:  (1)  reaction,  (2)  learning  gains,  (3)  performance,  (4)  education  system 
changes,  and  (5)  impact  on  the  greater  society.  Research  protocols  were  also  tested  to  assess  their  effectiveness  in 
measuring  the  effects  of  KaAMS  materials  on  teachers,  students,  and  stakeholders  in  the  surrounding  community, 
namely  parents.  See  Table  2. 

Table  2:  Research  questions  and  assessment 

Research  Questions Assessment 

How  are  teachers  using  KaAMS  and  NASA  resources?  Performance 

How  are  teachers  changing  their  teaching  practices  (e.g.  teaching  strategies.  Performance/ 
incorporation  of  NASA  resources,  etc.)  over  time  as  a result  of  using  System  Change 

KaAMS  and  NASA  resources? 

How  are  student  levels  of  interest  in  pursuing  science-related  career  Learning  Gains 

changing  over  time  as  a result  of  using  KaAMS  and  NASA  resources? 

How  does  the  use  of  KaAMS  diffuse  to  the  surrounding  school  system?  Impact  on  Greater 

Society 

All  formative  evaluation  instruments  were  administered  throughout  the  alpha  testing  phases  to  gather 
feedback  from  teachers  while  preparing  for  and  using  the  KaAMS  materials.  Interviews,  observations,  and  focus 
groups  were  also  conducted  at  least  once  per  week  with  teachers  and  students  during  the  6-month  alpha  classroom 
trials.  The  summative  instruments  were  administered  to  teachers  at  the  end  of  the  KaAMS  classroom  trials  and  final 
interviews  and  focus  groups  were  conducted  with  teachers  and  students.  The  research  instruments  were  administered 
to  teachers  and  students  for  pre-  and  post-test  data  collection  and  at  and  additional  1 -month  follow-up  period  for 
students.  Parents  were  surveyed  at  the  beginning  and  end  of  the  school  year. 

Subjects 

Three  middle  schools  in  a rural  Pennsylvania  school  district  participated  in  the  KaAMS  alpha  test 
classroom  trials.  East,  West,  and  Distant.  Six  different  classrooms  from  these  schools  were  actively  involved.  Four 
were  6*^  grade,  one  was  7*^  grade,  and  one  was  an  8*^  grade  honors  class.  The  six  teachers  who  participated  provided 
data  about  themselves,  their  classrooms,  and  success  of  using  KaAMS  materials  during  the  alpha  testing  cycle. 
Teaching  experience  ranged  from  3 to  23  years;  initial  preferences  for  primary  teaching  strategies  included  hands-on 
activities,  collaborative  activities,  role  play,  and  problem-based  learning;  half  of  the  teachers  had  moderate  success 
using  web  resources  in  their  classrooms  the  other  half  had  not  used  such  resources  in  their  classrooms. 

Data  were  collected  from  a total  of  144  students,  82  were  boys,  59  girls  and  3 did  not  respond  to  the  gender 
question.  On  average,  the  students  had  a moderate  level  of  interest  in  pursuing  science. 

One  hundred  and  fifty  three  parents  of  KaAMS  students  returned  surveys  indicating  their  initial  perceptions  of 
science  in  their  school  and  child’s  success  and  interest  in  science  as  well  as  reporting  their  highest  attained  level  of 
education.  A majority  of  the  parents  did  not  have  college  degrees,  worked  in  non-science  related  jobs,  and  had  a 
neutral  opinion  of  their  child’s  school’s  science  program. 

Measures  and  Instruments 

Formative  and  Summative  evaluation:  A series  of  instruments,  observation  protocols,  and  interview 
protocols  were  developed  to  collect  formative  and  summative  data  from  the  teachers  and  students  during  the  alpha 
testing  development  cycle. 

Teachers  were  asked  to  review  the  KaAMS  lesson  plans,  prepare  to  use  the  lesson  plans  in  their 
classrooms,  and  complete  evaluation  surveys  after  each  lesson  and  at  the  end  of  the  trial  indicating  ease  of  use;  value 
of  resources,  instructions,  and  assessment  guidelines  provided;  success  of  activities;  amount  of  preparation  time; 
descriptions  of  the  classroom  activity  during  KaAMS  lessons;  and  general  feelings  about  using  KaAMS  for  teaching 
and  learning.  Teachers  were  also  asked  to  share  feedback  during  interviews  and  focus  groups  including  responses  to 
questions  such  as:  What  did  you  like/not  like  about  the  supporting  website?  What  parts  of  the  lesson  plans  did  you 
use  - why?  What  additional  support  materials  did  you  need  to  use  these  materials?  What  additional  materials  did  the 
students  need?  What  would  you  change? 
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Periodically  students  were  asked,  during  interviews  and  focus  groups,  to  respond  to  questions  such  as: 
What  did  you  like/not  like  about  the  KaAMS  activities?  How  useful  were  the  internet  sites?  What  was  happening  in 
the  classroom  during  KaAMS?  What  did  you  learn?  and  what  would  have  made  these  activities  more  useful  to  you? 
Observational  data  were  collected  several  times  during  the  classroom  trial  that  lasted  between  3 and  6 months, 
depending  on  the  classroom  teacher.  Observation  data  were  collected  on  how  the  teachers  used  the  materials,  how 
the  students  participated  in  the  activities,  and  artifacts  developed  by  the  teacher  or  students  during  the  KaAMS 
lessons. 

Research:T\\z  research  questions  were  focused  on  the  teachers,  students,  and  parents.  Teachers  completed 
an  on-line  instrument  eliciting  background  information,  preferences  for  classroom  activities,  and  attitudes  toward  the 
use  of  web  resources  in  the  classroom.  The  instrument  was  a combination  of  an  attitude  survey  previously  developed 
and  validated  for  similar  research  (Koszalka,  2000),  a series  of  questions  related  to  perceptions  of  their  school’s 
ability  to  support  the  use  of  internet  technology  in  the  classroom  (McCarthy,  Grabowski,  & Koszalka,  1998),  and 
preferences  for  teaching  styles  (Grabowski,  Koszalka,  & McCarthy,  2000;  Koszalka,  Grabowski,  & McCarthy, 
2000).  This  instrument  was  administered  at  the  beginning  of  the  classroom  trial  period  and  the  end  (pre-post  test). 

Data  were  collected  on  student  level  of  career  interest  in  science,  pre-,  post,  and  1 -month  after  using 
KaAMS  materials.  Student  career  interest  surveys  were  purchased  from  the  APA.  The  survey  also  included  a series 
of  questions  developed  to  assess  reflective  thinking  (Koszalka,  et  al.,  2001)  and  gather  demographic  data. 

Parents  were  asked  to  complete  surveys  at  the  beginning  and  end  of  the  school  year  to  assess  their 
perceptions  of  their  child’s  school’s  science  program.  The  questions  were  taken  from  previous  research  on 
measuring  parents’  perceptions  of  school  programs. 

Results 


The  initial  formative  feedback  provided  guidance  in  designing  support  structures  for  the  KaAMS  website 
that  helped  the  alpha  teachers  connect  NASA  science  to  their  curriculum  and  prompt  active  student  involvement,  as 
scientists,  during  science  class.  The  results  from  the  formative  and  summative  evaluation  resulted  in:  development 
of  enhanced  lesson  plan  structures  for  the  KaAMS  website,  new  content  support  for  teachers  that  strengthened  the 
relationship  between  the  overall  problem  scenario  and  learning  activities,  further  instructions  to  ‘coach’  teachers  in 
using  PBL,  web  technology,  and  activities  that  prompt  student  reflection,  stronger  ties  between  lesson  plans  and 
national  education  standards  and  curriculum  requirements,  and  enhanced  activities  that  will  better  meet  kids’  needs. 
The  initial  research  findings  from  the  pilot  classrooms  were  very  encouraging.  Although  caution  is  warranted  in 
interpreting  these  results,  analysis  of  the  research  data  collected  during  the  alpha  testing  cycle  showed  significant, 
yet  minor  changes  in  teachers,  students,  and  parents  after  the  use  of  KaAMS  in  the  classroom.  Table  3 summarizes 
research  findings  in  accordance  with  the  KaAMS  project  research  questions: 

Table  3.  Research  questions  and  Alpha  Preliminary  Findings 

Research  Questions ^ Alpha  Preliminary  Findings 

How  are  teachers  using  KaAMS  and  NASA  Teachers  noted  the  flexibility  of  KaAMS  resources  and  used  them 

resources? in  a variety  of  ways  to  enhance  or  change  the  way  they  teach. 

How  are  teachers  changing  their  teaching  Several  of  the  teachers  tried  new  ways  of  integrating  the  web  and 

practices  over  time  as  a result  of  using  collaborative  activities  into  their  teaching;  changed  their  preferred 

KaAMS  and  NASA  resources?  method  of  teaching  and  the  types  of  resources  they  used  regularly 

in  their  classrooms,  and  their  attitudes  toward  using  web 

resources  in  the  classroom. 

How  are  student  levels  of  interest  in  pursuing  Significant  increases  in  student  level  of  science  career  interest 
science-related  career  changing  over  time 

as  a result  of  using  KaAMS? 

How  does  the  use  of  the  KaAMS  products  Parent  perceptions  of  their  child’s  school’s  emphasis  on  science, 

diffuse  to  the  surrounding  school  system?  school’s  ability  to  provide  good  science  experiences,  and  use  of 

appropriate  science  resources  were  higher  at  the  end  of  the  school 
year  than  in  the  beginning. 

We  believe  that  we  are  providing  teachers  with  a venue  and  structure  for  using  NASA  web-based  materials 
in  their  classroom  in  meaningful  and  contextualized  ways  that  will  support  student  knowledge  development  in  the 
content  and  processes  of  science.  Through  their  high  quality  materials,  NASA  can  make  an  impact  on  science  in  the 
classroom,  which  in  combination  with  KaAMS  strategies  can  change  teaching  practice,  impress  middle  school  kids 
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with  the  importance  of  and  strategies  for  conducting  good  science — the  ultimate  goal  being  to  influence  career 
aspirations  of  these  kids  toward  science. 
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Abstract 

Designing  and  building  quality  distance  education  is  a challenge  facing  many  organizations.  Option  Six  is 
an  independent  company  building  customized  e-learning  solutions.  Over  the  last  two  years,  the 
instructional  designers  and  user  experience  analysts  at  Option  Six  worked  to  help  develop  a four-stage 
process  for  evaluating  e-learning  courses.  The  process  is  built  around  Donald  Norman  *s  (2000)  definition 
of  human-centered  design.  This  paper  describes  the  process  used  by  Option  Six  and  outlines  the  benefits 
and  challenge  of  human-centered  design  for  distance  education. 

Introduction 

Companies  and  universities  are  increasingly  moving  to  online  delivery  for  much  of  their  training  and 
education  needs.  Start-ups,  consulting  companies,  and  even  universities  are  touting  their  ability  to  deliver  quality  e- 
leaming  to  employees  worldwide.  With  so  many  entities  vying  for  students  on  a global  scale,  the  issue  of  quality  is 
becoming  an  increasing  concern.  Evaluating  distance  education  is  problematic  in  a number  of  ways.  First, 
traditional  student  evaluations  at  the  end  of  a face-to-face  course  typically  focus  on  the  experience  students  have 
with  a particular  instructor.  With  most  online  education,  these  methods  are  not  valid  as  a single  measure  because  the 
instructor’s  role  is  only  one  part  of  a much  broader  experience.  Technology,  the  user  interface,  and  the  design  of 
content  are  all  keys  in  understanding  the  distance  learner’s  experience. 

Second,  in  a face-to-face  course,  the  instructor  has  the  benefit  of  being  able  to  see  the  students’  reactions  to 
a number  of  varied  teaching  exercises.  Nodding  heads,  yawns,  and  verbal  interaction  all  serve  as  tacit  “formative” 
evaluation  of  a course  in  progress.  Distance  courses  do  not  have  such  luxury  and  not  having  that  interaction  can 
cause  severe  problems  for  the  students  and  the  instructor  (Wang-Chavez  & Branon  2000). 

Finally,  the  rapid  pace  of  change  in  e-leaming  makes  knowing  exactly  which  design  principles  are 
“correct”  difficult  to  determine  (Williams,  Paprock,  & Covington,  1 999).  Donald  Norman  President  of  UNext 
Learning  Systems  and  author  of  such  books  as  “The  Design  of  Every  Day  Things”  and  “The  Invisible  Computer”, 
has  noted  that  computers  are  complex  and  hard  to  use  because  we  are  asking  them  to  do  complex  tasks  (Norman, 
2000).  Few  tasks  are  as  complex  as  education  and  finding  the  “best  way”  to  design  online  education  is  likely  to  be 
an  elusive  goal  for  quite  some  time.  Human-centered  development,  however,  utilizes  input  from  students  in  the 
target  audience  during  the  earliest  stages  of  development.  By  having  students  involved  early,  unnecessary 
complexity  can  be  eliminated  and  learning  can  be  maximized. 

Option  Six 

Located  in  Bloomington,  IN,  Option  Six  opened  in  January  2000  as  the  Bloomington  Development  Center 
(BDC)  for  UNext,  Incorporated.  In  September  2001  the  BDC  spun  off  to  form  a new  company,  building  and 
designing  online  courses  for  a number  organizations.  The  team  at  Option  Six  has  built  courses  with  faculty  from 
Stanford,  Columbia,  Camegie-Mellon,  The  London  School  of  Economics,  the  University  of  Chicago,  and  Indiana 
University.  Students  at  a variety  of  organizations  including  General  Motors,  Merrill  Lynch,  UCLA,  and  MIT  have 
taken  courses  built  by  Option  Six  team  members. 

In  order  to  create  high-quality  courses.  Option  Six  utilizes  a team-based  approach  driven  by  human- 
centered  design  principles.  Each  team  is  comprised  of  members  from  a variety  of  disciplines.  The  point-person  for 
the  team  is  an  instructional  designer.  Instructional  designers  work  with  subject-matter  experts  (SME)  to  determine 
the  pedagogical  strategy  for  the  course.  Editors  set  the  tone  for  the  course  and  ensure  consistency  in  language  usage. 
Visual  designers  create  the  look  and  feel  for  a course,  including  graphics,  and  work  closely  with  multimedia 
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developers  to  build  interactive  components  (Flash,  video,  etc.).  User  Experience  (UE)  analysts  work  with  the  team 
to  gather  data  from  students  in  the  target  audience,  which  is  the  core  of  the  human-centered  design  process. 

Human-Centered  Design  Process 

Donald  Norman,  former  President  of  UNext  Learning  Systems,  defines  human-centered  design  in  his  book, 
The  Invisible  Computer: 

“It’s  a process  of  product  development  that  starts  with  users  and  their  needs  rather  than  with  the 

technology.  The  goal  is  a technology  that  serves  the  user,  where  the  technology  fits  the  task  and 

the  complexity  is  that  of  the  task,  not  the  tool.”  (p.l  85). 

Option  Six  uses  an  online  course  development  process  that  is  driven  by  this  definition.  The  process  has 
four  primary  components  that  focus  development  around  the  student,  rather  than  the  technology.  Guiding  the 
development  of  the  process  is  a desire  to  produce  the  highest  quality  online  courses  with  cost-effective  user-testing 
methods.  Each  phase  is  designed  to  maximize  resources  while  minimizing  the  impact  on  the  overall  development 
timeline. 

Each  stage  in  the  process  has  a specific  objective.  Early  stages  are  designed  to  catch  major  design  flaws 
and  to  identify  conceptual  difficulties  with  the  material.  Large  structural  flaws  in  the  interface  or  content  tend  to  be 
the  most  expensive  and  time-consuming  problems  to  correct  late  in  the  process.  The  difficulty  is  that  the  early 
stages  of  development  are  the  most  conceptual  and  can  be  somewhat  difficult  to  test  with  large  numbers  of  users. 

The  result  is  a process  that  starts  with  expert  evaluators  (User  Experience  analysts)  conducting  design 
consulting.  In  the  second  phase,  participatory  design,  two  or  three  students  test  components  of  the  course  to  identify 
learning  issues.  The  third  phase  is  an  accelerated  integration  test  in  a laboratory.  Finally,  an  instructional  pilot  is 
run  with  students  working  in  naturalistic  setting  (work,  home,  etc.)  at  a pace  that  would  be  expected  for  the  course. 
At  any  point  during  this  process  it  is  the  data  from  the  students  that  determines  whether  the  course  is  ready  for  the 
next  phase  of  development.  Multiple  revisions  to  materials  are  anticipated  and  even  expected. 

Design  Consulting 

Design  consulting  is  not  a stage  in  the  process  but  is  an  ongoing  element  throughout  development.  In  team 
meetings,  this  often  means  an  analyst  provides  past  student  testing  data  to  inform  decisions.  Most  importantly,  the 
analyst  acts  as  an  objective  set  of  ‘fresh  eyes’  looking  at  a course  in  early  stages  of  development.  The  goal  of  the 
analyst  in  this  phase  is  not  to  replace  student  testing  but  to  act  as  a ‘user  advocate’  within  the  design  team.  Once 
student  testing  begins,  the  design-consulting  role  includes  making  recommendations  based  on  user  feedback  and 
interim  evaluation  of  product  improvements.  Several  distinct  techniques  are  used  to  maximize  the  skills  of  the 
analyst  in  conjunction  with  student  data. 

Cognitive  Walkthrough 

Structured  inspection  methods  are  one  important  way  for  UE  analysts  to  determine  the  viability  of  course 
organization.  Cognitive  walkthroughs  are  a common  technique  for  usability  experts  and  involve  following  a 
predetermined  path  through  material  to  look  for  potential  points  of  failure  (Wharton,  Rieman,  Clayton,  & Poison, 
1994).  The  effectiveness  of  cognitive  walkthroughs  is  dependent  on  the  expertise  of  the  evaluator  and  will  not 
necessarily  catch  all  problems  (John  & Mashyna,  1997).  As  a part  of  a larger  user-testing  strategy,  however,  they 
are  quite  effective  for  providing  an  objective  eye  to  inform  the  design  team  of  major  flaws  in  interface  design. 

Heuristic  Evaluation 

Though  Option  Six  does  not  employ  heuristic  evaluations  for  all  courses,  this  method  has  value  when 
looking  at  new  interfaces  or  small  course  components.  Heuristic  evaluations  involve  setting  metrics  for  an  analyst 
to  use  when  assessing  a course.  The  technique  is  less  structured  than  a cognitive  walkthrough  and  allows  exploration 
of  the  course  environment  (Sears,  1997).  Analysts  are  looking  for  aspects  of  the  course  that  do  not  meet  certain 
broad  standards  (readability,  ease  of  understanding,  technical  problems,  etc.)  Again,  such  a technique  by  itself  does 
not  ensure  a high-quality  course  but,  as  a first-stage  pass  through  the  material,  it  catches  many  major  flaws. 

Participatory  Design 


346 


S35 


Participatory  design  is  the  fist  phase  of  a three-phase  testing  process.  Components  of  the  course  are  tested 
with  two  or  three  students  from  the  intended  target  audience.  The  emphasis  in  this  phase  of  testing  is  on  high-level 
conceptual  understanding  of  the  course  materials.  Students  work  through  a combination  of  early  screen  designs  and 
paper  prototypes  of  course  materials  and  give  continuous  feedback  to  the  analyst  about  whether  concepts  and 
procedures  are  clear  and  the  interface  design  is  usable.  As  the  title  implies,  the  user  becomes  a co-designer  in  the 
design  process  by  providing  suggestions  for  improvement  rather  than  simply  identifying  problems. 

The  timeline  for  a project  is  built  around  the  idea  that  development  will  require  multiple  iterations  based  on 
this  early  student  feedback.  Complex  content  or  media  often  undergo  multiple  changes  in  order  to  ensure  students 
are  able  to  effectively  navigate  and  learn.  Many  of  the  worst  problems  with  the  material  are  identified  and  corrected 
at  this  point  in  the  process,  saving  time  and  money  as  the  project  moves  toward  completion. 

Integration  Testing 

When  development  is  nearly  complete,  all  of  the  components  are  assembled  in  electronic  form  and 
presented  as  a complete  course.  Six  paid  students,  who  have  the  same  background  as  the  intended  target  audience, 
work  in  a lab  environment  to  take  the  course.  In  this  phase,  there  is  less  emphasis  on  having  the  student  interact 
with  the  design  team.  The  timeframe  is  compressed  (students  work  six  hours  per  day  online)  and  data  is  gathered 
through  observation  and  interviews,  rather  than  participatory  design  sessions. 

In  university  credit  courses,  an  onsite  instructor  is  also  added  during  this  phase  to  grade  assignments  and 
provide  feedback  to  the  team  about  whether  students  are  meeting  the  learning  objectives.  The  analyst  works  with 
observation,  interview,  and  instructor  data  to  develop  a profile  of  overall  course  effectiveness.  Problems  with 
individual  parts  of  the  course  are  noted  and  the  development  team  makes  revisions  based  on  the  feedback.  Severe 
problems  (e.g.,  overall  course  structure  is  ineffective)  are  rare  at  this  stage  but  if  any  are  found,  the  entire  course  is 
retested  after  revisions. 

Instructional  Pilot  Testing 

After  the  development  team  is  confident  all  major  issues  have  been  resolved,  the  completed  course  is 
uploaded  to  external  web  servers  and  readied  for  a ‘real  world’  test.  Eight  students  who  match  the  target  audience 
are  recruited  and  paid  to  work  on  the  course  over  a realistic  timeframe  in  their  own  environment.  The  amount  of 
time  varies  with  the  length  of  time  students  would  be  expected  to  spend  on  a course.  A 30-hour  course  (roughly  one 
university  credit  hour)  that  should  take  about  six  weeks  to  complete  would  be  tested  over  a three-week  period.  This 
timeframe  is  still  slightly  compressed  to  meet  development  timelines  but  is  much  more  realistic  than  in  earlier 
phases  of  testing. 

A User  Experience  analyst  gathers  data  from  telephone  interviews,  online  survey  forms,  instructor  feedback 
and  analysis  of  online  student  interaction.  The  focus  is  on  understanding  what  potential  issues  exist  for  instructors 
and  students  as  the  course  is  released  to  paying  students.  If  any  development  issues  are  discovered  they  are  also 
corrected  during  this  phase.  Once  this  phase  of  testing  is  complete,  the  course  is  ready  for  release. 


Lessons  Learned 

Using  the  process  outlined  above  on  nearly  30  courses  has  yielded  some  interesting  lessons  on  taking  a 
human-centered  approach  to  online  course  design.  Perhaps  the  single  most  important  lesson  was  that  the  process 
itself  involved  iterative  design.  The  initial  implementation  of  the  human-centered  process  was  too  slow  and  too 
expensive  to  be  fiscally  viable.  The  process  outlined  in  this  paper  is  the  result  of  much  trial  and  error  and  additional 
changes  will  certainly  be  made  in  the  future. 

Benefits 


One  lesson  learned  was  that  using  a human-centered  process  is  key  in  building  high-quality  online  courses. 
Subsequent  commercial  trials  and  release  of  a number  of  courses  showed  that  customers  were  very  pleased  with  the 
quality  of  the  courses.  Most  major  technical  problems  and  flaws  were  discovered  prior  to  release,  thus  saving 
money  by  limiting  the  need  for  technical  support. 

An  early  model  of  the  process  focused  on  student  testing  near  the  end  of  the  development  cycle.  At  that 
point,  however,  a great  deal  of  time  and  effort  had  been  put  into  programming,  graphic  design,  and  course  structure. 
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Major  problems  were  often  discovered  near  the  end  of  the  timeline,  which  jeopardized  due  dates  and  cost  more 
money  to  fix.  By  making  students  part  of  the  design  team  (participatory  design  phase),  most  major  problems  are 
caught  and  corrected  before  too  much  effort  is  expended.  Interestingly,  there  are  generally  only  two  or  three 
students  in  this  phase,  which  indicates  that  it  only  takes  a few  users  to  uncover  major  problems. 

Instructional  designers  and  some  members  of  the  development  team  were  initially  skeptical  of  whether  user 
testing  could  improve  the  product.  One  of  the  unintended  benefits  was  that  by  seeing  their  designs  being  used  by 
students,  developers  were  able  to  improve  their  own  understanding  of  how  design  could  be  shaped  to  improve  online 
learning. 

Challenges 

Creating  a human-centered  design  process  involves  many  challenges.  The  goal  is  to  ensure  a quality  user 
experience  with  all  aspects  of  an  online  course.  Initially,  the  User  Experience  group  in  the  Bloomington  office  was 
staffed  with  usability  and  information  science  professionals.  It  became  apparent  that  guaranteeing  a quality 
experience  for  online  learning  would  require  more  than  ‘traditional’  interface  design  testing.  Adding  UE  team 
members  with  instructional  design  experience  and  visual  design  knowledge  broadened  the  group’s  understanding  of 
how  students  were  interacting  and  whether  they  were  learning.  The  first  lesson  learned  was  that  having  a 
multidisciplinary  User  Experience  group  is  essential  for  evaluating  online  learning  (Norman,  2000). 

As  many  organizations  are  discovering,  creating  quality  e-leaming  can  be  an  expensive  proposition. 

Finding  students  in  the  target  audience  for  business  courses  (mostly  business  professionals)  required  that  they  be 
well  paid  for  their  time.  For  lengthy  courses,  the  cost  for  testing,  including  facilities,  UE  salaries,  and  pay  for 
students  can  run  as  high  as  $35,000.  This  cost  is  minimal  for  an  organization  looking  to  roll  out  a course  to 
thousands  of  employees  but  most  universities  will  not  find  an  acceptable  return  on  investment  for  this  level  of 
testing.  In  fact,  the  elimination  of  potential  problems  is  an  investment  that  can  pay  big  dividends  when  the 
economies  of  scale  a large  enough. 

Another  issue  with  this  model  is  that  students  are  paid  for  their  time.  Paid  students  will  obviously  have  a 
very  different  motivation  for  completing  a course  than  a paying  student.  This  is  a difficult  problem  to  overcome 
when  looking  for  people  with  specific  skills  and  backgrounds.  One  potential  solution  to  this  problem  is  to  offer 
businesses  ‘free’  training  if  employees  are  willing  to  provide  feedback  to  the  design  team.  This  approach  is  helpful 
but  can  only  be  used  in  the  latter  testing  phases  when  the  course  is  nearly  complete. 

Conclusion 

While  the  process  described  in  this  paper  is  extremely  expensive,  there  are  low  cost  alternatives  to  each 
phase  that  can  improve  the  quality  of  online  courses.  For  example,  in  a university  setting,  design  consulting  can  be 
accomplished  by  having  a colleague  look  at  paper  prototypes  of  materials.  Even  if  they  do  not  have  time  to  work 
through  an  entire  course,  students  who  have  taken  a face-to-face  version  of  a course  make  great  participatory  design 
subjects.  Simply  asking  a student  walkthrough  how  they  would  get  through  an  assignment  will  reveal  many  flaws  in 
the  initial  design. 

Integration  testing  is  the  most  problematic  phase  of  the  process  to  conduct  on  a budget.  It  is  unlikely  that 
most  universities  can  afford  to  hire  students  to  take  a semester-long  course  before  it  is  taught.  One  alternative, 
however,  is  to  continue  evaluating  the  course  while  it  is  being  taught.  This  is  done  implicitly  in  face-to-face  courses 
and  must  be  explicit  in  distance  courses.  Sending  a mid-semester  survey  to  ask  students  how  a class  is  going  can 
provide  data  to  make  corrections  and  improve  students’  perception  of  the  course  (Wang -Chavez  & Branon  2000). 

Additionally,  as  a designer  builds  more  courses  within  a particular  domain,  the  need  for  testing  should 
decrease.  Option  Six  originally  had  a much  more  extensive  testing  process  but  as  the  development  teams  gained 
experience,  less  testing  was  needed  to  achieve  the  same  result.  Regardless  of  the  level  of  project,  getting  input  from 
students  is  an  essential  component  in  quality  online  course  development. 

Building  a human-centered  process  for  online  course  development  has  provided  a number  of  opportunities 
and  challenges.  After  observing  students  taking  a number  of  web-based  courses  the  most  important  conclusion  is 
that  no  designer  can  anticipate  every  issue.  By  putting  students  at  the  center  of  the  design  process,  a team  can 
proceed  with  confidence  and  prevent  costly  issues  with  the  final  product. 
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Abstract 

The  usability  test  team  attempted  to  identify  design  problems  that  limit  the  ability  of  instructors  at  the 
Indiana  University  to  use  data  entry  forms  on  the  School  of  Education  Web  site.  The  forms  permit 
instructors  to  publish  information  about  themselves  and  about  courses  they  teach  on  the  School  of 
Education  web  site.  We  asked  faculty  and  graduate  student  instructors  to  perform  typical  tasks  with  the 
web  forms  under  observation.  We  requested  that  the  participants  describe  what  they  were  thinking  and 
feeling  as  they  attempted  to  perform  the  task.  From  these  observations,  we  identified  several  key  design 
problems  that  prevented  or  frustrated  the  participants.  We  also  recommended  solutions  to  eliminate  these 
problems. 

Introduction 

Over  the  past  couple  of  years  the  use  of  the  World-Wide  Web  (WWW)  has  increased  dramatically.  People 
use  the  WWW  to  gather  information  that  informs  their  day-to-day  decision  making  processes.  Organizations  of 
every  kind  are  also  making  information  readily  available  on  the  Web.  The  School  of  Education  at  Indiana 
University  has  recently  decided  to  make  information  about  course  instructors,  and  information  about  classes  taught 
in  the  School  of  Education  available  on  their  public  web  server  (http://education.indiana.edu).  Instnictors  can  add 
information  to  their  listing  under  the  faculty/staff  directory.  Instructors  can  also  add  information  about  classes  they 
teach  to  the  School  of  Education  course  catalog.  The  ability  of  users  to  effectively  perform  tasks  with  web  sites  and 
data  entry  interfaces  is  a major  factor  in  the  total  cost  of  a computer  information  system.  Poor  design  can  add  to  the 
cost  of  an  information  system  by  reducing  efficiency,  and  adding  to  instructional  and  performance  support  costs 
(Landauer,  1 995).  Design  problems  that  hinder  the  ability  of  instructors  to  use  these  forms  may  also  hinder  the 
adoption  of  these  forms  by  the  School  of  Education  community,  and  also  add  to  the  hidden  costs  of  adding  these 
forms. 

People  use  computers  in  order  to  achieve  goals  within  a situated  context  (Suchman,  1987).  Usability  testing 
involves  observing  people  attempting  to  achieve  specific  goals  within  a specific  context.  Frick  & Boling  (1999) 
claim  that  three  factors  are  considered  important  in  the  design  of  a usability  test  for  a web  site: 

1.  The  participants  must  match  the  intended  audience  of  the  web  site. 

2.  The  participants  must  be  given  authentic  tasks  that  they  may  be  required  to  perform  as  part  of  their 
use  of  the  web  site. 

3.  The  context  in  which  the  test  is  performed  should  match  the  actual  work  context  in  which  the 
users  would  perform  authentic  tasks. 

Methodology 

In  this  research,  we  focused  our  data  gathering  efforts  on  the  target  population  of  faculty  and  staff  of  the 
School  of  Education.  This  group  also  includes  graduate  students  working  as  Assistant  Instructors  in  the  school.  The 
following  sections  will  explain  the  sample  population,  the  research  questions,  the  process  of  collecting  data,  and  the 
process  of  debriefing  and  data  analysis. 

Population  Sample 

We  used  a convenience  sample  in  order  to  try  and  get  a number  of  participants  from  our  target  population 
with  medium  to  low  HTML  and  web  skills.  We  felt  that  with  a small  population  sample  of  8-10  participants,  and 
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participants  with  less  web  experience  would  accentuate  problems  in  the  web  forms  interface  we  were  testing.  In 
addition  to  seeking  participants  with  low  web  skills,  we  also  sought  to  get  participants  from  many  different 
departments.  We  also  tried,  without  success,  to  get  an  even  gender  representation  in  our  population  sample.  Table  1 
shows  information  about  the  nine  participants  in  our  usability  tests. 


Table  1.  Participants  in  the  Usability  Test 


Participant 

Position 

Skill  Level 

Department 

Gender 

Participant  1 

Research  Faculty 

Low 

Dean’s  Office 

Female 

Participant  2 

Faculty 

Low 

Early  Childhood 

Male 

Participant  3 

AI 

Medium-low 

Language  Education 

Male 

Participant  4 

Faculty 

Low 

Instructional  Systems 
Technology 

Male 

Participant  5 

Faculty 

Low 

Educational  Leadership  and 
Policy  Studies 

Male 

Participant  6 

AI 

Medium-low 

Math  Education 

Male 

Participant  7 

Faculty 

Low 

Special  Education 

Male 

Participant  8 

Faculty 

Low 

Educational  Leadership  and 
Policy  Studies 

Male 

Participant  9 

Faculty 

Low 

Secondary  Education 

Male 

Research  Questions 

There  are  two  main  questions  that  we  investigated  as  part  of  this  project: 

1.  What  are  the  design  problems  that  prevent  or  discourage  faculty  and  staff  from  using  the  web 
forms  to  add/update  their  online  profile  information? 

2.  What  are  the  design  problems  that  prevent  or  discourage  faculty  and  staff  from  using  the  web 
forms  to  add/update  a syllabus  link  to  a course  in  the  School  of  Education  course  catalog? 

These  are  both  important  questions  to  address  because  if  the  faculty  and  staff  cannot  easily  update  this 
information,  then  there  is  a high  likelihood  that  they  will  not  take  the  time  to  add/update  their  personal  information 
and  course  information.  This  could  potentially  result  in  frustration  by  users  seeking  information  using  the  school’s 
web  pages. 

Data  Collection 

We  observed  the  participants  performing  the  following  two  tasks: 

1.  Add/update  their  personal  profiles  to  the  School  of  Education  faculty  and  staff  directories  on  the 
Web. 

2.  Add  a syllabus  link  to  a course  in  the  School  of  Education  course  catalog  on  the  Web. 

We  tried  to  make  the  tasks  as  authentic  as  possible  by 

• Having  the  participants  perform  the  tasks  in  the  environments  where  they  would  normally 
perform  the  tasks  (e.g.,  their  offices  or  public  computing  labs). 

• We  contacted  the  participants  ahead  of  time  and  asked  them  to  have  a copy  of  their 
syllabus  (or  know  the  link  if  already  online)  ready  to  use  in  the  testing.  This  allowed  the 
ones  who  came  prepared  with  their  own  syllabi,  to  post  their  syllabi  to  the  web  for  an 
actual  course.  (We  also  had  a copy  of  a syllabus  on  disk  in  MS  Word  format  that  a 
couple  of  individuals  who  had  forgotten  used.  In  these  cases,  they  posted  to  a test  course 
R999  that  was  set  up  by  the  Web  director.) 

We  also  swapped  the  order  that  different  participants  were  asked  to  do  the  two  tasks  in  order  to  try  and 
eliminate  overlooking  problems  that  might  occur  by  doing  the  tasks  in  different  orders. 

We  asked  the  participants  to  “think  aloud”  while  they  were  performing  the  tasks.  Meanwhile,  the  observers 
took  notes  of  difficult  ies  and  problems  they  were  having.  We  also  timed  each  of  the  tasks. 

Data  Analysis 
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After  all  of  the  usability  testing  observation  sessions  had  been  completed,  the  four  members  of  our  project 
team  met  to  debrief  and  analyze  our  findings.  We  grouped  similar  problems  together  and  recorded  a short 
description  of  the  problem  along  with  the  number  of  participants  that  reported  the  problem.  We  then  ranked  the 
problems  according  to  the  following  criteria: 

1.  Most  critical:  Problem  prevented  the  subject  from  accortplishing  the  desired  task  or  the  subject 
reported  high  levels  of  frustration  in  regards  to  this  problem. 

2.  Critical:  Problem  resulted  in  confusion  or  misdirection  that  did  not  prevent  the  subject  from 
accomplishing  the  desired  task. 

3.  Not-critical:  Problem  was  reported  as  confusing  or  undesirable  but  there  is  no  urgent  need  to  fix 
this  problem  to  insure  functionality. 

4.  Low-priority:  Problem  should  be  examined  on  a case-by-case  basis. 

We  then  linked  the  problems  to  specific  stages  in  the  process  at  which  the  problem  occurred.  We  also  provided 
recommendations  for  fixing  the  problem,  as  well  as  recommendations  for  re-designing  tbe  site  as  a whole. 

Findings  and  Recommendations 

Task  1:  Add/Update  Profile 

In  our  usability  test  we  asked  the  participants  to  add  or  update  their  personal  profile.  The  personal  profile 
is  published  with  the  staff  and  faculty  directory,  and  is  linked  to  entries  in  the  course  catalog.  Figure  1 shows  the 
task  of  adding/updating  a profile  broken  down  into  six  sequential  steps.  The  web  forms  interface  consists  of  three 
separate  pages.  The  first  page  (Demographics)  asks  for  the  instructor’s  name  and  email  address.  The  second  page 
(Contact  Info)  asks  for  office  location  and  telephone  number.  The  third  page  (Personal  Info)  asks  for  a large  amount 
of  information  including  job  title,  department,  home  page  and  optional  fields  for  more  information  and  links  to  other 
web  sites. 

Most  participants  experienced  problems  on  the  Personal  Info  page.  Participants  reported  frustration  with 
the  large  size  of  the  page  and  the  variety  of  information  requested.  Some  participants  also  reported  problems 
interpreting  instructions  and  anticipating  how  the  information  they  entered  would  be  displayed. 


Figure  2.  The  Six  Steps  Required  to  Complete  the  Task  of  Adding/Updating  a Profile 


Step  1:  Getting  to  the  Profiles  Page 

This  section  of  the  report  deals  with  challenges  that  the  users  had  in  finding  the  web  forms  that  they  could 
use  to  add  or  update  their  personal  profiles.  Table  2 outlines  our  findings  and  recommendations  for  improvement  of 
this  step  in  the  process. 


Table  2.  Findings  and  Recommendations  for  Getting  to  Profile  Web  Forms 


Findings  (Observations) 

Rating 

Recommendation 

Three  participants  experienced  some  difficulties 
in  finding  the  first  profile  web  form  page.  These 
individuals  followed  the  path  that  took  them  to 
the  Faculty  and  Staff  Directory.  They  would  go 
to  their  existing  profile  and  from  that  page  (the 
old  profile  formats)  there  was  not  a link  to 
update  their  individual  profile.  Although  there 
was  a link  to  update  profiles  from  the  Faculty 
and  Staff  Directory  page  it  was  not  easy  for 
these  participants  to  locate. 

1 

The  impact  of  this  problem  may  be  reduced  by  making 
the  link  on  the  Faculty  and  Staff  Directory  more 
obvious.  Currently  the  only  word  that  is  linked  is  the 
word  "here"  that  is  embedded  in  the  middle  of  a couple 
of  lines  of  text.  The  link  text  could  be  changed  to 
something  like  "Add/Update  Your  Profile"  and  placed 
on  its  own  line. 

Three  participants  expressed  uncertainty  about  . 

3 

Try  re-writing  documentation  to  eliminate  ambiguity. 
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which  password  they  should  use  in  order  to 
update  their  profiles.  Only  one  participant 
actually  typed  in  the  wrong  password  and  had  to 
go  back  and  re-enter  another  password. 


Unfortunately  this  problem  needs  to  be  addressed  by 
University  Information  Technology  Services  and  not 
the  designers  of  the  web  forms. 


Step  2:  Adding/Updating  Demographic  Information 

This  section  of  the  report  deals  with  challenges  that  the  users  had  in  adding  or  changing  their  name  and 
email  address.  Table  3 outlines  our  findings  and  recommendations  for  improvement  of  this  step  in  the  process. 


Table  3.  Findings  and  Recommendations  for  Adding  Demographic  Information 


Findings  (Observations) 

Rating 

Recommendation 

Two  participants  discovered  that  the  middle 
name  did  not  appear  in  the  final  profile.  One  of 
the  individuals  uses  his  middle  name  as  his 
preferred  name,  and  ended  up  getting  around  the 
system  by  placing  his  middle  name  in  the  first 
name  slot. 

2 

Don’t  ask  for  a name  that  will  not  be  included  on  the 
profile  page.  Both  of  the  following  solutions  might  be 
acceptable: 

• Include  all  three  names  in  the  profile  page. 

• Have  a preferred  name  slot  and  a last  name 
slot. 

• Have  one  text  blank  labeled  "Full  Name  to 
appear  in  profile" 

Two  of  the  participants  were  confused  by  the 
wording  on  the  submit  button  on  this  page 
which  read,  "Save  the  data  in  this  form  and  add 
your  office  location  information."  They  didn't 
realize  that  the  "and  add  your  office  location 
information"  part  of  the  text  was  referring  to 
what  they  would  do  on  the  next  page. 

2 

Our  recommendation  for  this  option  is  to  add  a wizard 
style  navigation  header  and  footer  to  each  of  the  profile 
pages.  (This  recommendation  will  also  address  several 
other  findings  from  the  usability  test.)  For  details  on 
this  recommendation,  see  the  General 
Recommendations  for  Add/Update  Profiles  Pages 
section  of  this  report. 

If  the  wizard  style  navigation  recommendation  is  not 
taken,  the  wording  of  the  button  could  be  changed  to 
something  like,  "Save  the  demographic  info  and  go 
onto  the  next  step." 

Step  3:  Adding/Updating  Contact  Information 

This  section  of  the  report  deals  with  challenges  that  the  users  had  in  adding  office  location  and  contact 
information  to  their  personal  profile.  Table  4 outlines  our  findings  and  recommendations  for  improvement  of  this 
step  in  the  process. 


Table  4.  Findings  and  Recommendations  for  Adding  Contact  Information 


Findings  (Observations) 

Rating 

Recommendation 

One  individual  didn't  realize  that  when  he 
updated  his  room  number  he  also  had  to  click  on 
the  corresponding  radio  button.  (Incidentally,  if 
the  radio  button  is  selected  without  entering  a 
room  number  in  the  text  box,  you  are  taken  to  a 
blank  screen  which  says,  “Your  form  was 
Processed”  and  nothing  else.) 

1 

The  form  could  be  simplified  by  having  a box  in  which 
the  room  number  could  be  placed  and  instructions  to 
leave  the  box  empty  if  they  don’t  have  a room. 

The  current  radio  buttons  could  be  kept  and  JavaScript 
could  be  used  to  select  the  appropriate  radio  button 
when  a number  is  entered  in  the  text  box. 

Step  4:  Adding/Updating  Personal  Information 

This  section  of  the  report  deals  with  challenges  that  the  users  had  adding  personal  information  about 
themselves  and  what  they  do  in  the  School  of  Education.  Table  5 outlines  our  findings  and  recommendations  for 
improvement  of  this  step  in  the  process. 
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Table  5.  Findings  and  Recommendations  for  Adding  Personal  Information 


Findings  (Observations) 

Rating 

Recommendation 

Five  of  the  participants  reported  skipping  the 
optional  “More  about  me”  fields  because  of 
perceived  complexity  of  the  instructions  for 
entering  HTML.  They  were  not  able  to  add 
information  to  their  profiles  that  they  wanted  to 
because  they  felt  that  it  would  be  too  complex 
for  them  to  do  it. 

1 

Simplify  the  task  by  using  formatted  plain  text  only. 
OR 

Give  radio  button  choice  for  each  text  box  indicating 
“Plain  Text  (default)”  or  “HTML”  like  in  AltaVista 
Forum  (AVF) 

Participants  didn’t  seem  to  understand  if  text 
boxes  would  be  WYSIWYG  or  HTML.  In  one 
case  a participant  entered  multiple  titles 
separated  by  a carriage  return  and  was  surprised 
to  see  them  squashed  together  on  the  final 
profile.  (There  are  inconsistencies  - for  example 
the  "What  You  Do"  section  converts  completely 
to  HTML  with  no  hard  carriage  returns  while 
the  "More  About  Me"  section  allows  links  to  be 
added  but  also  recognizes  carriage  returns.) 

1 

Simplify  the  task  by  using  formatted  plain  text  only. 
OR 

Give  radio  button  choice  for  each  text  box  indicating 
“Plain  Text  (default)”  or  “HTML”  like  in  AVF 
OR 

Use  the  server-side  CGI  script  to  check  text  block  for 
any  HTML  tags.  If  there  are  tags  format  as  HTML 
otherwise  format  as  plain  text. 

Three  participants  were  a little  confused  by  the 
text  box  labeled  “Name  of  your  office/unit.  At 
least  one  filled  in  the  name  of  his  office  even 
though  he  had  checked  the  box  for  the  office 
above.  The  indentation  of  the  fields  relative  to 
the  other  options  seemed  confusing  to  the 
participants.  People  tended  not  to  read  the  * 
instructions  carefully. 

2 

Align  all  choices  equally.  Have  a checkbox  directly 
below  the  others  followed  by  a blank  text  box  for  them 
to  enter  their  office/unit  if  it  is  different.  A * could  be 
placed  at  the  end  of  the  text  box  and  another  * could  be 
placed  below  with  instructions  about  how  to  enter  the 
URL  for  the  “other”  unit. 

Two  participants  expressed  uncertainty  about 
what  to  put  into  the  section  labeled  "What  Yo  u 
Do."  The  concern  was  whether  this  was  just  for 
faculty /staff  job  responsibilities  or  if  it  could 
contain  other  kinds  of  information  too. 

2 

This  problem  could  easily  be  solved  by  providing  more 
explicit  guidelines  about  what  should  go  in  this  area. 
Also,  being  able  to  see  an  example  of  a typical  profile 
might  allow  them  to  see  where  it  would  be  most 
appropriate  to  stick  different  pieces  of  information 
about  themselves. 

It  was  our  observation  that  participants  were 
overwhelmed  with  the  amount  of  information 
being  requested  on  this  page. 

2 

This  problem  could  be  solved  by  using  the  wizard  style 
navigation  (see  description  in  section  General 
Recommendations  for  Add/Update  Profile  Page). 

Using  this  system  this  step  could  easily  be  divided  into 
several  more  simple  steps. 

Step  5:  Previewing  and  Publishing  a Profile 

This  section  of  the  report  deals  with  challenges  that  the  users  had  in  previewing  and  publishing  their 
profile.  Table  6 outlines  our  findings  and  recommendations  for  improvement  of  this  step  in  the  process. 


Table  6.  Findings  and  Recommendations  for  Previewing  and  Publishing  a Profile 


Findings  (Observations) 

Rating 

Recommendation 

Three  of  the  participants  considered  clicking  the 
"Update  this  profile"  link  that  is  embedded  into 
the  profile  preview  instead  of  the  publish 
preview  button.  This  generally  happened 
because  they  were  scrolled  down  to  the  middle 
of  the  page  where  they  could  not  see  the  buttons 
and  the  link  text  sounded  like  it  would  do  what 
they  wanted  to  do. 

1 

To  solve  this  problem  we  suggest  that  changing  the 
wording  of  the  link  text  from  "Update  this  profile"  to 
something  like  "Start  the  update  profile  wizard"  or  "Go 
to  the  Update  Profile  Pages."  We  feel  that  a different 
wording  like  this  will  be  harder  to  confuse  with 
publishing  or  actually  saving  the  updates. 

Three  participants  in  the  usability  test  did  not 

1 

Doing  something  to  make  the  reload  reminder  obvious, 
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remember  to  click  the  reload  button  after  going 
back  to  make  changes  to  their  profiles,  even 
though  the  first  line  of  instructions  reminded 
them  too.  In  general,  they  read  the  instruction 
block  the  first  time  they  arrived  at  the  page  but 
did  not  re-read  it  on  subsequent  visits. 

even  without  reading  the  instruction  block  over,  could 
solve  this  problem.  A couple  of  possibilities  might  be 
changing  the  color  of  the  word  "reload"  and  making  it 
bold.  Or  the  word  reload  could  be  made  to  blink  on 
and  off. 

OR 

A server-side  solution  might  be  to  write  the  temporary 
profile  to  a file  with  a different  name  each  time  - 
perhaps  <timestamp>.html. 

When  the  user  publishes  the  profile  the  temporary  file 
is  written  and  deleted. 

OR 

The  tradeoff  might  make  using  a meta-tag  that 
automatically  forces  the  page  to  be  reloaded  every  time 
worth  it. 

Several  participants  commented  that  it  would 
have  been  helpful  to  be  able  to  see  examples  of 
profiles  or  even  preview  their  own  profiles 
during  the  process  instead  of  having  to  wait  until 
the  very  end  and  then  go  back  to  make  changes 
at  the  appropriate  step. 

2 

This  problem  could  be  taken  care  of  by  implementing 
the  solution  that  is  mentioned  under  wizard  style 
navigation  section  later  in  the  report.  Under  this 
system,  the  users  could  preview  their  profile  at  any  step 
through  the  process. 

One  individual  didn't  readily  see  the  information 
entered  in  the  "More  About  Me"  fields  on  the 
profile  preview.  He  finally  found  them  below 
the  map  and  said  that  he  felt  like  they  should 
have  belonged  above  the  map. 

3 

Move  the  link  to  the  “More  about  me”  entries  from  the 
middle  of  the  demographic  information  to  a more 
prominent  location. 

Step  6:  Post-publishing  steps 

This  section  of  the  report  deals  with  challenges  that  the  users  had  with  navigational  steps  required  after  the 
profile  is  published  on  the  web.  Table  7 outlines  our  findings  and  recommendations  for  improvement  of  this  step  in 
the  process. 


Table  7.  Findings  and  Recommendations  for  Post-Publishing  Steps 


Findings  (Observations) 

Rating 

Recommendation 

One  person  exited  the  web  browser  directly  after 
publishing  the  profile  because  he  was  following 
the  red  highlighted  text  at  the  bottom  of  the  page 
literally.  Then  he  started  his  web  browser  right 
back  up. 

4 

The  wording  of  the  message  could  be  changed  to  not 
imply  that  the  browser  should  be  closed  right  then,  but 
that  it  should  be  closed  before  they  leave  the  machine. 

One  person  indicated  that  he  felt  like  there  were 
too  many  link  options  on  the  last  page.  He  felt 
that  it  was  confusing. 

4 

We  are  not  sure  that  this  is  a problem  that  needs  fixing. 

General  Recommendations  for  Add/Update  Profile  Pages 

We  have  one  general  recommendation  for  the  profile  submission  forms.  This  is  a general  recommendation 
because  it  addresses  several  of  the  findings  observed  in  the  usability  testing  and  mentioned  earlier  in  the  document. 
Our  general  recommendation  is  to  change  the  navigation  system  to  a "wizard  style  navigation".  What  we  mean  by 
this  is  explained  in  the  following  section. 

Wizard  Style  Navigation 

A "wizard"  is  a tool  that  helps  to  step  a person  through  a linear  process  by  displaying  the  current  position  in 
the  process  as  well  as  providing  controls  to  move  forward  or  backwards  in  the  process.  We  think  that  including  a 
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wizard  navigation  header  and  footer  on  each  page  would  be  an  ideal  way  to  address  many  of  the  issues  mentioned 
earlier  in  the  findings.  The  wizard  navigation  would  graphically  outline  the  steps  and  indicate  where  the  person  is  in 
the  overall  process.  It  should  contain  buttons  to  allow  the  user  to  progress  forward  through  the  steps  as  well  as 
backwards  through  the  steps.  Ideally,  the  navigation  bar  would  also  have  a preview  button  and  a publish  button 
which  would  allow  the  user  to  preview  or  publish  the  current  state  of  the  profile  at  any  time  in  the  process. 

Task  2:  Add/Update  Syllabus  Link  to  Course  Catalog 

This  task  tests  web  forms  that  allow  instructors  to  publish  information  about  a course  they  teach  on  the 
School  of  Education  course  catalog.  The  web  forms  give  instructors  four  options  for  entering  a course  description  or 
syllabus: 

1.  Enter  a syllabus  as  HTML- formatted  text. 

2.  Enter  a syllabus  as  plain  text  with  carriage  returns. 

3.  Enter  a link  to  an  existing  syllabus  on  the  WWW. 

4.  Enter  a link  to  Oncourse,  a web-based  academic  conferencing  system. 

Seven  participants  tried  the  HTML  path.  One  participant  tried  the  Plain  Text  path  and  two  tried  the 
syllabus  link  path.  No  individuals  tried  the  Oncourse  path.  One  participant  tried  multiple  paths  in  order  to  achieve 
good  results. 

The  most  critical  problem  we  found  was  that  out  of  seven  participants  who  attempted  to  enter  HTML- 
formatted  text,  six  found  that  following  the  instructions  did  not  produce  the  desired  results.  The  instructions  on  the 
web  form  direct  the  participants  to  convert  their  syllabus  from  a word  processor  document  to  HTML  text  using 
Microsoft  Word  97,  then  paste  the  resulting  HTML-coded  text  into  a web  form  text  box.  In  most  cases  Word  97 
failed  to  produce  readable  HTMLr formatted  text. 


Step  1 Step  2 Step  3 Step  4 Step  5 Step  6 


Figure  3.  The  Six  Steps  for  Adding  an  Entry  to  the  Schedule  of  Classes 


Step  1:  Getting  To  the  Add  or  Update  a Syllabus  Page 

This  section  of  the  report  deals  with  challenges  the  users  had  finding  the  web  forms  that  permit  them  to  add 
a course  entry  to  School  of  Education  web  site.  Table  8 outlines  our  findings  and  recommendations  for 
improvement  of  this  step  in  the  process. 


Table  8.  Findings  and  Recommendations  for  Getting  to  the  Add  a Course  Web  Form 


Findings  (Observations) 

Rating 

Recommendation 

The  link  text  (on  page  updates.html)  indicates 
that  you  can  add  a syllabus  link  but  not  update  a 
syllabus  link. 

2 

Change  the  link  text  to  say  "Add/Update  ..." 

The  link  text  to  enter  the  forms  from  the  course 
catalog  is  somewhat  confusing.  It  states 
"Add/Modify  your  course  web  site." 

2 

The  text  should  be  changed  to  read  something  link 
"Add/Modify  syllabus  link  for  this  course"  or 
something  similar.  In  addition  the  text  at  the  top  of  the 
course  catalog  table  should  be  made  clearer  by 
changing  it  to  "Syllabus  Link(s)"  instead  of  just 
"Link(s)" 

Step  2:  Entering  a Course  Number  and  Choosing  a Format 

This  section  of  the  report  deals  with  challenges  the  users  had  entering  a course  number  and  choosing  a 
syllabus  format.  Table  9 outlines  our  findings  and  recommendations  for  improvement  of  this  step  in  the  process. 
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Table  9.  Findings  and  Recommendations  for  Entering  a Course  Number  and  Choosing  a Format 


Findings  (Observations) 

Rating 

Recommendation 

Four  participants  typed  a lower  case  letter  first 
for  the  course  number. 

2 

Use  server-side  CGI  to  set  the  case  of  the  letter  rather 
than  forcing  user  to  do  it. 

When  the  participants  got  the  error  message  for 
typing  in  a lower  case  letter  or  not  selecting  a 
radio  button  the  error  message  is  incorrect  and 
says  "Click  the  back  button  and  enter  the 
filename  in  CAPITAL  letters." 

2 

Check  all  error  messages  and  make  sure  that  they  are 
appropriate  for  the  error.  The  two  errors  mentioned 
should  have  different  error  messages  and  the  message 
for  the  first  error  should  read  "Course  Number"  instead 
of  "filename." 

Step  3m:  Adding  an  HTML-Formatted  Syllabus 

This  section  of  the  report  deals  with  challenges  the  users  had  adding  an  HTML-formatted  syllabus  to  the 
School  of  Education  web  site.  Table  10  outlines  our  findings  and  recommendations  for  improvement  of  this  step  in 
the  process. 


Table  10.  Findings  and  Recommendations  for  Adding  an  HTML  Formatted  Syllabus 


Findings  (Observations) 

Rating 

Recommendation 

Six  participants  followed  the  instructions  to 
convert  a word -processed  document  to  HTML 
and  didn't  get  results  that  they  were  satisfied 
with.  (Tables  and  tab-formatted  documents 
notably  had  problems  and  participants  didn't 
have  skills  to  make  changes.) 

1 

See  general  recommendations  below. 

Five  of  the  participants  complained  about  being 
forced  to  use  MSWord.  Their  word  processor  of 
choice  was  WordPerfect. 

1 

Same  as  above. 

The  forms  don't  allow  the  user  to  edit  a syllabus 
that  has  been  added.  To  make  one  change  to  the 
syllabus  they  must  edit  their  Word  document 
and  go  through  the  whole  conversion  process 
again.  (This  can  be  painful  for  just  a spelling 
change  or  something.) 

1 

Allowing  to  upload  syllabi  in  their  original  format 
would  solve  this  problem. 

OR 

Load  the  syllabus  HTML  text  into  the  text  box  when  a 
syllabus  already  exists. 

Five  individuals  commented  on  the  reference  to 
the  "Title  Box  beloW  when  the  title  box  was 
really  above 

2 

Change  the  text  to  read  "above"  instead  of  "below." 

Following  the  HTML  instructions  was  difficult 
because  the  users  had  to  switch  back  and  forth 
between  the  word  processing  window  and  the 
browser  window.  Each  time  the  users  had  to  re- 
find  where  they  were  in  the  process.  (One 
individual  skipped  the  step  of  changing  the  word 
processor  to  "view  html  source"  and  therefore 
ended  up  in  the  end  with  a bunch  of  words  all 
scrunched  together  - spaces  removed.) 

2 

Suggest  in  the  instructions  that  the  user  print  out  the 
instructions  to  reduce  the  chances  of  skipping  steps. 

Step  3.b:  Adding  a Plain  Text  Formatted  Syllabus 

This  section  of  the  report  deals  with  challenges  the  users  adding  a plain  text  syllabus  to  the  School  of 
Education  web  site.  Table  1 1 outlines  our  findings  and  recommendations  for  improvement  of  this  step  in  the 
process. 


Table  II.  Findings  and  Recommendations  for  Adding  a Plain  Text  Formatted  Syllabus 


Findings  (Observations) 

Rating 

Recommendation 

One  participant  pasted  plain  text  copied  from  a 

2 

Strengthen  wording  in  instructions  to  indicate  that  most 
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word -processing  document  into  textbox  but  was 
unsatisfied  with  results. 


text  formatting  will  be  lost. 


Steps  3.c:  Adding  a Link  to  an  Existing  Syllabus  Elsewhere  on  the  WWW 

This  section  of  the  report  deals  with  challenges  the  users  had  adding  a hypertext  link  from  the  School  of 
Education  course  catalog  to  an  existing  syllabus  elsewhere  on  the  WWW.  Table  12  outlines  our  findings  and 
recommendations  for  improvement  of  this  step  in  the  process. 


Table  12.  Findings  and  Recommendations  for  Adding  a Link  to  an  Existing  Syllabus  Elsewhere  on  the  WWW 


Findings  (Observations) 

Rating 

Recommendation 

Participant  was  unable  to  find  syllabus 
published  on  School  of  Education  web  server 
prior  to  installation  of  new  web  forms. 

3 

Provide  suggestions  for  finding  URL  of  published 
syllabus.  Also  consider  providing  links  to  Indiana 
University  web  search  tool. 

Steps  3.d:  Adding  a Link  to  Oncourse 

This  section  of  the  report  deals  with  challenges  the  users  had  adding  a hypertext  link  from  the  School  of 
Education  course  catalog  to  Oncourse,  an  online  academic  information  system  developed  by  Indiana  University 
Information  Technology  Services.  No  participants  selected  this  option. 

Step  4:  Adding  Additional  Notes  to  the  Course  Entry 

This  section  of  the  report  deals  with  challenges  the  users  had  adding  additional  notes  to  the  course  entry. 
Table  13  outlines  our  findings  and  recommendations  for  improvement  of  this  step  in  the  process. 


Table  13.  Findings  and  Recommendations  for  Adding  Additional  Notes  to  a Course  Entry 


Findings  (Observations) 

Rating 

Recommendation 

Four  of  the  users  did  not  even  enter  anything  in 
the  notes  block  because  they  were  just  not  sure 
what  kind  of  information  should  go  there.  One 
individual  put  several  paragraphs  about  himself 
in  the  notes  area. 

2 

The  instructions  should  be  changed  to  indicate  what 
kind  of  information  should  be  included  in  the  notes. 
AND 

An  example  or  preview  might  help. 

Step  5:  Viewing  Summary  Information 

This  section  of  the  report  deals  with  challenges  the  users  had  interpreting  summary  information  provided 
about  a course  entry.  No  problems  were  observed  with  this  step. 

Step  6:  Post-Publishing  Steps 

This  section  of  the  report  deals  with  challenges  the  users  had  viewing  updated  course  entries  after  they 
were  published.  Table  14  outlines  our  findings  and  recommendations  for  improvement  of  this  step  in  the  process. 


Table  14.  Findings  and  Recommendations  for  Post  Publishing  Steps 


Findings  (Observations) 

Rating 

Recommendation 

Participant  went  back  through  web  forms  to 
make  changes.  Participant  then  tried  to  view 
changes  but  failed  to  reload  web  page. 

2 

Add  tag  to  web  pages  to  force  the  course  schedule  and 
syllabus  to  reload  if  possible. 

General  Recommendations  for  Adding  or  Updating  a Course  Entry  to  the  School  of  Education  Course 
Catalog 


Converting  a word -processing  document  to  HTML  text  caused  serious  problems  for  all  but  one  of  the 
individuals  that  attempted  the  process.  This  procedure  resulted  in  four  different  types  of  problems  affecting  a large 
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percentage  of  our  participants.  We  do  not  feel  we  have  enough  information  to  properly  recommend  a solution  to 
this  problem.  Possible  solutions  may  include  providing  the  ability  to  upload  word  processing  documents, 
conversion  of  word  processing  documents  to  Adobe  PDF,  or  providing  technical  assistance. 

Participants  also  reported  problems  with  the  instructions  provided  on  the  web  forms  and  with  labels  of  text 
fields.  We  recommend  that  the  instructions  and  descriptive  text  be  edited  to  correct  errors  and  reduce  ambiguity. 

Limitations 

Three  of  the  main  limitations  to  our  findings  involve  the  selection  of  our  participants: 

1.  Only  one  of  our  participants  was  female. 

2.  Only  two  of  our  participants  were  Associate  Instructors. 

3.  We  did  not  recruit  participants  that  had  high  levels  of  computer  experience. 

We  attempted  to  recruit  more  women  and  associate  instructors.  However  we  were  unable  to  arrange  test 
sessions  with  most  of  the  women  and  associate  instructors  we  contacted.  It  is  possible  that  having  greater 
representation  in  both  of  these  groups  may  have  added  insights  to  the  information  we  gathered.  We  also  did  not 
recruit  participants  with  high  levels  of  computer  experience.  We  deliberately  selected  a convenience  sample  of 
individuals  with  low  to  moderate  computer  experience  in  order  to  maximize  observation  of  design  problems 
experienced  by  novice  users.  As  a result,  we  don’t  make  any  generalizations  regarding  the  ability  of  instructors  in 
the  School  of  Education  to  use  these  forms. 

Additionally,  none  of  the  people  that  we  did  the  usability  testing  with  had  done  either  of  the  tasks  before. 
Therefore,  our  tests  did  a good  job  of  testing  what  a novice  to  the  system  would  do  in  adding  information  to  the 
system,  but  didn't  explore  what  challenges  would  be  faced  by  someone  who  had  already  entered  data  andjust 
wanted  to  make  a small  change  (e.g.,  a new  phone  number,  link,  or  spelling  correction/date  change  to  syllabus).  We 
have  a hunch  that  there  may  be  additional  problems  that  surface  when  this  aspect  of  the  system  is  tested. 

Some  other  problems  came  out  during  testing.  None  of  the  participants  tried  to  add  an  Oncourse  syllabus  to 
the  course  catalog.  Although  the  process  is  very  similar  to  adding  a syllabus  link,  some  testing  of  this  should  be 
done  to  ensure  usability.  Several  participants  also  reported  that  they  would  have  abandoned  some  steps  in  the 
process  if  they  had  not  been  participating  in  a usability  study.  Participants  reported  that  being  observed  encouraged 
them  to  tackle  problems  that  would  have  led  them  to  give  up. 

Conclusion 

We  found  mult^le  problems  with  the  design  of  the  web  forms  that  hindered  or  prevented  our  participants 
from  successfully  completing  these  tasks.  Re-designing  the  web  forms  in  order  to  reduce  or  eliminate  these 
problems  is  in  our  opinion  essential  to  ensuring  the  adoption  of  these  web  forms  within  the  School  of  Education. 

The  designers  of  this  web  site  should  address  the  following  problems  in  future  versions  of  the  web  site: 

1.  The  recommended  procedure  for  adding  an  HTML- formatted  syllabus  to  the  School  of  Education 
web  site  fails  to  produce  satisfactory  results.  The  designers  should  find  tools  that  produce  reliable 
results,  or  eliminate  this  option.  The  School  of  Education  may  need  to  make  organizational 
changes  such  as  employing  specialists  to  assist  instructors  in  preparing  electronic  versions  if  this 
feature  is  desired. 

2.  The  Personal  Information  page  causes  problems  for  uses  due  to  its  length  and  complexity.  At  a 
minimum  the  form  should  be  separated  into  two  pages,  one  for  required  information  and  one  for 
optional  information.  In  addition  the  designers  should  make  sure  that  all  multi-line  text  fields 
handle  carriage  returns  properly. 

3.  The  designers  should  add  navigational  elements  to  the  web  forms  that  provide  the  user  with 
indications  of  what  steps  they  have  completed  and  what  steps  they  still  need  to  complete. 
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Abstract 

This  paper  reports  research  in  progress.  It  presents  information  in  the  literature  on  converting  traditional 
courses  into  Web-Based  format.  The  paper  is  divided  into  four  parts.  The  first  part  makes  an  introduction 
to  Distance  Education  (DE)  and  Web-Based  Instruction  (WBI)  which  combines  features  from  both  face-to- 
face  classroom  instruction  and  DE.  The  second  part  discusses  current  Instructional  Design  (ID)  models  for 
WBI.  In  the  third  part,  findings  from  the  literature  on  converting  existing  courses  into  Web-based  format 
are  discussed.  This  part  includes  information  in  the  following  categories:  How  to  start  the  conversion  to 
the  WBI,  student  and  instructor  support,  design  issues,  interaction  among  people,  and  assessment.  In  the 
last  part,  a model  proposing  an  orderly  process  for  converting  courses  into  Web-Based  format  will  be 
explained. 

Introduction 

While  organizations  and  schools  (K-12,  colleges,  and  universities)  have  been  using  the  Internet  to  distribute 
information  for  a long  time,  the  implications  for  the  instructional  design  and  delivery  of  instruction  over  the  Internet 
have  not  been  sufficiently  explored.  Many  instructors  are  given  the  task  of  converting  existing  courses  into  Web- 
based  format.  However,  since  very  little  has  been  written  on  this  subject,  instructors  are  left  to  learn  the  conversion 
process  by  trial  and  error  (Lightfoot,  2000).  As  a result,  instructors  might  create  WBI  courses  equal  to  electronic 
page  turning  by  following  the  traditional  ID  models  and  utilizing  old  methods  of  face-to-face  delivery.  This 
literature  review  will  unearth  the  guidelines  for  converting  existing  courses  into  Web-based  format.  These  guidelines 
will  help  the  practitioners  by  offering  a systematic  way  for  conversion  thus  eliminating  trial  and  error. 

Definitions 

In  the  literature,  I came  up  with  such  terms  as  Internet-based  training,  Internet-based  instruction,  computer- 
based  distance  education,  Web-based  learning,  Web-based  distance  education,  and  Web-based  instruction.  For  the 
sake  of  simplicity,  and  to  prevent  confusion,  I will  use  Web-based  instruction  (WBI)  through  the  text. 

Delivery  of  Education 

Delivery  of  the  education  has  changed  throughout  the  history.  Today,  there  are  two  main  delivery  methods: 
Face-to-face  education  and  distance  education  (Shave,  1 998). 

In  the  industrial  age,  formal  education  system  followed  the  factory  metaphor.  A campus  (factory)  was  built 
and  the  students  and  teachers  (employees  and  management)  arrived  at  it  and  worked  there  (Shave,  1998).  In  this 
system,  main  delivery  method  for  instruction  has  been  face-to-face. 

In  the  traditional  face-to-face  method  the  instructor  can  allocate  time  for  the  learners,  learners  can  interact 
with  other  learners  and  with  the  instructor,  and  immediate  feedback  is  possible.  However,  learning  should  occur  at  a 
particular  time  and  place.  Besides,  interaction  between  the  people  in  the  learning  process  is  difficult  outside  the 
classroom. 

Distance  Education  (DE) 

DE  as  a method  of  delivery  of  instruction  is  a promising  technology,  which  can  eliminate  these  problems  of 
face-to-face  method.  “The  term  distance  learning  or  distance  education  refers  to  the  teaching  learning  arrangement 
in  which  the  teacher  and  the  learner  are  separated  by  geography  and  time.”  (Williams  et.  al.,  1999). 

Keegan  (1980)  characterizes  DE  with  six  key  elements,  one  of  which  is  the  use  of  media  to  link  teacher  and 
learner.  Different  communication  and  media  technologies  can  provide  the  link  between  teacher  and  learner. 
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Examples  include  television,  computer  transmission,  audio  or  computer  conferencing,  videocassettes  or  discs,  and 
correspondence  (Heinich  et.  al.,  1999). 

In  order  to  describe  the  media  technologies  used  in  DE,  “The  4-Square  Map  of  Groupware  Options”  model 
developed  by  Johansen  et.  al.  (1 991 ) provides  a convenient  way.  This  model  is  developed  on  two  basic 
configurations  that  teams  must  overcome  as  they  work:  time  and  place.  Based  on  these  configurations,  the  4-Square 
model  classifies  four  types  of  technologies  that  support  the  group  process.  They  are: 

• Same  time/different  place 

• Same  place/different  time 

• Same  time/same  place 

• Different  time/different  place 

Web-Based  Instruction  (WBI) 

WBl  is  a type  of  technology  that  is  in  the  different  time/different  place  category.  The  most  valuable  asset 
WBI  brings  to  the  DE  field  is  that  it  allows  the  flexibility  for  learning  to  be  occurred  at  any  time  and  any  place 
(Tipton  et.  al.,  1998).  With  this  flexibility,  WBI  can  offer  academic  programs  to  a larger  potential  audience,  which 
Ehrmann  (1990)  refers  as  the  New  Majority.  These  people  do  not  have  time  to  a full-time  study  on  campus.  For  this 
reason,  traditional  face-to-face  format  cannot  serve  them.  WBI  may  provide  instruction  to  these  people  at  times  and 
places  more  appropriate  to  their  lifestyles. 

Many  colleges  and  universities  today  are  restructuring  their  existing  courses  to  be  offered  via  the  Internet  to 
target  the  New  Majority  (see,  for  example,  Bichelmeyer  et.  al.,  2000).  As  the  demand  grows,  the  number  of  WBI 
providers  will  increase.  Besides,  the  number  of  students  who  have  been  using  the  Internet  for  communication  and 
research  is  increasing  in  vast  amounts.  For  this  reason,  it  might  be  possible  that  non  New  Majority  students  also  may 
jump  to  the  WBI  bandwagon,  thus  making  the  market  even  bigger. 

WBI  not  only  provide  any  time  and  anywhere  delivery  of  education,  but  it  also  does  so  as  effective  as  face- 
to-face  education.  Several  researchers  conducted  experimental  studies  to  compare  the  student  performance  of  Web- 
based  versus  traditional  class  format  (see,  for  example,  Schutte,  2001 , and  Jones,  1999).  They  found  higher  scores 
for  WBI  or  no  significant  difference  between  the  two  conditions.  Besides  the  effectiveness,  some  course  evaluations 
showed  that  WBI  courses  were  appealing  to  students  and  students  were  positive  about  WBI  (see,  for  example, 
Friedrich,  1999,  and  Ingebritsen  et.  al.,  1998). 

There  are  some  other  advantages  of  WBI.  They  are  potential  to  reach  a global  audience  (Duffy,  1992), 
support  for  multiple  learning  styles  (Ingebritsen  et.  al.,  1997),  lower  development  and  operating  costs  (Kuchinke  et. 
al.,  2001),  and  being  a diverse  information  resource  (McManus,  2001). 

We  can  find  disadvantages  with  every  new  innovation  however  and  WBI  is  no  exception.  Some  of  the 
problems  listed  in  the  order  of  most  cited  are  limited  bandwidth  (McManus,  2001),  access  to  hardware  and  Internet 
(Shave,  1998),  social  isolation  (Lichty,  1997),  and  reliance  on  learner  (Ingebritsen  et.  al.,  1998).  Another  problem 
lies  in  the  conversion  of  delivery  of  instruction  from  face-to  face  to  online  method. 

Instructional  Design  Model  for  WBI 

The  purpose  of  an  Instructional  Design  (ID)  model  is  “to  convey  key  concepts  and  processes  to  be  included 
in  a particular  approach”  (Molenda  et.  al.,  1996).  Models  tell  us  the  critical  success  factors  to  instructional  design. 

To  be  more  specific,  an  ID  model  tells  us  what  to  do  and  when  to  do  it,  and  it  barely  tells  how  to  do  it.  Molenda  et. 
al.  (1996)  point  to  the  importance  of  an  ID  model  by  this  example:  “...experience  leads  the  expert  away  from  the 
cookbook  and  toward  improvisation.  But  for  the  apprentice  chef,  the  cookbook  is  the  vital  link  to  maintaining 
quality  and  consistency  from  day  to  day.”  The  ID  model  is  the  cookbook  that  will  address  these  concerns  for 
instructional  design. 

There  seems  to  be  a lack  of  an  ID  model  and  conversion  model  for  WBI.  Since  the  Internet  has  not  been 
used  very  long  as  a means  for  instruction,  currently  there  are  no  time  -proven  models  for  WBI  (Edwards,  1999). 

What  makes  this  worse  is  the  “misconception  that  the  digital  media  is  able  to  translate  printed  matter  directly  to  the 
screen”  (Lichty,  1997).  Following  this  misconception,  many  instructors  and  designers  have  created  ineffective  WBI 
courses  which  do  not  utilize  the  opportunities  Internet  has  offered. 

Computer  mediated  communication  supports  the  new  paradigm  of  knowledge-building  model  for  the 
delivery  of  learning.  Traditional  education  is  based  upon  a paradigm  generally  called  “knowledge  reproduction 
model.”  The  components  of  this  model  are  verbal  lecture,  printed  handouts,  drill  and  practice  sessions,  structured 
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classroom  activities,  and  office  hours  (Lightfoot,  2000).  In  this  model,  students  are  seen  as  passive  learners.  The 
purpose  of  teaching  is  to  transfer  static  body  of  knowledge  from  its  source  to  the  student. 

According  to  Imel  (1997)  the  most  important  distinguishing  characteristic  of  WBI  is  the  emphasis  on 
instruction  and  not  just  on  information  delivery.  When  information  delivery  is  the  basic  concern,  WBI  will  resemble 
computer-based  instruction,  which  has  been  criticized  as  being  restrained  by  behaviorist  learning  theory.  WBI 
should  be  designed  by  basing  it  upon  the  cognitive-based  theories  of  learning,  where  learners  purposefully  interact 
with  the  environment,  actively  participate,  thus  following  the  knowledge-building  paradigm. 

Current  ID  Models  for  WBI 

Long  ago,  after  the  old  king  died  and  the  prince  was  crowned,  the  crowd  used  to  chant,  'The  King  is  dead... 
long  live  the  King"  to  acknowledge  the  passing  of  the  old  king  and  the  acceptance  of  the  new  one.  For  WBI,  the  old 
ID  models  no  longer  work.  The  king  is  dead.  It  is  time  to  look  at  new  models  of  instructional  design  (Hisey,  2000). 

However,  when  we  take  a look  into  the  current  ID  models  for  WBI  identified  by  Edwards  (1999),  we 
witness  that  the  instructional  designers  lament  for  the  old  king.  The  ID  models  for  WBI  used  by  practitioners 
resemble  the  old  models.  Current  WBI  models  in  the  literature  are  Reeves  and  Reeves’  model,  Ritchie  and 
Hoffman’s  model,  and  Duchastel’s  model. 

Reeves  and  Reeves  WBI  model  includes  ten  components  for  learning  on  the  World  Wide  Web  (WWW) 
called  dimensions.  These  dimensions  are  (1)  pedagogical  philosophy,  (2)  learning  theory,  (3)  goal  orientation,  (4) 
task  orientation,  (5)  source  of  motivation,  (6)  teacher  role,  (7)  metacognitive  support,  (8)  collaborative  learning  (9) 
cultural  sensitivity  and  (10)  structural  flexibility  (Edwards,  1999). 

Ritchie  and  Hoffman’s  WBI  model  adapts  traditional  instructional  design  principles  into  WBI.  These  are: 
motivating  the  learner,  explaining  what  is  to  be  learned,  helping  the  learner  recall  previous  knowledge,  providing 
instructional  material,  providing  guidance  and  feedback,  testing  comprehension,  and  providing  enrichment  or 
remediation  (Edwards,  1999).  In  this  form,  this  model  resembles  Gagne’s  nine  events  of  instruction. 

Duchastel’s  WBI  model  (Duchastel,  1996)  proposes  innovative  changes  to  the  traditional  instruction 
model.  He  defines  several  functions  in  his  model  for  WBI.  He  contrasts  these  functions  with  the  traditional  ones 
found  in  traditional  teaching.  These  functions  are  specifying  the  goals  to  be  pursued  by  learners  instead  of 
specifying  content  to  be  learned,  evaluating  learners  at  the  task  level  instead  of  relying  on  standard  testing,  having 
learners  study  and  work  cooperatively  and  collaboratively  instead  of  studying  alone,  producing  knowledge  rather 
than  communicating  it,  and  creating  global  learning  communities  instead  of  restricting  programs  to  local 
interactions. 

Edwards  (1999)  identified  five  recurring  themes  common  between  these  three  models.  They  are  need  for 
clear  goals,  including  collaborative  learning  teams,  incorporating  motivational  aspects  into  WBI,  turning  instructors 
into  facilitators  who  provide  guidance  and  feedback,  and  facilitating  the  production  of  knowledge  and  the 
development  of  skills  in  WBI. 

He  also  developed  a model  for  WBI  based  on  these  themes.  He  aimed  his  model  specifically  for  higher 
education  and  adult  learners,  but  he  also  added  that  elements  of  this  model  could  be  applied  to  any  type  of  WBI.  The 
basic  foundations  of  Edward’s  WBI  model  are  goal-oriented,  motivational  provisions,  student-centered, 
guidance/feedback  provisions,  collaborative  learning  strategies,  and  project-based. 

It  is  difficult  to  state  that  the  models  identified  by  Reeves  and  Reeves,  Ritchie  and  Hoffman,  Duchastel,  and 
Edwards  are  design  models.  Their  ideas  include  useful  elements  which  make  up  successful  WBI  courses,  but  the 
claimed  models  do  not  tell  us  what  to  do  and  when  to  do  it  during  the  design  of  WBI.  Therefore,  there  is  a need  for  a 
more  robust  ID  model  for  WBI. 

Findings  from  the  Literature  on  the  Current  Practices  in  WBI 

I will  summarize  the  practices  of  WBI  from  the  literature  under  the  following  categories:  How  to  start  the 
conversion  to  the  WBI,  student  and  instructor  support,  design  issues,  interaction  among  people,  and  assessment.  I 
found  these  categories  by  blending  existing  model  components  and  by  putting  together  common  practices  followed 
by  the  practitioners. 

Imagine  yourself  driving  without  a road  map  in  a remote  area,  which  you  have  never  been  before.  How 
would  you  find  your  way  to  your  target?  One  approach  would  be  trial  and  error.  You  could  take  a highway  exit  you 
strongly  believe  will  bring  you  to  your  target.  Or,  you  could  stop  by  at  a gas  station  and  ask  to  the  clerk.  Whatever 
approach  would  be  taken,  finding  your  target  without  a map  would  cause  you  to  lose  valuable  time  and  resources. 
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Instructional  designers  and  instructors  trying  to  convert  their  courses  into  WBI  format  without  a time  - 
proven  solid  model  are  similar  to  those  drivers  trying  to  find  their  target  in  a remote  area  without  a road  map. 
Practitioners  follow  several  approaches  in  the  conversion  process. 

Practitioners  usually  tend  to  start  the  conversion  of  instruction  to  Web-based  format  by  prototyping  the 
WBI  at  informational  level  (Lightfoot,  2000).  To  restate  it  another  way,  designers  usually  develop  a site  first  as  a 
supplement  to  the  in-class  course.  Shave  (1998)  proposes  four  levels  to  use  the  Internet  as  part  of  a course.  They  are: 

• Informational  (Level  1)  in  which  the  Internet  is  used  to  provide  information  to  students  that  is 
relevant  to  the  administration  of  the  class.  Example  information  items  are  timetables,  syllabi,  and  class 
notices. 

• Supplementary  (Level  2)  in  which  additional  resources  are  provided  for  students.  Resources  may 
include  additional  references  and  useful  hints. 

• Dependent  (Level  3)  in  which  the  major  components  of  the  course  are  on  the  Internet  and  students 
need  to  access  these  as  part  of  the  course. 

• Fully  On-line  (Level  4)  in  which  the  entire  course  and  activities  are  on  the  Internet. 

Friedrich  et.  al.  (1999)  go  on  to  say  that  even  a simple  online  syllabus  can  be  advantageous  for  both  the 
instructor  and  the  student  in  the  conversion  process.  These  authors  developed  a Web  site,  which  supplemented  their 
graduate  course  in  statistics  and  measurement.  What  included  on  this  site  were  course  syllabus,  course  procedures 
and  policies,  course  schedule,  individual  units,  and  resources.  Shotsberger  (1996)  surveyed  existing  efforts  at  using 
WWW  for  instructional  purposes  and  found  that  most  existing  sites  were  intended  as  an  adjunct  to  the  classroom. 

Modeling  the  existing  Web-based  courses  is  another  start  point  for  conversion.  Modeling  helps  to 
overcome  limited  time  and  limited  experience  problems.  Friedrich  et.  al.  (1999)  derived  the  basic  site  structure  for 
their  WBI  from  existing  Web  courses  that  they  browsed.  Ingebritsen  et.  al.  (1998)  modeled  their  online  lectures  after 
a face-to-face  lecture  experience. 

Student  and  instructor  support  is  an  important  component  in  WBI.  Duffy  (1992)  states  that  a WBI  program 
will  not  function  without  proper  support.  Student  support  may  be  in  the  form  of  an  orientation.  Instructor  support 
may  be  provided  by  offering  technology  training  to  the  faculty,  and  by  creating  ID  teams  and  technical  support 
teams. 

When  1 started  studying  at  Indiana  University'  I got  an  orientation  from  the  university,  from  the 
Instructional  Systems  Technology  (1ST)  department,  from  the  education  library,  and  etc.  These  orientations 
introduced  me  to  the  environment,  to  the  resources,  and  to  the  important  things  I needed  to  know  to  be  successful  in 
my  program.  WBI  is  a new  learning  environment  for  many  students  and  such  an  orientation  will  not  only  be  an 
icebreaker  to  the  learner  but  it  will  also  show  deficiencies  in  the  WBI  to  the  designers  before  actually  it  starts. 

Kuchinke  et.  al.  (2001)  stated  that  eliminating  as  many  technological  barriers  as  possible  before  the 
beginning  of  the  actual  WBI  is  critical  to  a successful  start.  To  do  this,  they  offered  an  online  tutorial  and  practice 
sessions  two  weeks  prior  to  the  first  course  in  their  WBI.  Their  tutorial  included  such  components  as  practice 
sessions  and  assignments.  The  tutorial  helped  the  students  become  proficient  with  all  hardware  and  software 
requirements  before  beginning  of  the  course.  Friedrich  et.  al.  (1999)  recommends  that  universities  must  support  the 
students  and  train  them  for  online  courses.  This  training  shall  be  in  the  form  of  a workshop  including  such  topics  as 
how  to  take  an  online  course,  how  online  courses  are  different  from  classroom  based  courses,  what  skills  are  needed 
to  be  good  online  students,  and  etc. 

A lot  of  studies  mentioned  lack  of  instructor  experience  in  WBI  as  a problem  (see,  for  example,  Lightfoot, 
2000,  and  Friedrich,  1999).  Some  authors  recommended  creating  a permanent  instructional  design  team  and 
technical  support  team  to  overcome  this  problem  (see,  for  example,  Ingebritsen  et.  al.,  1997,  and  Kuchinke  et.  al., 
2001).  These  teams  would  consist  of  instructional  designers/developers,  and  technical  support  staff  who  will  act  as 
consultants  to  the  instructors.  This  consultation  may  be  in  the  form  of  offering  advice  for  interactivity  and  group 
activities,  assisting  the  instructor  with  creation  of  instructional  materials,  providing  technology  resources,  providing 
technical  assistance  and  training,  and  providing  troubleshooting.  Teams  might  be  located  at  a resource  center.  The 
resource  center  might  also  be  used  for  placing  the  equipment  necessary  for  creating  and  delivering  instructional 
materials.  Ingebritsen  et.  al.  (1997)  utilized  such  a resource  center  while  they  delivered  their  online  course.  The 
center  they  used  included  equipment  such  as  various  Web  servers  and  Web  authoring  computer  terminals,  and  staff 
such  as  technology  specialists  and  undergraduate  students.  Kuchinke  et.  al.  (2001)  received  technical  assistance 
from  a technical  support  team  consisted  of  six  half-time  staff  with  expertise  in  Web  design  and  development. 

I will  categorize  design  issues  under  three  categories:  Web  site  structure,  design  principles,  and  formative 
evaluation. 


ERIC 


363 


852 


Lightfoot  (2000)  proposes  that  overall  Web  site  structure  should  be  organized  into  a Web  tree  that  is  wide 
rather  than  deep.  Making  sites  that  are  wide  prevents  user  from  getting  lost  and  spending  excessive  time  looking  for 
things.  The  components  Lightfoot  (2000)  planned  to  include  in  his  Web  site  are  announcements,  syllabus,  course 
handouts,  lecture  notes,  assignments,  grades  testing,  and  student  support.  He  puts  all  these  components  as  second- 
level  pages  under  course  homepage.  He  further  goes  to  the  third -level  and  defines  these  categories  under  student 
support:  bug  reports,  suggestions.  Frequently  Asked  Questions  (FAQ),  student  discussion,  text  search,  and  other 
useful  links.  Lightfoot  concludes  that  his  implementation  didn’t  match  this  plan,  and  it  differed.  What  Friedrich  et. 
al.  (1999)  added  to  these  components  in  his  WBI  site  are  course  objectives,  course  schedule,  technical  requirements 
and  “about  this  site”  pages. 

Authors  seemed  to  follow  a common  agreement  concerning  design  principles.  Lightfoot  (2000)  indicated 
that  Web  pages  in  WBI  should  be  visually  appealing,  consistent,  and  attractive.  Shotsberger  (1996)  added  simplicity 
to  these. 

Formative  evaluation  is  seen  as  an  often-used  method  in  many  WBI  cases  to  modify  the  design.  Lightfoot 
(2000)  viewed  his  WBI  as  a prototype  and  made  additions  to  and  subtractions  from  his  initial  design  thus 
continuously  evaluating  it.  Tipton  et.  al.  (1998)  conducted  formative  evaluation  with  students  following  each  of  the 
design,  development,  and  implementation  of  their  WBI.  Changes  were  made  to  correct  minor  deficiencies  during  the 
formative  evaluation.  Friedrich  et.  al.  (1999)  indicated  that  optimal  Web  design  will  result  from  fine-tuning  and 
several  iterations  of  initial  design. 

Using  synchronous  and  asynchronous  tools  is  very  common  to  provide  interaction  between  the  instructor 
and  students  and  among  students.  Asynchronous  tools  used  in  various  WBI  cases  include  such  tools  as  lectures 
(Shotsberger,  1996),  e-mail  messages  (Lichty,  1997,  and  Ingebritsen  et.  al.,  1997),  and  threaded  discussion  forums 
(Lichty,  1997,  and  Kuchinke  et.  al.  2001) 

The  most  popular  synchronous  tool  used  in  various  WBI  cases  is  the  chat  function  (see,  for  example, 
Ingebritsen,  1998,  and  Kuchinke  et.  al.  2001). 

When  assessment  is  the  case,  some  WBI  cases  utilized  traditional  tests  and  quizzes  as  a form  of  assessment 
(see,  for  example,  Ingebritsen  et.  al.,  1997,  and  Lightfoot,  2000).  However,  Friedrich  et.  al.  (1999)  indicated  that 
student  assessment  in  a Web-based  course  should  be  different  from  traditional  techniques.  They  evaluated  student 
achievement  in  their  statistics  and  measurement  WBI  course  with  such  authentic  tasks  as  written  reports  and  test 
construction.  Kuchinke  et.  al.  (2001)  made  a similar  conclusion  and  indicated  that  assessment  in  WBI  should  be  in 
performance-based  mode  and  it  should  include  fewer  objective  tests.  Some  authors  indicated  that  immediate, 
frequent,  and  specific  feedback  is  essential  to  provide  accurate  information  regarding  student  performance  (Friedrich 
et.  al.,  1999,  Tipton  et.  al.,  1998,  and  Kuchinke  et.  al.,  2001). 

Proposed  Design  Model 

In  this  part,  a model  proposing  an  orderly  process  for  converting  courses  into  Web-Based  format  will  be 
explained.  Design  models  can  be  built  in  two  ways;  conceptually,  and  empirically.  The  following  model  is  a 
combination  of  the  information  presented  in  the  literature  on  this  topic  and  the  author’s  previous  experience  with 
WBI  design.  Therefore,  it  is  a model  containing  conceptual  and  empirical  elements.  The  model  is  categorized  in  9 
phases. 

It  is  recommended  that  people  using  the  model  go  phase  1 through  phase  9.  However,  different  steps  might 
be  taken  in  each  of  the  phases.  In  addition,  you  do  not  need  to  implement  each  of  the  steps  in  the  phases.  For 
example,  if  you  do  not  plan  to  use  audio  and  video  components  in  your  course,  you  may  not  need  to  provide  a 
facility  for  recording,  digitizing,  and  editing  audio  and  video  files.  Therefore,  you  may  skip  step  4.3. 

Instructional  Systems  Technology  (1ST)  department  at  Indiana  University  has  started  a three-year  Master’s 
Program  to  give  working  professionals  an  opportunity  to  get  an  M.S.  degree  in  instructional  systems  technology 
field.  As  part  of  this  initiative,  residential  courses  in  the  1ST  program  have  been  converted  into  WBI  format.  R51 1 
‘Instructional  Technology  Foundations  I’  was  such  a core  course  to  be  converted  from  residential  format  to  WBI 
format.  The  purpose  of  this  course  is  to  provide  an  introduction  to  the  field  and  profession  of  instructional 
technology.  The  author  of  this  paper  used  the  guidelines  offered  in  the  proposed  design  model  with  success  with  a 
design  team  to  convert  this  course  into  WBI  format.  Therefore,  the  model  has  been  used  in  a higher  education 
setting  for  converting  a course  that  aimed  to  teach  concepts  and  knowledge.  However,  the  model  can  also  be  utilized 
in  other  settings  and  learning  domains. 

Phase  1 - Pre  WBI  Efforts. 
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1.  Infuse  the  technology  into  the  course  prior  to  WBI.  This  may  be  in  the  form  of  word  processing  use, 
spreadsheet  use,  e-mail  use,  Web  use,  and  etc. 

1.1.  In  doing  so,  go  from  level  1 (informational)  to  level  4 (fully  on-line).  Designers  and  instructors  should 
develop  a site  first  as  a supplement  to  the  in-class  course,  in  which  they  provide  information  to  students 
that  are  relevant  to  the  administration  of  the  class.  Then  they  should  go  to  level  2 (supplementary  level) 
and  level  3 (dependent).  After  completing  these  stages,  they  should  implement  the  fully  on-line  level 
(level  4).  Following  these  stages  will  allow  the  parts  to  be  implemented  in  chunks.  In  this  way,  the 
transition  will  be  easier  and  lower  levels  will  provide  a base.  Besides,  instructors  and  designers  will  get 
more  experience  as  the  levels  increase  and  they  will  have  knowledge  about  potential  problems  and  their 
solutions. 

2.  Mirror  closely  the  content,  structure  and  requirements  of  the  traditional  program.  There  must  be  equity  between 
on-site  course  and  WBI  course  in  terms  of  academic  rigor. 

3.  Model  the  existing  Web-based  courses  on  the  Internet.  The  existing  Web-based  courses  may  provide  the 
designers  ideas  on  this  issue. 

4.  For  departmental  programs  (such  as  a master’s  degree  program),  start  with  core  courses  and  add  other  courses 
by  time.  For  this  purpose,  survey  the  faculty  and  ask  the  following  two  questions: 

4.1.  What  courses  have  the  potential  to  convert  to  a WBI  course?  Some  courses  may  not  be  effectively 
delivered  via  the  web. 

4.2.  What  methods  could  best  be  used  to  deliver  the  course? 

Phase  2 - Create  a resource  center  to  support  on-line  course  development  efforts. 

1.  Create  a permanent  technical  support  / technical  assistance  team.  The  purposes  of  this  team  would  be: 

1.1.  To  ensure  that  all  the  technology  is  in  place  and  working  properly, 

1.2.  To  troubleshoot  and  provide  technical  assistance  during  course  delivery, 

1.3.  To  identify  problems  and  suggest  solutions, 

1.4.  To  maintain  network, 

1.5.  To  upgrade  hardware  and  software, 

1.6.  To  implement  web  design  and  development  efforts. 

2.  Create  a permanent  instructional  design  team.  The  design  team  might  include  two  sub-teams: 

2.1.  One  group  might  focus  on  analysis  & design. 

2.2.  The  other  group  might  focus  on  development  & production. 

3.  Provide  technology  training.  The  technology  used  must  be  transparent  to  both  faculty  and  students  during  the 
implementation  of  WBI. 

3. 1 . Provide  student  training. 

3.1.1.  Provide  on-campus  workshops. 

3.1.2.  Provide  Web-based  workshops. 

3.2.  Provide  faculty  training.  Faculty  should  not  be  distracted  by  the  technology  used  in  the  delivery  process. 
Their  task  should  be  addressing  curriculum  issues  instead. 

3.2.1.  Provide  on-campus  workshops. 

3.2.2.  Provide  Web-based  workshops. 

4.  Provide  technology  resources. 

4.1.  Provide  software  technology.  The  following  types  of  software  technologies  might  be  provided: 

4.1.1.  Server  software 

4. 1.1.1.  Webserver 

4. 1 . 1 .2.  Real  media  server 

4. 1.1. 3.  FTP  server 

4. 1.1. 4.  Other  server  software 

4.1.2.  Office  applications 

4. 1.2.1.  Word  processing  software 

4. 1.2. 2.  Spreadsheet  software 

4. 1.2. 3.  Database  software 

4. 1 .3.  Web  design  software 

4. 1 .4.  Graphics  software 

4.2.  Provide  hardware  technology.  The  following  types  of  hardware  technologies  might  be  provided: 

4.2.1.  Servers 

4.2.2.  Web  authoring  stations 
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4.3.  Provide  facility  for  recording,  digitizing,  and  editing  audio  and  video  files. 

Phase  3 - Make  an  analysis.  The  analysis  could  be  implemented  by  surveying  the  potential  or  registered  students. 

1.  Identify  requirements.  Complete  disclosure  of  requirements  will  help  potential  students  make  an  informed 
decisbn  about  whether  this  type  of  learning  environment  is  appropriate  for  them. 

1.1.  Identify  requirements  for  Information  Technology  (IT). 

2.  Make  a learner  analysis.  Possible  data  sources  are  learner  introductions  or  self-reports  done  for  prior  courses, 
learner  preferences  expressed  in  prior  course  evaluations,  and  instructors’  impressions  of  the  salient 
characteristics  of  the  course. 

2. 1 . Make  an  analysis  of  technology  learners  possess. 

2.1.1.  Analyze  learners’  hardware  configuration. 

2. 1.1.1.  Memory  size 

2. 1.1. 2.  Processor  speed 

2. 1.1. 3.  Sound  card  availability 

2. 1.1. 4.  Speaker  availability 

2. 1.1. 5.  CD-ROM  drive  availability 

2. 1.1. 6.  CD-Bumer  Availability 

2.1.2.  Analyze  learners’  software  availability. 

2. 1.2.1.  Browsers 

2. 1.2. 2.  Browser  plug-ins 

2. 1.2. 3.  Utilities 

2. 1.2.4.  Office  applications 

2. 1 .2.4. 1 . Word  processing  software 

2. 1.2. 4. 2.  Spreadsheet  software 

2. 1 .2.4.3 . Database  software 

2. 1 .2.4.4.  Statistics  software 

2.1.3.  Analyze  learners’  Internet  access. 

2. 1.3.1.  Connection  speed 

2.2.  Make  an  analysis  of  the  venues  learners  learn. 

2.2.1.  Home 

2.2.2.  Workplace 

2.2.3.  School 

2.3.  Make  an  analysis  of  previous  experience  learners  have  with  Information  Technology  (IT). 

2.3.1.  Word  processing  experience 

2.3.2.  E-mail  experience 

2.3.3.  Internet  experience 

2.3.4.  Videoconferencing  experience 

2.3.5.  Level  of  confidence  for  using  IT 

3.  Analyze  recommendations  made  by  the  stakeholders  (designers,  instructors,  administrators,  and  etc.)  of 
previous  WBI  courses  offered  in  the  institution  (i.e.,  at  the  department,  at  the  university,  and  etc.) 

4.  Analyze  the  course  being  converted. 

4.1.  Analyze  previous  years’  student  evaluation  data  (if  available)  of  the  on-site  course. 

4.2.  Obtain  a description  of  the  previous  content  and  approach  used  in  the  residential  version  of  the  course 
(The  easiest  way  is  to  obtain  the  course  syllabus). 

5.  Analyze  the  existing  course  management  software  (i.e.,  SiteScape,  WebCT,  BlackBoard,  Oncourse,  and  etc.) 
and  select  the  most  appropriate  one  aligned  with  course  content,  and  course  activities. 

6.  Decide  on  pedagogy  (Problem  based,  group  work,  and  etc.). 

Phase  4 - Identify  instructional  strategies.  Provide  students  ways  in  which  they  can  practice  the  knowledge  from 
WBI  course  in  meaningful  ways. 

1.  Make  the  student  an  active  participant  in  the  learning  process. 

1.1.  Follow  a student-centered  model  instead  of  teacher-centered  model. 

1.2.  Put  the  instructor  as  the  mentor/supporter  in  the  WBI  model. 

1.3.  Provide  collaborative  learning  options  to  overcome  isolation  in  distance  learning. 

1.3.1.  Provide  interactivity  using  different  forms 

1 .3.2.  Use  multiple  sources  of  information 


1. 3.2.1.  Develop  partnerships  to  share  resources. 

1 .3.2.2.  Provide  access  to  authentic  research  databases. 

1 .3.2.3.  Provide  access  to  educational  resources  from  other  colleges  or  universities. 

1.4.  Utilize  multiple  lines  of  communication  among  participants. 

1.4.1.  Allow  synchronous  communication  between  remote  parties.  The  following  Internet  tools  might  be 
used  for  this  purpose: 

1.4. 1.1.  Chat  tools 

1 .4. 1 .2.  Instant  messaging  tools 

1.4. 1.3.  Audio  conferencing  tools 

1 .4.1.4.  Video  conferencing  tools 

1 .4.2.  Allow  asynchronous  communication  between  remo  te  parties.  The  following  Internet  tools  might 
be  used  for  this  purpose: 

1.4. 2.1.  e-mail 

1 .4.2.2.  Threaded  discussion  groups 

1.5.  Foster  interaction  and  collaboration  among  students. 

1 .6.  Allow  students  build  informal  networks  for  supporting  each  other  professionally  and  personally.  The 
following  tools  might  be  used  for  this  purpose: 

1.6.1.  Electronic  cafe 

1.6.2.  Phone 

2.  Support  multiple  learning  styles. 

2. 1 . Allow  students  learn  by  seeing  and  hearing  information. 

2.2.  Allow  students  learn  by  doing  active  learning  assignments. 

2.3.  Allow  students  learn  by  reading  material. 

3.  Provide  problem-solving  activities. 

Phase  5 - Provide  a well  planned  administrative  structure.  This  structure  should  facilitate  the  communications  and 
exchanp  between  the  university,  the  department  and  distance  student.  Although  WBl  students  contact  with  the 
institution  less  frequent  than  conventional  instruction,  the  interaction  is  more  meaningful. 

1.  Provide  secretarial  personnel. 

2.  Provide  clerical  personnel. 

3.  Provide  para-professional  personnel. 

Phase  6 - Design  and  develop  the  WBI. 

1.  Design  team  members  should  have  a shared  vision  of  how  this  conversion  project  has  to  come  together. 

2.  Organize  the  development  efforts  with  the  following  labels: 

2. 1 . Content  Materials  (the  resources  provided  by  the  client) 

2.2.  Instructional  design  documentation  (the  instructional  units  developed  by  the  development  team) 

2.3.  Interface  design  documentation  (templates,  web  site  mock  ups,  sketches) 

2.4.  Multimedia  resources  (multimedia  components  of  the  course  and  graphics) 

2.5.  Sample  documentation  (examples  of  navigation  and  ID  documentation  from  previous  WBI  courses) 

2.6.  Project  management  documentation  (production  plans,  roles,  interim  reports) 

3.  Organize  the  design  and  development  team  around  the  following  roles: 

3.1.  Project  manager 

3.2.  Technology  manager 

3.3.  Documentation  manager 

3.4.  Content  manager 

3.5.  Regular  member 

4.  Meet  periodically  with  the  client  to  make  major  decisions  and  to  update  them  on  the  design  progress 

5.  Prepare  a general  template  and  use  it  for  each  of  the  course  modules/weeks/structure. 

6.  Balance  the  residential  content  of  the  course  by  reducing/adding  the  content. 

7.  Include  the  following  components  on  the  WBI  web  site: 

7.1.  A homepage 

7.2.  A detailed  course  syllabus 

7.3.  Expectations  from  students 

7.4.  Deadlines 

7.5.  Grading  criteria 
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7.6.  Course  objectives 

7.7.  Course  procedures  and  policies 

7.8.  Course  schedule 

7.9.  Links  to  course  content 

7.10.  Resources 

8.  Supplement  course  content  with  multimedia  components. 

9.  Be  consistent  throughout  the  web  site  in  terms  of  format  (i.e.,  same  PowerPoint  format)  and  phrasing. 

10.  Provide  documentation  to  the  course  instructor. 

Phase  7 - Eliminate  technological  barriers  as  much  as  possible  before  the  beginning  of  the  actual  WBI  course. 

1.  Support  and  train  the  students  for  online  courses.  In  traditional  classroom  instruction,  students  are  ready  to 
participate  in  the  course.  However,  learning  in  a WBI  course  requires  a set  of  skills,  knowledge  and  abilities 
some  students  are  not  prepared  for.  Preparation  for  both  technical  aspects  and  for  the  distance-learning  mode  is 
necessary.  In  this  sense,  the  following  should  be  implemented: 

1.1.  Provide  students  an  orientation  period  (online  tutorial  and  practice  sessions)  before  the  actual  beginning  of 
the  class.  These  sessions  will  help  students  become  familiar  with  the  tools  and  techniques  of  online 
learning  before  the  beginning  of  the  WBI  course. 

1.2.  Provide  students  a basic  computer  literacy  course.  The  following  topics  might  be  covered  in  this  course: 

1.2.1.  Multimedia  software 

1.2.2.  Web  authoring  software 

1 .2.3.  Conferencing  tools 

1.2.4.  Sending  e-mail 

2.  Support  and  train  the  instructors. 

Phase  8 - Assess  the  students. 

1.  Put  emphasis  on  application  rather  than  content  acquisition  during  the  assessment.  Assess  the  student 
achievement  with  authentic  assessment  tasks.  These  authentic  assessment  tasks  might  be: 

1.1.  Deliverable  driven 

1.2.  Discussion  driven 

1.3.  Presentation  driven 

2.  Provide  formative  evaluation  techniques. 

2.1.  Utilize  short  answer  quizzes. 

3.  Provide  summative  evaluation  techniques.  In  doing  so,  use  fewer  objective  tests  than  traditional  settings. 

3.1 . If  you  utilize  traditional  assessment  techniques,  allow  learners  to  use  a cheat  sheet  to  curtail  academic 
dishonesty. 

4.  Provide  grades  with  password  protection  so  that  students  can  access  only  to  their  own  grades. 

5.  Provide  timely  feedback  on  all  kinds  of  assignments. 

Phase  9-  Evaluate  the  WBI. 

1.  Provide  ongoing  formative  evaluation.  Keep  the  features  that  are  deemed  useful,  eliminate  others  or  modify 
them. 

1.1.  Maintain  the  web  site  regularly. 

1.2.  Eliminate  the  bugs. 

1.3.  Proofread  and  revise  the  site  content.  Look  for  typos,  punctuation  and  capitalization  problems, 
grammatical  errors,  and  content  that  seems  out  of  plan. 

1 .4.  Satisfy  with  the  short-term  results  and  use  the  initial  feedback  to  continue  with  the  development  of  the 
WBI. 

1 .5.  Survey  students  regarding  their  perceptions  of  the  WBI.  For  this  purpose,  use  a method  that  allows 
students  to  make  anonymous  comments  on  the  class  while  the  WBI  is  underway. 

1 .6.  Test  the  site  across  different  browsers  and  computer  platforms  (Windows  and  Mac). 

2.  Calculate  potential  savings  by  converting  the  courses  to  WBI  mode,  and  Return  on  Investment  (ROI).  Effective 
WBI  courses  require  the  allocation  of  a variety  of  resources.  These  resources  make  up  the  overall  cost  of  the 
WBI  course.  In  return,  the  institution  expects  benefits  as  the  result  of  the  investment.  If  benefits  exceed  costs, 
the  WBI  course  can  be  considered  as  a viable  academic  alternative. 

Conclusion 
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This  literature  review  showed  that  without  a time -proven  WBI  model,  practitioners  have  been  trying  to 
convert  existing  courses  into  Web-based  format.  Since  they  do  not  follow  a common  model,  there  seems  to  be  an 
inconsistency  in  the  approach  taken.  Everybody  is  trying  to  do  something,  but  they  are  losing  valuable  time  and 
resources.  There  is  an  absolute  need  for  a solid  model  for  both  WBI  and  conversion  of  instruction  to  Web -based 
format. 

Although  practitioners  did  not  follow  a common  model,  their  conversion  of  instruction  to  Web-based 
format  included  some  common  categories  such  as  how  to  start  the  conversion,  how  student  and  instructor  support 
should  be,  design  issues,  how  interaction  among  people  should  be,  and  how  assessment  should  be.  Examples  for 
each  of  these  categories  were  given. 

There  are  more  to  issues  to  cover  in  this  study  such  as  collaboration,  administration,  and  other 
situationalities.  However,  limited  space  and  time  does  not  permit  to  cover  all  of  them.  Since  this  is  a research  in 
progress  such  issues  will  be  extended  in  the  future. 
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Introduction 

In  1998,  the  Department  of  Education  implemented  “principles  of  effectiveness”  requiring  that  all  Safe  and 
Drug-Free  Schools  and  Communities  (SDFSCP)-funded  programs  be  research-based.  To  help  schools  identify  and 
adopt  research-based  drug  and  violence  prevention  strategies,  the  1999  Safe  and  Drug-Free  School  appropriation  of 
$566  million  included  $35  million  for  the  first  year  of  a new  Middle  School  Drug  Prevention  and  School  Safety 
Program  Coordinator  initiative. 

This  new  initiative  supports  the  hiring  and  training  of  full-time  Middle  School  Coordinators  (MSCs)  to 
oversee  implementation  of  drug  prevention  and  school  safety  programs  for  students.  Specifically,  MSCs  are 
expected  to: 

• Identify  research-based  drug  and  violence  prevention  strategies. 

• Assist  schools  in  adopting  the  most  successful  strategies. 

• Develop,  conduct,  and  analyze  assessments  of  school  crime  and  drug  problems. 

• Work  with  community  agencies  and  organizations  to  ensure  students’  needs  are  met. 

• Encourage  parents  and  students  to  participate  in  the  identification  and  implementation  of  research-based 

prevention  efforts. 

• Assist  in  the  development  and  implementation  of  evaluation  strategies. 

• Identify  additional  funding  sources  for  prevention  and  school  safety  programming. 

• Provide  state  education  agencies  feedback  on  successful  programs  and  activities. 

• Coordinate  with  student  assistance  and  employee  assistance  programs. 

To  help  recently  hired  Middle  School  Drug  Prevention  and  School  Safety  Coordinators  meet  the  roles  and 
responsibilities  of  their  new  jobs,  the  U.  S.  Department  of  Education,  Safe  and  Drug-Free  Schools  contracted  with 
Education  Development  Center  (EDC)  and  its  partner  Social  Science  Research  (SSRE)  to  create  a national  center  for 
training  and  assistance. 

Schools  and  communities  face  an  urgent  challenge  to  design  effective  solutions  to  the  complex  problems  of 
violence,  alcohol,  tobacco,  and  other  drug  use.  To  assist  schools  in  the  selection  and  implementation  of  effective 
prevention  programs  that  are  responsive  to  their  needs,  the  U.  S.  Department  of  Education  has  undertaken  several 
initiatives  designed  to  enhance  schools'  understanding  of  what  works  and  expand  the  inventory  of  effective 
programs.  Among  these  initiatives  is  the  Safe  and  Drug-Free  Schools  Middle  School  Drug  Prevention  and  School 
Safety  Coordinator  Program,  which  supports  the  recruiting,  hiring,  and  training  of  one  or  more  full-time  staff  for 
three  years  to  guide  the  implementation  of  drug  prevention  and  school  safety  programs  for  middle  school  students. 

A well-trained,  full-time  coordinator  who  is  familiar  with  the  research  on  effective  prevention 
programming  and  who  bases  programming  on  sound  planning  that  involves  assessment,  measurable  goals  and 
objectives,  effective  research-based  strategies,  and  evaluation,  should  be  able  to  implement  prevention  programs  that 
better  meet  the  needs  of  the  students  in  their  schools. 

Coordinators,  however,  face  several  challenges  to  implementing  effective  prevention  programs: 
determining  which  strategies  and  programs  are  effective  at  reducing  substance  use  and  violence  among  young 
adolescents,  monitoring  program  activities  implemented  in  the  school;  community  members  who  do  not  believe 
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there  are  drug  problems  among  their  youth;  and  other  school  district  priorities  that-while  they  may  be  desirable  for 
other  reasons  may  interfere  with  prevention  efforts. 

The  goal  of  the  Training  Center  is  to  work  with  middle  school  coordinators  through  face-to-face  trainings 
and  web-based  continuing  education  to  enhance  their  understanding  of  research-based  programs  and  equip  them 
with  the  skills  necessary  to  identify  research-based  strategies  based  on  an  assessment  of  needs  in  their  district. 

The  U.S.  Department  of  Education’s  Safe  and  Drug-Free  Schools  and  Communities  Program  (SDFSCP), 
funded  under  Title  IV  of  the  improving  America’s  Schools  Act  of  1994,  provides  funds  for  virtually  every  school 
district  in  America  to  support  drug  and  violence  prevention  programs. 

Background 

Three  online  continuing  education  events  for  United  States  Department  of  Education  Middle  School  Drug 
Prevention  and  School  Safety  Coordinators  were  held  between  April  and  June  2001.  The  purpose  of  these  events 
was  to  provide  a menu  of  skill-based,  interactive  learning  activities  that  supplement  the  core  training  workshop  (see 
Harding  and  Formica  2000). 

One  hundred  and  eighty-six  (186)  of  the  634  MSCs  (29%)  who  participated  in  the  five-day  core  training 
workshops  registered  for  at  least  one  of  the  three  online  events.  Other  events  will  be  offered  in  the  future. 

This  report  summarizes  evaluation  findings  across  the  three  online  events.  Separate  reports  presenting 
evaluation  findings  from  each  of  the  events  are  also  available  (see  Harding  and  Formica  2001a,  2001b;  Formica  and 
Harding  2001). 

Goals  and  Content  of  the  Online  Events 

The  three  continuing  education  online  events  were  intended  to:  (1)  provide  support  for  the  implementation 
of  best  practices  that  enable  organizational  change,  (2)  build  on  the  foundation  established  by  the  five-day  core 
training,  and  (3)  foster  the  exchange  of  information  and  ideas  that  can  transfer  knowledge  into  practice. 

Each  of  the  continuing  education  online  events  is  designed  and  sequenced  to  assist  MSCs  in  addressing  a 
set  of  critical  factors  for  the  successful  implementation  of  effective  prevention  programs:  (1)  assessing  local  needs 
and  assets,  (2)  designing  programs  to  meet  desired  results,  using  measurable  goals,  (3)  selecting  and  implementing 
programs  that  are  based  on  research,  and  (4)  evaluating  and  refining  program  efforts. 

The  first  online  continuing  education  event,  “Using  Existing  Data  in  Your  Needs  Assessment,”  was 
designed  to  help  MSCs  locate  and  use  available  local,  state,  and  national  data  to  determine  drug  and  violence 
prevention  priorities  and  select  prevention  programs  for  their  schools  and  communities. 

The  second  continuing  education  event,  “Identifying  Priorities  and  Strategies  for  Your  Prevention 
Initiative,”  was  designed  to  facilitate  the  development  of  a comprehensive  prevention  plan.  This  event  targeted 
MSCs  who  had  collected  information  about  local  needs  and  were  in  the  process  of  collaborating  with  school  and 
community  partners  to  translate  community  data  into  prevention  priorities  and  long-term  outcome  statements,  and  to 
identify  research-based  strategies  that  will  help  achieve  those  outcomes. 

The  third  continuing  education  event,  “Promoting  Prevention  Through  School-Community  Partnerships,” 
was  designed  to  provide  MSCs  with  tools  and  resources  to  build  school-community  partnerships  that  will  support 
and  enhance  their  overall  prevention  initiative. 

Organization  and  Structure  of  the  Online  Events 

Each  of  the  online  events  consisted  of  seven  components:  (1)  an  introduction,  (2)  a set  of  clearly  defined 
skills  and  strategies  to  be  addressed,  (3)  a set  of  methods,  materials,  and  timelines,  (4)  structured  activities,  (5) 
discussion  areas,  (6)  an  event  summary,  and  (7)  client  self-assessment  and  evaluation  surveys.  The  models  used  to 
structure  the  events  included: 

• Library  Model:  This  includes  access  to  online  resources  such  as  journals,  reading  lists,  websites, 
and  other  subject-related  sites  rich  in  relevant  information 

• Textbook  Model:  This  model  includes  the  use  of  course  and  lecture,  slide  presentations,  and  other 
class-related  documents  in  various  media  formats. 
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• Computer-mediated  Communication  Model  (CMC):  This  model  features  collaborative  learning  for 
communication  and  eliciting  feedback  through  the  use  of  asynchronous  and  synchronous 
interaction  with  a facilitator  or  moderator. 

• Virtual  Classroom  Model:  This  model  includes  the  three  aforementioned  models  plus  additional 
elements  of  interactive,  computer-based  instruction  such  as  simulations,  games,  and  various  forms 
of  synchronous  interaction. 

These  models  were  used  in  various  combinations  for  each  event  with  the  predominate  mix  being  that  of  the 
library,  textbook,  and  CMC  models. 

Evaluation  Methods 

Coordinators  were  asked  to  complete  a post-event  online  survey.  These  surveys  asked  MSCs  to  provide 
demographic  information,  rate  the  overall  online  events  and  the  different  aspects  of  the  events  such  as  the  quality 
and  usefulness  of  the  information  provided,  answer  true/false  knowledge  items  about  event  content,  and  answer  self- 
report  knowledge  gain  items.  The  surveys  also  included  several  open-ended  items  that  allowed  coordinators  to 
report  the  most  and  least  helpful  features  of  the  overall  events  and  to  write  any  additional  comments  or  suggestions. 

For  the  purposes  of  this  summary  report,  statistical  analyses  were  run  on  the  data  collected  from  the  three 
different  continuing  education  online  events  to  determine  whether  or  not  there  were  any  significant  differences  in 
the  responses  given  by  the  coordinators.  Results  of  these  analyses  (ANOVAs)  support  combining  responses  across 
all  three  online  events.  Additional  analyses  were  conducted  to  determine  whether  any  differences  existed  between 
active  participants  and  auditors,  and  whether  participation  in  multiple  events  was  more  beneficial  that  participation 
in  a single  event. 

All  instruments  used  to  evaluate  the  events  were  submitted  to  the  Office  of  Management  Budget  (0MB) 
prior  to  the  events  to  receive  approval  under  the  1 800-00 1 1 Master  Plan  for  Customer  Surveys  and  Focus  Groups. 

Results  & Discussion 

in  general,  results  concerning  participants’  satisfaction  with  the  three  five-day  continuing  education  online 
events  were  very  positive.  On  a web-based  post-event  survey  completed  after  participation  in  each  of  the  online 
events,  coordinators  were  asked  to  rate  their  overall  satisfaction  with  the  online  events  on  a 5-point  scale:  Very 
Dissatisfied,  Somewhat  Dissatisfied,  Neutral,  Somewhat  Satisfied,  and  Very  Satisfied.  Eighty-two  percent  (82%) 
indicated  that  they  were  either  “Very  Satisfied”  (42%)  or  “Somewhat  Satisfied”  (40%)  with  the  online  events. 
Participants  were  also  asked  to  indicate  whether  or  not  they  would  recommend  the  online  events  to  other  MSCs, 
how  much  new  information/ideas  they  received  through  participation  in  the  events,  and  the  usefulness  of  the 
information  discussed  during  the  online  events.  These  results  were  also  very  positive.  Eighty-eight  percent 
indicated  that  they  would  either  “Strongly  Recommend”  (46%)  or  “Recommend”  (42%)  the  online  events  to  other 
MSCs,  79%  indicated  that  they  received  either  “ A Lot  of  New  Information/Ideas”  (27%)  or  “Soire  New 
Information/Ideas”  (52%),  and  87%  indicated  that  the  information  discussed  during  the  online  events  was  either 
“Very  Useful”  (43%)  or  “Somewhat  Useful”  (44%). 

Participants  were  also  asked  to  rate  their  satisfaction  with  five  elements  of  the  online  events  on  a 5-point 
scale:  Very  Dissatisfied,  Somewhat  Dissatisfied,  Neutral,  Somewhat  Satisfied,  and  Very  Satisfied.  Eighty-seven 
percent  indicated  that  they  were  “Somewhat”  or  “Very  Satisfied”  with  the  quality  of  the  materials,  84%  with  the 
organization/layout  of  the  event  website,  83%  with  the  download  speed  of  the  web  pages,  82%  with  the  links 
provided  to  other  websites,  and  70%  with  the  role  of  the  online  facilitators. 

Additional  analyses  were  conducted  comparing  active  participants  versus  auditors.  The  differences 
between  active  participants  and  auditors  were  both  small  and  statistically  not  significant  for  the  amount  of  new 
information  they  acquired,  the  usefulness  of  the  information  they  received,  and  the  time  they  spent  on  the  events,  the 
true/fals  e items  tailored  to  each  event,  overall  satisfaction  with  the  events,  and  willingness  to  recommend  the  events 
to  other  MSCs.  However,  although  non -significant,  auditors  did  report  that  they  gained  more  knowledge  for  all  of 
the  dimensions  assessed  for  events  two  and  three. 

As  mentioned  earlier  in  the  report,  the  online  courses  were  originally  designed  such  that  there  was  a very 
clear  distinction  made  between  active  participants  and  auditors.  In  reality,  the  differences  between  the  two  groups 
were  very  minimal.  Given  the  lack  of  distinction  between  the  two  groups,  it  is  not  unexpected  to  find  that  they  did 
not  systemically  differ  on  the  majority  of  the  dimensions  assessed.  Unfortunately,  this  limits  our  ability  to  assess 
with  any  certainty  whether  or  not  the  additional  components  received  by  active  participants  (e.g.,  facilitated 
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discussion  areas)  had  a more  positive  effect  on  these  MSCs  or  whether  these  additional  components  made  no 
difference.  In  light  of  the  high  level  of  performance  and  satisfaction  measures  across  groups,  we  might  assume  that 
both  groups  benefited  from  these  additional  components,  but  further  research  with  more  precise  distinction  between 
groups  is  needed. 

A second  set  of  analyses  compared  MSCs  who  participated  in  multiple  events  with  those  who  participated 
in  only  one  event.  In  general,  participation  in  multiple  events  was  associated  with  higher  levels  of  performance  and 
satisfaction  with  the  events.  For  example,  MSCs  who  participated  in  multiple  events  were  significantly  more  likely 
to  indicate  that  they  received  more  information  from  the  events  and  more  apt  to  report  increases  in  knowledge 
gained  (p<.05). 

Although  not  significant,  MSCs  who  participated  in  multiple  events  were  also  more  likely  to  rate  the 
usefulness  of  the  information,  the  ease  of  navigation,  the  ease  of  use  of  the  discussion/chat  area,  overall  satisfaction 
with  the  event,  and  willingness  to  recommend  the  event  to  other  MSCs  more  highly  than  MSCs  who  participated  in 
a single  event. 

While  the  different  online  events  are  designed  to  be  mutually  exclusive  of  one  another,  there  is  a certain 
degree  of  overlap  and  a logical  progression  of  themes  from  needs  assessment  through  identifying  priorities  to 
building  partnerships.  The  pattern  of  results  for  MSCs  who  participated  in  multiple  events,  especially  the  significant 
increases  in  information  received  and  self-reported  knowledge  gain,  is  consistent  with  the  cumulative  nature  of  the 
different  events.  In  addition,  the  format  and  timeframe  for  the  events  (one  week  with  50  active  participants  and  an 
unlimited  number  of  auditors  per  event)  suggest  that  further  studies  examining  alternative  designs  for  continuing 
education  web-based  events  for  adults  is  warranted.  These  findings  also  suggest  that  it  is  important  to  encourage 
MSCs  to  participate  in  multiple  events. 
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Abstract 

This  paper  reports  the  results  of  a two-part  qualitative  study  into  student  strategies  for  learning  in  distance 
learning  courses  offered  at  Teachers  College,  Columbia  University,  Verbal  Protocol  Analysis  was  used  to 
gather  information  on  the  cognitive  strategies  students  used  to  solve  problems  encountered  as  they 
navigated  through  the  site.  A semi -structured  interview  was  also  conducted  with  participants  to  gain 
insight  into  student  attitudes  and  perspectives  on  learning  in  the  online  environment.  The  theoretical 
framework  of  Activity  Theory  was  used  to  develop  a deeper  understanding  of  the  systemic  roots  of  tensions 
in  the  online  classroom  environment.  Conclusions  include  suggestions  for  specific  teaching  practices  and 
design  solutions  to  alleviate  these  tensions. 

Introduction 

Online  education  is  a relatively  new  phenomenon,  which  is  increasingly  seen  by  institutions  and  educators 
as  a viable  alternative  to  the  traditional  face-to-face  class.  Past  research  into  online  education  has  focused  primarily 
on  media  comparison  studies.  These  studies  have  generally  resulted  in  a ‘no  significant  difference’  finding,  which 
Lockee,  Burton  and  Cross  (1999)  have  criticized  as  useless  research,  “...failure  to  reject  the  null  hypothesis  means 
Just  that  and  nothing  more;  just  as  a legal  finding  of  not  guilty  does  not  mean  innocent.”  (Lockee  et  al.  p.  38). 

Studies  that  rely  on  comparing  achievement  on  classroom  assessment  measures  between  in-person  courses  and 
online  courses  tell  us  nothing  about  the  actual  effect  of  the  distance  learning  media  on  the  experience  of  the 
individual  learner.  Mclsaac  and  Gunawardena  (1996)  recommend  that  future  research  focus,  in  part,  on 
*‘Examin[ing]  the  characteristics  of  the  distance  learner  and  investigat[ing]  the  collaborative  effects  of  media 
attributes  and  cognition”  (p.  431). 

The  need  for  further  and  more  focused  research  was  reiterated  by  Harasim  (1996)  who  claimed  that  it  was 
no  longer  necessary  to  prove  the  viability  of  educational  computer  networking;  rather,  further  research  should  study 
the  learning  processes  in  online  environments  and  “the  patterns  of  human  interaction  in  decision-making,  problem- 
solving, and  knowledge  building.” 

Therefore,  this  qualitative  study  departs  not  with  the  goal  in  mind  of  trying  to  prove  or  disprove  the 
effectiveness  of  the  online  medium,  but  rather  from  the  assumption  that  the  Internet  is  a viable  delivery  mechanism, 
and  the  design  of  Internet-based  programs  needs  to  take  into  account  the  processes  and  strategies  used  by  the  learner 
in  order  to  increase  effectiveness. 

Methodology  and  Analytic  Framework 

This  pilot  study  used  a grounded  theory  approach  (Glaser  and  Strauss,  1967)  to  qualitative  research  in  order 
to  find  patterns  and  strategies  for  learning  that  emerge  from  the  learner’s  perspective,  using  the  process  of  sorting, 
coding  and  comparing  the  data  collected  in  interviews  and  observations  of  students  enrolled  in  online  courses 
through  Teachers  College  Distance  Learning  Project  (DLP).  Two  online  platforms  are  used  in  the  DLP,  Blackboard 
and  Prometheus.  These  commercial  educational  web  site  templates  provide  a course  structure  for  teachers  as  well  as 
communications  tools  to  facilitate  interaction. 

Engstrom’s  Activity  Theory  model  was  used  as  the  framework  for  analyzing  student  behavior  in  the  online 
environment.  Theoretical  propositions  concerning  the  components  of  successful  online  course  design  emerged  from 
careful  analysis  of  (i)  the  behaviors  and  attitudes  students  demonstrate  as  they  move  through  their  online  course;  (ii) 
the  level  of  interaction  between  the  learner  and  the  different  aspects  of  the  online  course;  (iii)  the  steps  and  decisions 
that  the  learner  makes  while  navigating  the  course  site. 

Procedures 

Using  a technique  known  as  Verbal  Protocol  Analysis  (VP A),  based  on  Ericsson  and  Simon’s  “Thinking 
Aloud  Processes”  (1993:78),  the  participants  were  instructed  to  verbalize  their  thoughts  as  they  navigate  through  the 
course  web  site.  This  process  was  captured  on  videotape  by  placing  the  camera  behind  the  participant  and  focusing 
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on  the  computer  screen,  while  a microphone  recorded  the  speech.  The  purpose  of  the  verbal  protocol  analysis  was  to 
observe  what  actually  happens  when  students  participate  in  an  online  course,  and  to  reveal — as  they  happen — the 
points  at  which  students  make  decisions  which  will  potentially  affect  their  learning  experience.  During  a semi- 
structured  interview,  the  participants  were  asked  to  reflect  more  deeply  on  their  actions  as  exhibited  during  the  VPA 
task. 

The  think-aloud  technique  requires  the  participant  to  verbalize  her  thoughts  as  she  works  through  a 
cognitive  task.  The  procedure  is  designed  to  reveal  the  cognitive  processes  the  participant  is  undergoing  as  she 
performs  the  task.  Social  verbalization  includes  explanation  and  description  of  one’s  thoughts  while  performing  a 
task.  Social  verbalization  is  distinct  from  think-aloud  through  the  addition  of  explanatory  and  descriptive  elements. 
In  order  to  minimize  the  amount  of  social  verbalization  and  maximize  the  think-aloud  activity,  the  researchers 
executed  the  following  procedure  as  outlined  by  Ericsson  and  Simon  (1993): 

First  the  experimental  situation  is  arranged  to  make  clear  that  social  interaction  is  not  intended,  and  the 
experimenter  is  seated  behind  the  subject  and  hence  is  not  visible.  The  “think-aloud”  instruction  explicitly 
warns  the  subjects  against  explanation  and  verbal  description.  Second,  after  the  instruction  is  presented, 
the  subjects  are  given  practice  problems  in  which  it  is  easy  to  verbalize  concurrently  and  from  which  they 
attain  familiarity  with  the  normal  content  of  think-aloud  verbalizations  (p.  xiv). 

The  participants  were  instructed  to  verbalize  their  own  thoughts;  they  were  told  not  to  explain  anything  to  the 
researchers  and  that  in  fact  there  would  be  no  interaction  during  the  think-aloud  unless  the  participant  grew  quiet,  at 
which  point  the  researcher  would  prompt  the  participant  to  “keep  talking.”  The  participants  were  given  a math 
problem  and  a language  problem  to  solve  (Appendix  A),  as  a way  to  practice  the  think-aloud  technique.  The  think- 
aloud  procedure  is  difficult  for  individuals  to  execute.  According  to  Ericsson  and  Simon  (1983)  “When  an 
investigator  instructs  a subject  to  think-aloud,  some  subjects  may  misunderstand  the  instruction  and  produce  instead 
the  more  common  social  communication,  explaining  or  describing  the  process  to  the  experimenter”  (p.  xiv).  The 
participants  in  this  study  produced  both  social  communications,  explaining  and  describing  the  web  site  to  the 
researchers,  and  true  think-aloud  verbalizations.  However,  the  information  that  was  received  from  the  participants  is 
rich  data,  which  reveals  much  about  the  online  educational  experience  for  them. 

After  the  think-aloud  section  of  the  experiment,  brief  interviews  were  conducted  with  each  participant.  The 
interview  sought  to  uncover  further  information  about  the  approaches  to  learning  the  participant  employed  in  the 
online  course.  The  interview  also  posed  questions  about  the  social  aspects  of  the  experience. 

Sample 


A total  of  four  participants  took  part  in  this  pilot  study.  The  participants  voluntarily  took  part  in  the  study 
and  they  represent  a convenience  sample  of  students  taking  online  courses  at  Teachers  College  in  the  spring,  2001 
semester.  The  four  students  were  all  enrolled  in  different  courses.  The  results  of  the  study  are  not  generalizable  to  a 
larger  population  and  serve  mainly  to  provide  insight  into  the  experience  of  the  individual  participants.  Table  1 
contains  the  ethnographic  and  questionnaire  data  obtained  from  the  participants.  Participant  names  are  fictitious. 


Table  1 
Participant 

Degree  Program 

A 

Karen 

MA  Sociology  & 

g 

e 

3 

Education 

0 

Myriam 

Ed.M.  International 

3 

Ed. 

2 

Ann 

MA  Sociology  & 

2 

Education 

5 

Jean 

Not  enrolled  in 

4 

Degree  Prog. 

8 

Ethnicity 

How  many  online  courses  have 
you  taken  (including  this  one)? 

White/Latina 

One 

Latina 

One 

White 

One 

White 

Two 

Findings 


Several  important  themes  emerged  from  the  analysis  of  the  transcripts  including:  web  site  design  flaws;  the 
use  of  cognitive  strategies  and  coping  strategies;  the  effects  of  virtuality;  and  the  effects  of  learning  differences  in 
the  online  classroom.  These  themes  have  important  implications  for  future  design  on  online  learning  environments, 
and  each  theme  on  its  own  could  be  the  focus  of  future  research. 

Web  Site  Design  Flaws 

Each  participant  expressed  some  confusion  regarding  orientation  to  the  organization  of  the  class  and  the 
flow  of  the  syllabus.  The  participants  attributed  this  confusion  to  the  instructor’s  layout  of  the  course  materials  in  the 
online  environment.  Ann  experienced  the  most  bewildering  design  of  all  four  participants.  The  online  course  she 
took  used  three  different  web  sites,  with  three  separate  URLs  to  convey  the  subject  matter  (Professor’s  self-created 
web  site),  conduct  the  online  discussions  (Prometheus  platform)  and  hold  the  synchronous  chats  (Blackboard 
platform).  During  the  think-aloud  section  Ann  made  mention  of  the  difficulty  in  navigating  through  the  various  sites 
associated  with  the  course  six  times.  The  Professor’s  self-created  web  site  was  particularly  frustrating  for  Ann. 
Among  the  many  navigational  problems  with  this  web  site,  the  Professor  used  a moving  JavaScript  window  as  a 
menu  bar.  This  box  moved  down  the  page  as  the  viewer  scrolled  down,  obscuring  parts  of  the  web  site  that  the 
viewer  wished  to  see.  In  the  interview  section  Ann  expressed  the  desire  for  a more  transparent  design.  “I  wish  they 
all  would  have  been  on  the  same  web  site.  I wish  there  would  have  been  like  a way  to  directly  link  between.  It  took 
me  a long  time  just  to  remember  the  names  [URL’s].”  This  course  design  was  a source  of  frustration  for  Ann,  which 
unnecessarily  interfered  with  the  learning  experience. 

Karen  also  experienced  confusion  and  frustration  with  the  layout  of  the  course.  The  course  Karen  took  was 
presented  using  the  Blackboard  platform.  Karen  stated  during  the  think-aloud,  “but  this  is  where  its  really 
confusing  because.  I'm  here  and  this  is  called  equity,  this  week's  called  equity,  but  if  you  go  back  to  course 
documents  where  it  gives  us  the  readings  to  prepare  for  class  you  don't  see  anything  listed  as  equity.”  As  Karen 
made  this  comment  she  was  jumping  back  and  forth  between  the  Communications  section  of  the  site  and  the  Course 
Documents  section  of  the  site  in  an  attempt  to  figure  out  which  discussion  topic  corresponded  with  the  current 
week’s  readings.  The  lack  of  consistent  labeling  between  the  course  documents  section  topic  listings  and  the 
corresponding  discussion  forum  in  the  communications  section  was  a design  flaw  that  caused  problems  for  Karen. 

To  cope  with  the  disorientation  she  described,  Karen  developed  the  strategy  of  referring  to  the  contents  of  the 
student  discussion  for  clues  as  to  what  the  current  topic  of  study  was. 

Use  of  Cognitive  and  Coping  Strategies 

Karen  used  this  strategy — referring  to  the  work  of  other  students — to  solve  problems  she  encountered  in  the 
online  environment,  for  both  organizational  and  educational  purposes.  Karen  scaffolded  her  own  comprehension  of 
assigned  texts  by  first  reading  the  comments  of  others  in  the  discussion  board  which  referred  to  these  assigned 
readings.  “I  would  usually  go  and  read  what  other  people  have  written  and  then  go  actually  do  my  reading  for  the 
week”  and  “I  usually  read  them  [other  people’s  comments  on  the  discussion  board]  so  I kinda  have  an  idea  of  what 
people  are  talking  about  before  I do  my  reading.”  This  strategy  provided  Karen  with  a meaningful  overview  of  the 
material  to  be  read.  The  focus  of  the  conversation  in  the  discussion  forum  alerted  Karen  to  the  nature  of  the  content 
of  the  material  to  be  read  and  to  what  her  fellow  students  found  most  noteworthy  as  a topic  of  discussion  within  the 
readings.  This  strategy  is  akin  to  the  notion  of  advance  organizers  postulated  by  Ausubel  et  al.  (1978).  As  Driscoll 
(1994)  summarizes,  “Advance  organizers  are  relevant  and  inclusive  introductory  materials,  provided  in  advance  of 
the  learning  materials”  (p.  126). 

Each  of  the  participants  in  this  pilot  study  mentioned  their  own  referral  to  other  student’s  work  as  a method 
of  learning  in  the  online  environment.  Ann  and  Karen  were  both  enrolled  in  theoretical  courses.  Ann  reported 
during  the  think-aloud  “If  I don’t  understand  the  question  from  over  here,  sometimes  I go  in  and  find  out  what  other 
people  have  written  about  it.”  Ann’s  strategy  is  similar  to  Karen’s;  that  is,  the  thoughts  of  classmates  are  used  to 
scaffold  reading  assignments  and  gain  clarification  on  the  topic  question  at  hand. 

Myriam  and  Jean  were  both  taking  design  courses.  They  would  look  at  other  student’s  design  ideas  to  help 
spark  their  own  creative  imaginations.  Myriam  remarked: 

I’m  learning  from  my  classmates  in  terms  of. ..when  I know  there  is  an  assignment  going  on  or  something, 

in  the  process  of  doing  it,  I check  what  they  [the  other  students]  have  been  doing  so  I learn  from  that  ‘oh 

wow  that’s  a good  idea.’ 
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Likewise,  Jean  mentions,  “I  want  to  look  at  other  students  work.  This  is  work  I’ve  not  done  yet.  I may  just  get  an 
idea.” 

It  is  possible  that  Karen  and  Ann  could  apply  their  strategies  in  a face-to-face  class;  they  could  listen  to  the 
class  discussion  of  the  readings  for  the  week  and  then  do  the  readings  subsequent  to  the  class.  However,  the  online 
environment  provides  a concrete  and  consistent  source  of  student  input.  Moreover,  it  is  highly  unlikely  that  Myriam 
and  Jean  could  have  access  to  fellow  student’s  designs  in  a face-to-face  class.  This  is  a unique  affordance  of  the 
online  course  for  Myriam  and  Jean. 

Another  strategy  that  was  deployed  by  all  of  the  participants  was  the  coping  strategy  of  printing  out  course 
materials  (such  as  lecture  notes  and  electronic  texts)  to  read  off-line.  The  commonly  used  cognitive  strategies  of 
highlighting,  underlining  or  otherwise  denoting  important  sections  of  a text  are  not  readily  available  in  a computer- 
based  environment.  Both  Jean  and  Ann  expressed  the  desire  for  a highlighting  tool  that  would  allow  them  to  take 
notes  in  an  electronic  text  the  way  they  normally  would  with  pen  and  paper.  According  to  Ann  “It’s  kind  of  strange 
reading  text  on  the  screen  as  opposed  to  reading  books.  It’s  harder  to  go  back  and  there’s  not  the  ability  to  highlight 
and  underline  and  figure  stuff  out,  which  is  what  I’m  used  to  doing.” 

These  actions  showed  that  students  are  coming  up  with  strategies  to  cope  with  the  unfamiliar  attributes  of 
this  new  learning  environment,  and  even  discovering  new  ways  to  learn  their  course  materials  as  a result.  Thus  it  is 
useful  to  differentiate  between  ‘learning  strategies’  (advanced  organizing),  and  ‘coping  strategies’  (printing  course 
readings)  as  two  observable  types  of  behavior.  The  former  can  be  described  as  a deliberate  action  that  contributes 
to  learning  the  course  content,  but  which  may  be  the  same  as  a strategy  used  in  face-to-face  courses.  The  latter,  a 
coping  strategy,  is  an  action  exhibited  by  the  participants  that  is  used  to  compensate  for  an  unfamiliar  and  unique 
attribute  of  the  online  experience.  In  many  cases  an  action  could  be  both  a coping  and  a learning  strategy,  or  more 
importantly,  the  coping  strategy  often  becomes  that  which  contributes  to  learning  the  course  content.  We  will  see 
other  examples  of  learning  and  coping  strategies  throughout  our  findings. 

Effects  of  Virtuality 

The  fact  that  one  is  not  able  to  utilize  well-known  cognitive  strategies  in  the  online  environment  is  a 
negative  effect  of  virtuality.  The  virtual  nature  of  the  online  course  gives  it  certain  characteristics  not  found  in  a 
face-to-face  class.  Freed  from  temporal  and  spatial  boundaries,  the  online  course  may  be  accessed  at  anytime,  from 
any  location.  Social  interaction  in  the  virtual  environment  is  largely  relegated  to  textual  exchanges  on  the  discussion 
board,  in  the  chat  room  or  through  email.  Body  language,  eye  contact  and  physical  appearance  are  communicative 
modes  not  available  in  the  online  course.  These  aspects  of  virtuality  have  an  effect  upon  the  student  experience.  The 
participants  reported  both  positive  and  negative  effects  of  virtuality.  The  synchronous  chat  was  Ann’s  favorite 
aspect  of  the  online  learning  experience.  Ann  emphasized  the  way  the  chat  format  allowed  her  to  have  more 
participation  in  the  class  due  to  the  suspension  of  the  “social  rules”  at  work  in  the  face-to-face  classroom. 

[T]here’s  something  about  getting  up  and  speaking  forth  your  idea  - which  you’re  not  even  sure  [of] 
because  you  are  still  formulating  - in  front  of  people  that  is  scary,  that  is  intimidating,  especially  if  you 
don’t  feel  like  you  know  the  people  in  the  class,  have  a good  rapport  with  everyone.  Whereas  the  first 
evening  of  the  chat  it  took  a little  bit  for  me  to  jump  in  there  and  put  forth  my  ideas,  but  it  was  fine  after 
that,  and  once  I felt  like  I had  a place  in  the  conversation,  it  was  a lot  easier  to  talk. 

Ann  found  the  “organic,  nonlinear”  nature  of  discussion  in  the  synchronous  chat  room  to  be  a compelling  learning 
activity: 


It’s  amazing  what  a different  kind  of  activity  the  learning  process  is,  because  you  are  constantly  having  all 
these  conversations  with  people  um  and  that  - you  have  to  jump  in,  you  have  to  participate  - you  can’t  just 
sit  back.  People  respond  to  a thread  and  then  another  thread  develops  from  it  and  then  its  more  organic,  it’s 
a lot  less  linear. 

Moreover,  the  dilemma  posed  by  ‘having  to  jump  in’  to  a synchronous  chat  turned  into  an  opportunity  for  Ann  to 
learn  the  content  better,  because  she  began  preparing  for  her  chat  by  reading  and  reviewing  her  course  materials  in 
detail  before  the  chat  session.  This  is  an  example  of  a coping  strategy  that  becomes  a learning  strategy  as  well. 

Interestingly,  it  is  the  lack  of  proximity  that  allowed  Ann  to  feel  more  confident  in  the  class.  She  felt  like 
she  got  to  know  the  students  in  the  online  class  in  a way  that  she  seldom  did  in  a face-to-face  class:  “You  have  a 
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sense  of  people  who  have  similar  ideas,  you  have  a sense  of  people,  what  people’s  interests  are,  I feel  probably  that  I 
know  people  in  the  class  better  than  I would  know  people  in  a regular  class.” 

However,  lack  of  proximity  was  also  the  major  negative  effect  ofvirtuality  cited  by  the  participants.  The 
feeling  of  isolation  and  the  lack  of  getting  to  know  the  Professor  and  the  other  students  in  the  class  was  a common 
issue.  Myriam  remarked:  “I  feel  that  I’m  like  maybe  too  much  by  myself  I don’t  like  that  part.  I don’t  feel  like  I 
have  that  much  feedback  and  I’m  sometimes  I feel  like  I am  lost.” 

Jean  also  mentioned  that  she  did  not  like  interacting  by  e-mail  and  that  she  missed  the  '‘spontaneous”  nature  of  face- 
to-face  communication.  “Personality  wise  I like  to  see  faces,  you  know.  I move  my  hands  when  I talk.  It’s  easier. 
You  are  more  spontaneous  when  you  interact  directly.  I think  it  is  difficult  to  interact  by  e-mail.”  Even  Ann  noted 
that  she  made  a point  to  go  by  the  Professor’s  physical  office  in  order  to  meet  him.  “I  actually  went  by  and 
introduced  myself  to  him  because  I thought  it  was  kind  of  strange  to  be  taking  the  course  without  ever  having  set 
eyes  on  him.” 

Karen  expressed  the  most  difficulty  with  the  effects  of  virtuality.  The  collapse  of  time  and  space 
boundaries  manifested  in  the  anytime,  anywhere  nature  of  the  online  class  was  a very  troubling  aspect  for  her.  She 
consistently  lamented  the  lack  of  face-to-face  elements  in  the  online  class.  Her  dissatisfaction  with  the  online 
experience  seemed  to  be  inversely  proportional  to  her  expectation  that  this  modality  would  be  similar  to  a traditional 
face-to-face  class.  In  both  the  think-aloud  section  of  the  session  and  the  interview  section,  Karen  implicitly  and 
explicitly  expressed  confusion,  frustration  and  difficulty  with  both  the  collapse  of  time  and  space  boundaries  and  the 
virtual  aspect  of  the  online  environment. 

Like  the  week  starts  on  Sunday,  so  these  classes  are  actually  seven  days  a week,  whereas  I'm  used  to 
thinking  of  school  between  like  Monday  and  Friday.  It's  like  this,  people  post  on  the  week-ends  so  it's  just 
even  more  work  it's  just : it's  like  its  ongoing  it's  never  complete. 

Karen  further  explicated  this  sense  of  feeling  overwhelmed  in  the  following  passage: 

[PJeople  add  new  ones  [comments  to  the  discussion  forum]  after  that  week  is  over,  which  is  confusing  for 
me.  Because  like  : ‘classroom  community’  was,  you  know,  three  weeks  ago  and:  there’s  47  new 
[comments  on  the  discussion  topic],  and  that  week  I read  them  all,  but  like  what  are  people  still  talking 
about? 

Karen’s  frustration  with  the  ongoing  nature  of  the  discussion  in  the  online  classroom  revolves  around  the  extra 
amount  of  work  entailed  in  keeping  up  with  several  discussions. 

It's  like  by  the  time  we  get  down  to  the  end  of  this  class  we're  gonna  have  to  read  everything  it's  gonna  take 
like  hours  to  go  through  and  read  everyone's  responses  and  how  responsible  are  we  supposed  to  be  for  all  of 
this?  In  a normal  class  you  just  go  each  week  and  the  topic  would  be  equity  and  you  only  have  to  be 
responsible  for  that. 

The  fact  that  students  can  access  the  conversation  at  any  time  during  the  week  and  the  fact  that  they  could 
still  contribute  to  the  conversation  after  the  class  had  moved  on  to  the  next  topic  is  a crucia  1 difference  in  the  online 
and  face-to-face  experiences.  The  opportunity  to  read  and  re-read  comments  in  the  student  discussion  board  creates 
a whole  new  learning  opportunity  for  students  in  the  virtual  classroom,  but  they  must  first  learn  to  abandon  their 
previous  temporal  and  spatial  notions  of  classroom  learning. 

In  the  interview  section  of  the  session,  when  asked  how  she  approached  learning  in  the  online  course, 

Karen  responded  “Well,  I try  to  fit  it  into  the  schema  that  I think  of  as  a normal  class.  And  it  doesn’t  really  seem  to 
fit.”  The  fact  that  Karen  came  to  the  online  class  with  the  traditional  face-to-face  class  schema  is  a deducible  one. 
Few  students  at  this  point  in  time  have  much  experience  with  online  education,  so  the  primary  conception  of  class  is 
the  traditional  face-to-face  mode. 

In  line  with  this  schema,  Karen  expresses  her  preference  for  the  proximal  pleasures  of  the  face-to-face 
experience.  “I  like  going  to  class,  I like  listening  to  discussions,  I like  being  in  the  room  having  that  moment  in  time 
with  a group  of  people  discussing  ideas.”  She  also  expresses  her  frustrations  with  the  virtual  nature  of  the  online 
class.  “I  feel  like  I'm  faking  the  class,  I don't  feel  like  I'm  really  taking  it.  I mean  I'm  doing  the  readings,  but  its  such 
an  isolated  experience.”  The  lack  of  proximity  appears  to  result  in  less  meaningful  learning  for  Karen: 
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Whereas  when,  online  I might  be  reading  twelve  comments  in  a row  and  so  your  click,  click,  click 
commenting.  I forget  what  the  other  people  said  and  it  kind  of  starts  to  blend  together.  Whereas,  when 
your  in  class  you  see  the  person  who  said  it  and  that’s  what  you  want  to  remember,  even  though  three  more 
comments  come  in  between  you  might  go  back  and  say  I remember  what  Susan  said  blah  blah  blah  blah 
blah.  Because  she’s  right  there,  its  in  the  moment,  its  an  interaction. 

It  is  possible  that  Karen  uses  visual  cues  to  encode  new  information.  In  a discussion  of  human  memory  and 
retrieval  Anderson  (1995)  explicates  various  contextual  cues  individuals  use  to  encode  information.  Karen  seems  to 
use  the  visual  cue  of  other  speakers’  physical  appearance  to  help  her  encode  information  in  a class.  The  comment 
above  implies  that  by  remembering  who  spoke,  Karen  also  remembers  what  it  is  the  person  said.  Therefore,  the  lack 
of  such  visual  cues  in  the  online  environment  could  prove  to  be  a true  impediment  to  cognitive  functioning  for 
Karen. 

Jean  also  sited  the  lack  of  visual  presence  and  identification  of  her  classmates  as  an  issue,  but  one  that  she 
was  able  to  cope  with  by  making  connections  between  students’  personal  profiles  and  their  course  participation.  For 
her  it  wasn’t  enough  to  read  only  the  names  of  those  participating  in  the  discussions,  or  what  they  had  to  say.  She 
needed  to  find  out  who  they  were,  what  they  did  for  a living,  and  construct  a personality  with  the  available  clues. 
These  included  not  only  the  personal  information  students  posted  about  themselves  online,  but  also  their  style  of 
writing  and  project  participation.  While  looking  at  her  classmate’s  projects,  Jean  remarked  “You  can  really  see  the 
personality  of  the  student,  the  way  I imagine  him”  and  “He  likes  to  be  funny,  this  Mark.” 

This  value  placed  on  getting  to  know  the  other  students  was  also  a coping  strategy  used  to  compensate  for 
the  lack  of  face-to-face  interaction  as  found  in  the  on-campus  course.  Jean  knows  that  there  are  other  people  who 
are  experiencing  the  course  and  working  on  the  assignments  just  like  she  is.  By  recalling  personal  facts  about  each 
classmate  as  she  reads  their  work,  Jean  is  able  to  synthesize  otherwise  disjointed  bits  of  information  into  a coherent 
set  of  content,  thus  her  coping  strategy  also  contributes  to  the  learning  process. 

Learning  Differences 

Karen  is  a student  in  a degree  program  at  Teachers  College,  and  she  comes  to  campus  regularly  for  courses. 
It  happened  that  this  course  wasn’t  offered  on  campus  this  semester,  so  she  had  to  take  it  online.  The  fact  that  she 
did  not  choose  the  online  environment  no  doubt  contributed  to  her  significantly  negative  attitude  towards  the 
experience,  but  she  also  described  herself  as  a social  learner,  and  a vocal  person  overall,  who  learns  best  in  groups. 
Karen  made  frequent  comments  about  her  preferred  way  of  learning  and  defined  herself  as  a “traditional  learner.” 
One  of  the  most  important  learning  outcomes  Karen  felt  she  experienced  from  her  online  class  was  the  appreciation 
and  empathy  she  developed  for  students  in  her  first  grade  class  who  have  differing  learning  styles.  “It  helped  me  to 
be  a better  teacher  in  that  I’m  more  understanding  of  my  students  who  have  learning  differences.” 

Karen’s  point  about  learning  differences  is  crucial  for  understanding  student  behavior  and  performance  in 
the  online  learning  environment.  Her  assessment  of  herself  as  a traditional  learner  may  have  inhibited  her  from 
taking  full  advantage  of  the  affordances  of  the  online  medium.  Throughout  the  think-aloud,  Karen  made  comments 
such  as  the  following.  “I  feel  like  there  is  probably  an  easier  way  to  be  doing  this  class  than  I’m  doing  it,  but  like  I’m 
always  doing  it  this  way,  where  I’m  not  quite  sure  what  we’re  supposed  to  be  reading  or  doing.”  However,  she  also 
found  ways  to  utilize  the  affordances  to  her  advantage,  such  as  the  aforementioned  strategy  of  reading  student 
comments  on  a topic  as  if  they  were  an  advance  organizer.  She  also  noted  that  the  ongoing  nature  of  the  student 
discourses  would  be  useful  if  “you  want  to  write  a paper  on  that  topic  or  something.” 

Had  Karen  been  able  to  adapt  her  learning  style  to  the  online  environment,  she  might  have  had  a very 
unique  learning  experience,  much  like  the  one  Ann  quite  unexpectedly  had  herself.  Ann  is  also  a student  in  a degree 
program  on  campus,  but  she  works  full  time  and  takes  only  one  other  class.  The  on  campus  section  of  the  course 
that  she  is  taking  online  was  offered  on  a different  day  than  the  other  course  she  took  on  campus,  and  so  rather  than 
coming  to  campus  two  days  per  week,  she  decided  to  enroll  in  the  online  section  of  the  course.  Therefore,  unlike 
Karen,  the  choice  to  take  an  on  line  course  was  a convenience,  and  so  she  may  have  entered  the  experience  with  a 
more  positive  attitude.  Whereas  Karen  relied  on  the  structure  of  the  in  class  course,  Ann  found  that  structure  too 
limiting,  too  linear.  “I  feel  when  you  are  in  a class  there  are  a lot  of  rules  that  people  follow,”  for  example,  one 
person  talks  at  a time,  students  don’t  interrupt  each  other,  they  follow  the  train  of  thought  that  the  last  person  said. 
Furthermore,  Ann  described  herself  as  shy  in  class,  and  she  felt  less  inhibited  online,  whereas  Karen  described 
herself  as  a very  vocal  person,  who  felt  she  couldn’t  be  heard  online. 

Ann  adapted  to  the  online  experience  in  many  ways.  For  example,  technologically,  she  found  innovative 
ways  to  work  with  multiple  browser  windows,  which  was  a coping  strategy  used  in  the  online  chats  so  that  she  could 
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refer  to  her  notes  and  the  Professor’s  Website  at  the  same  time.  Ann  was  particularly  pleased  with  the  online  chat 
feature  of  her  course,  which  proved  to  be  the  activity  that  transformed  her  learning  experience  and  her  beliefs  about 
learning.  Ann  reported  that  the  online  learning  class  “broke  me  out  of  what  I would  normally  say  learning  was  in 
the  regular  classroom...  I’m  kind  of  doing  a lot  of  it  independently.” 

Another  example  of  the  independent  nature  of  online  learning  is  the  asynchronous  discussion  board  feature 
of  her  class.  Ann  noted  that  it  was  challenging  to  be  on  her  own  and  required  to  write  short -answer  responses  to 
thought-provoking  questions:  "I  hadn't  had  anyone  to  talk  to  about  those  issues... it  was  me  doing  it  on  my  own 
...trying  to  come  up  with  links  between  pieces.”  Making  connections  is  something  that  was  normally  done  in  class, 
by  the  Professor,  or  by  other  students  asking  questions.  Ann  was  willing  to  take  risks  in  the  course  and  with  her  own 
participation,  which  may  have  made  all  the  difference  in  transforming  the  negative  feeling  of  isolation  into  a 
positive  feeling  of  independence. 

The  online  class  experience  created  many  potentially  frustrating  tensions  for  the  participants.  The  tensions 
arose  not  just  from  a lack  of  inexperience  with  the  online  medium,  but  also  from  the  whole  of  the  system  itself.  This 
assertion  is  evidenced  in  the  data  - poorly  laid  out  course  materials,  collapse  of  time  and  space  boundaries,  the 
virtual  nature  of  interaction,  isolation  --  all  of  these  elements  contributed  to  the  tensions  the  participants 
experienced. 

Activity  Theory 

Activity  theory  provides  a lens  through  which  we  may  view  the  whole  of  the  system  and  begin  to 
understand,  through  analysis,  the  causes  of  the  tensions  reported  in  the  findings.  This  analysis  also  makes  possible 
the  derivation  of  ideas  for  adjustments  to  the  system  that  will  result  in  the  easing  of  these  particular  tensions. 

Activity  theory  has  great  potential  for  the  refinement  and  efficacy  of  practice  in  any  system  — educational, 
organizational  or  social. 

Activity  theory  developed  from  the  work  of  Soviet  psychologists  Vygotsky  and  Leont’ev,  who  were  both 
greatly  influenced  by  the  philosophical  writings  of  Marx  and  Engels  (Engestrom,  1999).  Activity  theory  constitutes 
an  analytical  framework  that  stresses  the  importance  of  a cultural-historical  and  dialectical  approach  to  the 
understanding  of  human  activity  and  societal  or  systemic  change.  Vygotsky  (1978)  articulated  the  need  for  a new 
approach  to  understanding  human  activity  thusly: 

All  stimulus-response  methods  share  the  inadequacy  that  Engels  ascribes  to  naturalistic  approaches  to 
history.  Both  see  the  relation  between  human  behavior  and  nature  as  unidirectionally  reactive.  My 
collaborators  and  I,  however,  believe  that  human  behavior  comes  to  have  that  ‘transforming  reaction  on 
nature’  which  Engels  attributed  to  tools.  We  must,  then,  seek  methods  adequate  to  our  conception.  In 
conjunction  with  new  methods,  we  also  need  a new  analytic  framework  (p.61). 

These  ideas  were  expanded  upon  by  Leont’ev  (1978)  “...activity  is  not  a reaction  and  not  a totality  of  reactions  but  a 
system  that  has  structure,  its  own  internal  transitions  and  transformations,  its  own  development. ..In  all  of  its 
distinctness,  the  activity  of  the  human  individual  represents  a system  included  in  the  system  of  relationships  of 
society.”  (pp.50-51).  The  definition  of  human  activity  as  a multi-directional,  societally  embedded,  transformative 
system  stands  in  direct  contrast  to  the  western  notion  of  the  autonomous  individual,  which  is  perhaps  the  reason  why 
activity  theory  has  only  recently  gained  attention  in  the  west.  However,  in  the  last  two  decades  western  educational 
researchers  have  begun  to  utilize  activity  theory  as  an  analytical  framework  for  understanding  participant  activity  in 
educational  and  organizational  settings. 

Engestrom  has  led  the  way  in  adopting  an  activity  theoretical  approach  to  the  analysis  of  such  settings,  and 
has  contributed  to  the  further  development  of  the  theory  by  creating  a model  of  an  activity  system  that  can  be  readily 
applied  by  researchers.  According  to  Engestrom  and  Miettinen  (1999)  “Minimum  elements  of  this  system  include 
the  object,  subject,  mediating  artifacts  (signs  and  tools),  rules,  community,  and  division  of  labor”  (p.9).  The  last 
element  of  Engestrom’s  model  is  the  outcome  of  the  activity  system.  Barab  (2001)  notes  “Activity  theorists  are  not 
simply  concerned  with  doing  as  disembodied  action  but  are  referring  to  doing  to  transform  some  object,  with  a focus 
on  the  contextualized  activity  of  the  system  as  a whole”  (p.  2).  In  this  clarification,  the  outcome  is  the  transformed 
object.  Barab  (2001)  continues: 

By  subjects,  activity  theorists  are  referring  to  the  individuals  or  groups  whose  agency  is  selected  as  the 
point  of  view  for  the  analysis.  Objects  can  be  conceptual  understandings,  raw  materials,  or  even  problem 
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spaces  ‘at  which  the  activity  is  directed  and  which  is  molded  or  transformed  into  outcomes  with  the  help  of 
physical  and  symbolic,  external  and  internal  tools.’  (Engestrom,  1993,  p.  67)”  (p.  2). 

The  historicity  of  the  activity  system  is  an  important  concept  in  the  analytical  structure  of  activity 
theory.  As  Vygotsky  (1978)  notes  “The  psychological  development  of  humans  is  part  of  the  general  historical 
development  of  our  species  and  must  be  so  understood.”  (p.  60).  Engestrom  (1999)  defines  the  notion  of  historicity 
in  the  context  of  activity  theory  as  “identifying  the  past  cycles  of  the  activity  system.”  (p.  35). 

Three  principles  govern  the  interpretation  and  analysis  of  data  when  a researcher  takes  an  activity 
theoretical  perspective:  1)  the  activity  system  as  a whole  is  the  unit  of  analysis;  2)  the  history  of  the  activity  system 
must  be  taken  into  account;  and  3)  contradictions  within  the  activity  system  can  be  analyzed  “as  the  source  of 
disruption,  innovation,  change,  and  development  of  that  system,  including  its  individual  participants”  (Engestrom, 
1993  p.65).  The  analysis  of  the  participant’s  experience  in  the  online  class  is  based  on  these  three  principles. 

The  first  task  is  defining  the  elements  of  this  particular  activity  system.  For  the  purposes  of  this 
paper,  the  student  participant  is  the  subject  of  the  activity  system.  In  a different  analysis,  the  teacher  could  be  taken 
as  the  subject,  or  the  class  as  a whole  could  be  taken  as  the  subject.  This  shift  in  emphasis  would  necessarily 
produce  different  results;  how  significant  this  difference  would  be  is  unclear.  With  the  teacher  as  the  subject,  both 
the  object  and  the  outcome  would  change;  however,  the  rest  of  the  elements  of  the  system  would  remain  the  same. 
Therefore,  this  analysis,  while  significant  to  understanding  the  student’s  experience,  does  not  attempt  to  tell  the 
complete  story  of  the  class.  Figure  1.0  details  the  activity  system  of  the  online  class  that  Karen,  Ann,  Myriam  and 
Jean  participated  in  based  on  the  student  as  the  subject. 


Online  education  at  Teachers  College  has  existed  for  three  years.  The  courses  in  which  the  students 
participated  had  very  little  history  as  online  courses.  In  fact,  both  Karen  and  Ann  were  participating  in  courses  that 
were  given  as  an  online  course  for  the  first  time  (Spring,  2001).  Therefore,  there  is  no  direct  online  history  for  these 
particular  activity  systems;  rather,  the  history  is  significantly  linked  to  the  traditional  face-to-face  classroom.  The 
activity  system  for  online  education  is  quite  similar  to  a traditional  classroom  activity  system.  The  subject,  object, 
community  and  outcome  are  the  same  for  both.  It  is  important  to  note  that  the  historical  experience  of  both  the 
subject  and  the  community  in  the  online  classroom  is  the  face-to-face  classroom.  The  expectations  and  actions  of 
the  members  of  the  online  classroom  activity  system  have  been  conditioned  by  their  experience  in  the  traditional 
face-to-face  classroom. 

The  offering  of  these  courses  online  constitutes  an  expansion  of  the  definition  of  the  classroom  activity 
system  at  the  institutional  level.  A graduate  level  course  at  Teachers  College  is  no  longer  necessarily  geographically 
and  temporally  located.  This  expansion  represents  a historical  shift  at  the  institutional  level  with  great  consequences 
at  the  personal  level.  Engestrom  (1999)  discusses  the  significance  of  expansive  cycles  in  the  life  of  activity  systems. 


BEST  COPY  AVAILABLE  3*3 

S72 


‘“It  is  quite  natural  to  endeavor  to  represent  reproduction  as  cycles  resulting  in  the  formation  of  a new 
social  structure  on  the  basis  of  some  preceding  one’  (Schedrovitskii,  1988,  p.7;  italics  in  the  original).  Such 
an  irreversible  time  structure  may  be  called  mexpansive  cycle  (Engestrom,  1987). ..For  the  historical 
understanding  of  activity  systems,  expansive  cycles  are  of  crucial  importance”  (p.  33). 

The  new  social  structure  emerging  from  the  expansive  cycle  that  has  produced  the  online  course  is  occurring  at  all 
levels  of  society,  not  just  higher  education.  Castells  (1994)  defines  this  new  social  structure,  “The  control  over 
knowledge  and  information  decides  who  holds  power  in  society.  Technocrats  are  the  new  dominant  class”  (p.  41). 
Students  capable  of  technically  understanding  the  new  classroom  environment  have  an  advantage  over  those  who  do 
not. 

The  expansion  of  the  classroom  activity  system  from  face-to-face  to  virtual,  coupled  with  the  participant’s 
historically  based  expectations  combine  to  create  the  contradictions  and  tensions  felt  in  the  online  class  experience. 
Karen  reported  in  the  interview  section  that  prior  to  coming  to  Teachers  College  she  had  very  little  experience  with 
computers.  “Before  I came  to  Teachers  College,  I didn’t  even  have  e-mail  and  I didn’t  know  what  a web  address 
was.”  Her  expectation  of  the  online  class  was  that  it  would  be  similar  to  a “normal  class.”  Karen’s  description  of 
her  learning  experiences  in  the  traditional  face-to-face  class  was  deeply  grounded  in  the  physical,  geographic  and 
temporal  nature  of  that  experience.  The  lack  of  these  same  features  in  the  new  system  caused  a great  deal  of  distress 
for  her  and  possibly  impeded  her  intellectual  growth.  However,  it  is  important  to  note  that  Karen  found  solutions  to 
some  of  her  problems  through  referring  to  the  semiotic  and  concrete  tool  of  the  archived  student  discussions.  This 
archive  is  a new  feature  of  the  activ  ity  system  and  it  is  a potentially  rich  source  of  learning  within  the  new 
environment.  The  archive  of  the  student  discussion  represents  a new  educational  artifact  that  can  be  continually 
accessed  and  continually  created.  The  extemalization  and  dissemination  of  student  thought  is  a significant 
innovation  and  an  unparalleled  educational  tool  integral  to  the  new  activity  system  which  all  of  the  participants  took 
advantage  of 

The  design  flaws  that  frustrated  participants  in  the  new  activity  system  derive  fro  m the  inexperience  of  the 
teachers  in  the  new  environment.  In  two  of  the  courses,  the  teachers  had  never  previously  taught  an  online  class  and 
were  not  trained  in  the  instructional  design  of  educational  technology.  Therefore,  errors  in  web-site  architecture 
were  made,  including  dating  and  labeling  of  material  and  the  use  of  multiple  URLs,  which  caused  confusion  for 
participants.  The  division  of  labor  in  the  new  activity  system  calls  for  technological  knowledge  and  ability  for  all 
participants.  Previously,  content  knowledge,  administrative  knowledge  and  a small  amount  of  technological 
knowledge  (word  processor  and  copy  machine  functioning)  were  sufficient  for  efficacious  teaching.  In  the  new 
activity  system  we  must  add  to  this  list  the  specific  technological  skills  related  to  internet  based  communication 
(utilizing  ftp,  use  of  e-mail,  web  site  architecture,  understanding  of  hyper-linking,  use  of  communications  devices 
such  as  the  discussion  board  and  the  chat  room  to  name  the  most  basic  competencies);  and  teachers  must  give 
additional  thought  to  design  and  layout  of  course  materials  and  the  facilitation  of  student  participation. 

The  facilitation  of  student  participation  is  related  both  to  the  division  of  labor  and  to  rules  in  the  new 
activity  system.  The  rules  for  student  participation  revolved  around  posting  to  the  discussion  board.  “I  write 
something  just  so  that  I’m  getting  participation  credit,”  lamented  Karen.  The  new  division  of  labor  for  the  student 
includes  not  only  the  reading  assignments  and  the  essay  assignments,  but  also  weekly  written  participation  in  the 
discussion  board.  Karen  continues,  “sometimes  in  class  [face-to-face]  you  don’t  always  respond  [to  the 
discussion]. ..But  I feel  like  in  order  to  even  know,  for  someone  to  know  I was  there,  I need  to  [post  to  the  discussion 
board].”  This  tension  regarding  the  new  rule  of  participation  is  a frustration  for  Karen,  yet  it  is  also  a source  of 
learning  for  her.  Again  we  can  note  her  use  of  the  student  discussion  to  scaffold  learning  and  as  a potential  resource 
if  one  is  writing  a paper  directly  related  to  the  topic  of  discussion. 

Communicating  with  students  requires  attentiveness  not  only  to  the  discourse  on  the  discussion  board,  but 
also  to  individual  e-mails  sent  to  the  Professor.  Myriam’s  major  disappointment  in  the  online  class  was  the  lack  of 
interaction  with  the  teacher  and  the  TA;  “I  send  an  e-mail  and  I don’t  know  if  they  will  answer  or  not,  if  they  will 
reply  or  not  and  I regret  that  part.”  The  new  activity  system  requires  a different  type  of  time  commitment  from  the 
teacher.  Rather  than  holding  office  hours  and  a class  each  week,  the  teacher  in  the  online  environment  must  be 
ready  to  handle  student  questions  and  concerns  at  almost  anytime.  This  is  a daunting  obstacle  for  the  effective 
implementation  of  online  education. 

The  degree  of  student  agency  afforded  by  the  environment  represents  another  shift  in  the  division  of  labor 
and  rule  setting.  In  the  new  activity  system,  the  community  creates  rules  and  norms  through  their  actions. 

Therefore,  the  action  of  any  individual  within  the  community  has  authoritative  potential.  Those  community 
members  possessing  the  most  facility  and  ease  with  the  technological  environment  are  potential  rule  makers  and 
trendsetters  in  this  environment.  For  instance,  in  Karen’s  class  she  noted  that  students  seemed  to  post  new 
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comments  to  the  discussion  board  on  Sunday.  So,  she  made  a point  of  going  to  the  discussion  board  on  Monday  to 
read  the  new  posts.  Additionally,  Ann  noted  that  in  the  chat  room,  any  one  of  the  participants  could  initiate  a topic 
of  discussion;  it  is  not  dependent  on  the  Professor  or  the  TA.  While  this  level  of  agency  may  be  good  for  the 
community  as  a whole,  there  is  an  inherent  danger  of  alienation  for  the  less  technically  proficient  community 
members. 

This  danger  of  alienation  has  serious  implications  for  the  outcome  of  the  new  activity  system,  which  in  this 
case  is  the  comprehension  and  internalization  of  course  materials.  This  danger  inheres  for  students  not  only  from 
the  potential  of  the  empowerment/alienation  dynamic  brought  about  by  the  new  configuration  of  rule  making  and 
division  of  labor,  but  also  in  the  poor  layout  of  the  course  materials,  and  the  virtual  nature  of  the  tools.  Karen 
expresses  her  alienation  and  her  perception  of  its  effect  on  the  activity  system  outcome  “...it  feels  really  artificial  and 
it  feels  like  I’m. ..more  concerned,  like  I said  about,  just  what  I write  and  being  grammatically  correct  than  like  truly 
learning  and  getting  new  ideas  and  becoming  a better  teacher  which  was  my  whole  goal  of  coming  here.”  This 
negative  assessment  of  the  achievement  of  the  intended  outcome  by  Karen  was  counterbalanced  in  the  interview  by 
her  admission  that  the  experience  had  produced  a meaningful  unintended  outcome  for  her  as  a teacher.  She  states 
that  the  experience  has  made  her  a more  “empathetic  teacher.”  Through  her  own  reflection  on  the  difficulties  she 
had,  she  realized  that  different  learning  styles  contribute  to  student  success  and  satisfaction  in  a given  classroom 
experience;  she  is  bringing  this  new  understanding  back  to  her  first  grade  classroom. 

Discussion 

The  activity  theoretical  perspective  has  revealed  both  strengths  and  weaknesses  of  the  new  activity  system. 
Many  of  the  tensions  Karen  mentions  can  be  relieved  through  specific  adjustments  to  elements  of  the  system.  For 
instance,  the  inexperience  of  the  Professors  in  teaching  online  that  led  to  the  design  flaws  could  be  addressed  by  the 
University  through  a training  and  orientation  program  for  them.  Likewise,  orientation  sessions  for  the  students 
could  attenuate  the  prevalence  of  historically  based  classroom  expectations  and  facilitate  the  acquisition  of  technical 
knowledge  that  would  situate  each  student  as  a potential  leader  in  the  class. 

The  lack  of  visual  cues  to  aid  in  the  encoding  of  information  could  be  addressed  by  the  use  of  web  cams, 
video  teleconferencing  may  also  address  this  specific  issue.  The  negative  effects  of  the  virtual  nature  of  the  course 
(the  sense  of  isolation),  could  be  addressed  through  the  use  of  synchronous  chats  and  collaborative  assignments 
designed  to  facilitate  student  interaction.  These  adjustments  could  substantially  reduce  the  frustrations  that  may 
impede  learning  in  the  online  classroom. 

Activity  theory  as  a framework  for  understanding  student  behavior  and  experience  in  the  online  classroom 
is  extremely  useful.  The  complexity  of  Engestrom’s  model  allows  for  a thorough  view  of  factors  influencing  the 
subject.  The  emphasis  on  historical  perspective  situates  the  analysis  meaningfully  in  a broader  societal  context. 
Indeed,  participant  reports  of  their  online  experience  makes  most  sense  in  light  of  their  historically  based 
expectation  of  the  traditional  face-to-face  classroom.  The  participants  are  not  alone  in  the  holding  of  this 
expectation.  Society  at  large  is  undergoing  an  expansive  cycle  of  revolutionary  proportions  regarding  technological 
advances.  We  do  not  yet  understand  the  full  implications  of  these  developments.  Our  expectations  and 
prognostications  are  based  on  what  has  gone  before.  It  is  only  through  careful  investigation  of  these  emerging 
processes  that  we  will  be  able  to  understand  and  direct  their  significance  and  impact.  Activity  theory  is  a useful 
framework  for  such  an  effort. 

Implications  of  key  findings 

This  study  was  designed  to  provide  qualitative  information  about  the  experience  of  the  student  in  the  online 
course.  The  four  main  themes  that  emerged  from  the  transcripts,  web  site  design  flaws,  cognitive  and  coping 
strategies,  the  effects  of  virtuality  and  learning  differences  reveal  important  aspects  of  this  experience.  As  designers 
and  instructors,  we  need  to  pay  attention  to  these  aspects  and  use  them  as  a roadmap  to  improving  the  educational 
experience  of  the  online  student.  This  can  only  be  done  if  students  are  given  clear  expectations  about  the  course 
format  and  course  contents  at  the  outset.  For  example,  Karen’s  experience  of  ‘having’  to  go  back  and  re  -read 
messages  from  previous  weeks’  themes  could  have  been  turned  into  a learning  strategy  by  emphasizing  that  the 
online  environment  is  not  supposed  to  follow  the  same  schedule  as  the  campus  course.  That  the  lessons  are  never 
really  “done”  or  “complete”  in  an  online  environment  is  a new  concept,  but  it  should  be  one  that  is  exciting  and 
valuable  to  the  learner.  Students  should  also  be  aware  that  they  are  in  important  part  of  the  learning  process;  just  as 
they  might  eventually  depend  on  other  students  for  guidance  or  feedback,  so  those  students  depend  on  them  for  the 
same  reasons.  Adjusting  student  expectations  is  the  work  of  the  instructor.  Designers’  efforts  should  focus  on 
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alleviating  the  negative  effects  of  virtuality  and  in  creating  templates  that  directly  support  the  instructor  in 
developing  clear  layouts  for  the  presentation  and  pacing  of  course  materials. 

Methodological  considerations 

Although  we  have  many  enlightening  examples  of  how  students  behave  in  an  online  environment,  due  to 
the  diversity  of  courses  and  web  platforms  used,  it  is  possible  that  we  were  comparing  apples  with  oranges. 
Therefore,  the  direct  causal  relationships — if  there  indeed  were  any — between  inputs  such  as  student  disposition  and 
course  platform,  and  outputs  in  the  form  of  student  satisfaction  and  motivation  to  participate  were  unclear.  Further 
research  in  this  area  might  explore  some  of  these  causal  relationships  by  selecting  a more  homogenous  sample,  such 
as  students  who  participate  in  the  same  course,  or  students  who  are  not  currently  taking  on-campus  courses. 

Even  if  we  were  to  repeat  this  study  with  a similar  group  of  students,  we  would  try  to  get  more  personal 
information  from  the  students,  such  as  their  reasons  for  taking  an  online  course,  their  reasons  for  coming  to  Teachers 
College,  their  aspirations  and  plans.  Having  additional  information  about  their  values  and  beliefs  on  and  off  campus 
would  have  provided  us  with  more  factors  for  analysis  of  their  attitudes  and  behaviors  online.  Although  most  of  the 
participants  did  compare  some  aspects  of  their  behavior  online  to  their  behavior  in  class,  it  would  have  been 
interesting  to  compare  their  disposition  in  the  online  course  with  their  typical  disposition  in  class.  Additionally,  we 
might  have  asked  these  students  what  their  expectations  were  before  taking  the  online  course  and  if  these 
expectations  changed  at  all. 

This  research  project  raised  issues  of  importance  in  doing  qualitative  research  based  on  non-traditional 
learning  environments.  For  example,  how  does  one  observe  someone’s  participation  in  a course,  which  typically 
takes  place  in  isolation?  Using  the  technique  of  videotaping  and  verbal  protocol  analysis  may  have  been  an 
adequate  solution,  but  there  is  perhaps  a better  way  that  has  yet  to  be  discovered.  Additionally,  including  in  this  way 
students  who  are  located  remotely  would  have  been  costly  and  impractical. 

A final  issue,  that  is  not  new  to  online  learning,  is  how  to  actually  measure  learning  outcomes.  Karen,  Ann, 
Jean,  and  Myriam  all  had  unique  learning  experiences,  and  definitively  more  or  less  satisfying  experiences,  but  did 
any  one  student  actually  learn  more  or  less  than  the  others?  Did  they  perform  more  or  less  adequately?  We  can 
only  hypothesize  based  on  research  experience  with  classroom  learning  techniques,  but  a more  experimental, 
qualitative  approach  would  have  to  be  undertaken  in  order  to  say  with  certainty  that  different  strategies  or  course 
design  in  the  online  environment  really  contributed  to  improved  learning  outcomes. 
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Abstract 

The  digital  divide  is  a commonly  used  term  in  today's  society,  but  few  truly  understand  the  impact  that  it 
has  on  minority  students.  This  issue  is  one  that  goes  beyond  access  to  technology,  but  encompasses  issues 
of  equity  in  use.  This  article  will  examine  the  data  on  home  and  school  computer  use  by  minority  students. 
It  will  also  discuss  the  College  Reach -Out  Program  (CROP)  and  the  Community  Technology  Center  at 
Jacob  Riis  Neighborhood  Settlement  House,  which  are  two  programs  giving  minority  at-risk  students  non- 
traditional  experiences  with  technology. 

Introduction 

In  1 999,  the  Department  of  Commerce  published  a report  entitled,  Falling  through  the  Net:  Defining  the 
Digital  Divide,  in  which  they  found,  “providing  public  access  to  the  Internet  gives  certain  groups  the  opportunity  to 
advance  by  providing  them  with  technical  skills  which  are  needed  to  compete  in  the  digital  economy.”  Those  who 
have  access  to  technology  are  being  afforded  more  opportunities  than  ever  before,  but  one  must  also  think  about 
what  happens  to  those  being  left  behind.  There  is  a global  discussion  occurring  surrounding  the  issue  of  the  “haves 
and  have-nots.”  Most  research  on  the  subject  has  uncovered  what  has  come  to  be  known  as  the  “digital  divide”  or 
the  separation  between  those  who  have  access  to  and  can  effectively  use  technology  and  those  who  do  and  cannot. 

This  article  will  discuss  how  the  digital  divide  affects  minority  students  at  home  and  school.  Data  from  the 
Department  of  Commerce,  Educational  Testing  Service,  and  other  sources  show  that  this  population  of  students  is 
not  only  the  least  likely  to  have  access  to  computers  at  home,  but  also  the  least  likely  to  gain  access  at  school,  I 
discuss  one  program,  the  College  Reach-Out  Program  (CROP),  which  is  working  to  overcome  these  inequities  by 
exposing  minority  children  to  enriching  computer  activities.  It  is  my  opinion  that  alternative  access  opportunities 
must  be  created  for  these  students,  and  I will  review  the  work  I did  with  a group  of  minority  at-risk  students  and  the 
educational  program  they  were  involved  in.  The  project  the  students  and  I worked  on  gave  them  a different  view  of 
how  computers  fit  into  their  present  and  future  lives. 

Home  Computer  Ownership 

There  are  large  disparities  between  the  access  opportunities  of  the  rich  vs.  poor  and  ethnic  majority  vs. 
ethnic  minority  populations.  The  2000  report.  Falling  Through  the  Net:  Toward  Digital  Inclusion,  from  the  U.S. 
Department  of  Commerce  is  the  most  recent  data  available  on  home  computer  ownership.  It  is  evident  that  gains 
have  been  made  across  ethnic  groups  in  acquiring  home  computers.  As  can  be  seen  in  Figure  1,  all  groups  have 
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Figure  1,  Gains  in  Computer  Ownership  and  Internet  Access,  1 998^2000 

experienced  significant  increases  in  home  technology  ownership.  However,  what  is  not  immediately  evident  is  how 
this  growth  has  impacted  the  divide  in  access  opportunities  of  different  groups.  The  1998  gaps  between  Whites  and 
minority  groups,  with  regard  to  computer  ownership,  still  exist  today.  Between  African  Americans  and  Whites  there 
was  a 21.1  percentage  point  difference  from  in  1998  where  there  is  a 23.1-point  difference  today.  For  Hispanics,  the 
difference  was  23.4  in  1998  and  is  22.0  today.  Therefore,  even  though  all  groups  are  increasing  their  technology 
acquisition,  the  differences  in  access  rates  have  been  maintained. 
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Percentage  Point  Gap 

Figure  2.  Change  in  home  computer  ownership  gap,  1998-2000 

It  is  not  enough  for  a family  to  simply  own  a computer.  Connection  to  the  Internet  is  another  confounding  variable. 
Although,  as  with  computer  ownership,  Internet  access  rates  have  increased,  there  are  significantly  fewer  people 
who  have  access  to  the  Internet  than  computers  in  the  home.  (Figure  3)  This  is  not  only  true  for  the  African 
American  and  Hispanic  populations,  but  for  Whites  as  well.  Only  46.1%  of  Whites  have  Internet  access,  versus 
55.7%  that  own  a computer. 


White 
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Figure  3.  Computer  Ownership  and  Internet  Access,  2000 
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Unlike  computer  ownership,  the  gaps  between  Whites  and  minority  groups  regarding  Internet  access  have  not 
remained  stable,  but  widened.  In  1998  there  was  an  18.6  percentage  point  difference  between  African  Americans 
and  Whites  in  regard  to  Internet  access.  That  difference  has  increased  to  22.6  points.  This  increase  has  also 
occurred  between  Hispanics  and  Whites  going  from  a 17.2  percentage  point  difference  to  a 22.5-point  difference. 


Technology  Experiences  at  School 
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It  should  be  apparent  that  there  are  extreme  inequities  in  computer  ownership  and  Internet  access  in  the 
home.  Yet,  many  people  believe  that  creating  access  opportunities  at  schools,  libraries,  and  other  public  areas  will 
counteract  these  inequities.  However,  a look  at  the  data  on  computer  access  in  schools  shows  that  inequities  exist 
there  as  well.  A 1999  report  from  the  National  Center  for  Education  Statistics  reported  that  in  schools  with  a 
minority  population  greater  than  50%,  only  37%  of  the  instructional  rooms  have  computers.  This  is  compared  to 
57%  of  instructional  rooms  in  schools  with  a minority  population  less  than  6%. 

In  addition,  the  Educational  Testing  Service  found,  that  the  more  students  a school  has  belonging  to  a 
minority  or  low  socioeconomic  group,  the  higher  the  ratio  of  students  to  computers,  peaking  at  32  to  1 . This  is  more 
than  7 times  the  recommended  ratio  implying  that  poor  minority  students  lacking  access  to  computers  at  home  are 
also  not  being  given  equitable  access  at  school. 

The  inequities  associated  with  students  and  technology  do  not  end  there.  Let  us  suppose  we  are  in  a school 
that  has  overcome  the  last  two  inequities  and  has  a good  student  to  computer  ratio  for  their  students.  Problems  still 
exist.  Even  though  they  may  be  exposed  to  computers  on  a regular  basis,  the  types  of  activities  they  are  engaged  in 
often  times  deal  only  with  low-level  thinking  skills.  The  majority  of  the  students  from  the  CROP  program  reported 
to  me  that  they  used  computers  for  drill  and  practice  and  practicing  standardized  tests.  One  student  even  said,  when 
asked  how  she  uses  computers  in  schools,  “We  just  do  what  the  teacher  tells  us  to  do.” 

I do  not  believe  this  should  be  their  only  exposure  to  technology.  In  order  for  students  to  understand  how 
computers  fit  into  their  daily  and  future  lives  they  must  see  computers  integrated  into  their  education.  One 
experience  that  greatly  influenced  my  belief  in  using  technology  with  minority  at-risk  students  was  working  with 
CROP  2000.  This  program  is  detailed  below. 

The  College  Reach-Out  Program 

The  College  Reach-Out  Program  (CROP)  is  a collaborative  effort  of  three  Mid -Florida  institutions  of 
higher  education — Central  Florida  Community  College,  Santa  Fe  Community  College,  and  the  University  of 
Florida.  The  purposes  of  CROP  are  to  identify  and  recruit  economically  and  educationally  disadvantaged  students 
and  help  them  move  in  a direction  that  could  lead  them  to  college.  As  can  be  seen  in  Figure  4,  the  majority  of  the 
students  are  African  American. 


Caucasian  Hispanic 


Figure  4,  Ethnic  Background  of  CROP  Students 

CROP  is  a program  that  functions  throughout  the  year,  but  the  summer  is  a special  time  for  the  students. 
Every  summer,  the  rising  8^^  and  9^^  graders  spend  5 days  living  on  campus  at  the  University  of  Florida.  During  that 
time  they  live  in  the  dorms,  eat  in  the  cafeterias  and  interact  with  college  students  on  a personal  and  academic  level. 
Many  current  undergraduates  work  with  the  students  as  counselors,  and  are  not  only  employed  to  handle  disciplinary 
problems  but  to  also  give  the  students  an  opportunity  to  develop  one-on-one  relationships. 

For  their  summer  experience  these  students  had  the  opportunity  to  create  11/2  minute  movies  using  iMovie 
from  Apple  Computers  fwww.apple.com/imoviel.  The  students  were  encouraged  to  make  semi  -autobiographical 
movies,  but  many  chose  to  create  a story  instead.  The  only  requirement  was  that  they  make  something  they  could 
take  pride  in  presenting.  After  the  movies  were  completed,  they  were  transferred  to  individual  VHS  tapes  for  the 
children  to  take  home.  I believe  that  a good  way  to  understand  how  to  structure  new  activities  is  to  look  at  the 
triumphs  and  tragedies  of  those  who  have  done  it  in  the  past.  Therefore,  below  is  a summary  of  the  journal  that  I 
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kept  during  the  CROP  2000  summer  program.  I also  include  suggestions  for  those  wanting  to  create  their  own 
design  based  on  this  idea. 

CROP  2000  Technology  Project 

Although  I was  very  excited  about  what  we  would  be  doing  I wasn’t  sure  if  they  would  be  as  excited,  but 
the  second  I told  them  they  were  going  to  be  making  movies  I could  see  the  excitement  on  their  faces.  The  original 
plan  for  the  students  was  for  them  to  bring  a bunch  of  pictures  of  themselves  so  that  they  could  have  a variety  of 
images  in  the  movie,  but  none  of  the  kids  brought  pictures.  It  may  be  useful  for  those  who  want  to  use  photographs 
in  projects  to  have  the  students  bring  in  the  photos  days  before  the  project  is  scheduled  to  begin.  This  will  avoid  the 
problem  of  students  not  having  pictures  to  use  and  having  to  spend  time  looking  for  images. 

I talked  to  them  about  what  the  movies  were  going  to  be  about  and  told  them  what  the  different  roles 
(movie  director,  art  director,  musical  director,  and  narrator)  entailed.  Once  they  picked  their  roles,  I brought 
everyone  back  together  in  order  to  explain  more  in  depth  what  they  would  be  doing  for  the  remainder  of  the  day. 
Handouts  were  distributed  to  explain  the  roles,  but  as  is  true  when  teaching  anything,  it  is  best  to  orally 
communicate  your  ideas  as  well  as  putting  them  on  paper. 

At  that  time  they  were  free  to  work  for  the  remainder  of  the  hour.  Most  of  the  groups  spent  time  talking 
about  what  they  were  going  to  do  because  they  wanted  to  be  able  to  structure  their  pictures  around  a theme.  I spent 
my  time  walking  around  and  talking  to  the  students,  and  from  the  questions  they  asked  I it  seemed  that  the  hardest 
part  for  most  groups  was  developing  a theme.  It  was  refreshing  to  teach  a computer  class  where  the  hardest  part  for 
the  students  was  not  using  the  computers.  It  became  a tool  for  accomplishing  a task  instead  of  the  task  itself. 

1 was  naive  about  how  smoothly  the  project  would  go  and  it  slapped  me  in  the  face  on  the  second  day. 

They  came  with  just  as  much  energy  as  on  the  first  day  but  seemed  to  get  more  hyperactive  as  the  morning  went  on. 

Since,  as  I mentioned  before,  they  did  not  bring  any  photographs,  they  were  to  spend  10  minutes  looking  up 
pictures  on  the  Internet  of  someone  they  admire.  A problem,  however,  was  that  they  did  not  know  how  to  search  the 
Internet.  Considering  that  the  majority  of  these  students  reported  owning  a home  computer  (71%)  and  using  the 
Internet  at  school  (55%)  they  did  not  know  how  to  search  the  Internet.  Instead  there  was  a lot  of  “I  can’t  find 
anything”,  and  “Where  can  1 find  pictures?”  Most  teachers  will  not  be  working  with  students  that  they  have  never 
seen  before,  so  they  can  more  easily  gauge  student  expertise.  However,  for  those  working  with  new  students,  I 
would  suggest  assuming  they  don’t  know  anything.  That  way,  if  they  do  have  knowledge  coming  in,  it  will  only 
enhance  the  project  as  opposed  to  hindering  it. 

The  last  day  was  crunch  time.  Instead  of  allowing  the  kids  to  come  into  the  computer  lab  as  I usually  did,  I 
met  them  outside  to  give  explicit  instructions  for  what  they  needed  to  work  on,  but  even  with  these  instructions  all  of 
the  groups  stayed  an  extra  hour  to  finish  their  work.  After  they  finished,  we  watched  the  videos  and  they  were  all 
very  proud  of  what  they  had  accomplished. 

Student  Feedback 

As  can  be  seen  above,  the  program  was  not  easy  to  organize  and  implement.  Nevertheless,  what  I hope  you 
get  from  this  is  that  it  can  be  done,  and  is  something  students  will  enjoy.  I asked  the  students  to  write  comments 
about  their  experiences  working  on  this  project.  Some  of  the  comments  I received  were: 

• “I  never  knew  how  fun  it  could  be  making  a movie  on  the  computer.” 

• “It  can  be  kind  of  hard,  but  when  you  put  your  mind  into  it  you  can  do  it.” 

• “I  didn’t  know  you  could  take  pictures  and  put  them  on  the  computer.” 

• “I  got  to  look  at  things  on  the  computer  that  I usually  don’t  look  at.” 

• “I  didn’t  know  there  was  so  much  excitement.” 

These  types  of  responses  cannot  be  elicited  when  students  are  engaged  in  technology  use  that  merely  reduces  them 
to  passive  clickers. 

Jacob  Riis  Neighborhood  Settlement  House 

Community  Technology  Centers  are  one  of  the  ways  that  communities  nationwide  are  working  to  bridge 
the  digital  divide.  These  centers  allow  residents  of  communities  to  have  access  to  technology  as  well  as  take  classes 
to  improve  their  technical  skills.  This  community  center  is  located  in  the  Queensbridge  Public  Houses,  home  to 
12,000  residents,  5000  of  which  are  under  the  age  of  18.  The  Community  Technology  Center  within  offers 
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programming  for  all  residents  of  the  community  ranging  from  an  after-school  program  to  evening  adult  classes.  My 
current  position  is  as  the  Community  Technology  Center  coordinator  for  the  Jacob  Riis  Neighborhood  Settlement 
House.  In  that  position,  I am  responsible  for  developing,  staffing,  and  scheduling  all  technology  programs  offered  at 
Jacob  Riis.  One  exciting  program  that  works  with  children  is  the  Our  kids  After-School  Program. 

The  Our  kids  program  is  designed  for  students  ranging  in  age  from  5 to  10.  All  participants  come  to  a 
technology  class  twice  per  week  where  they  are  engaged  in  alternative  technology  experiences.  After-school 
programs  are  a great  place  to  try  innovative  and  exciting  technology  projects,  and  one  project  we  are  trying  with  our 
students  uses  the  program  Microworlds.  Microworlds  is  an  application  that  was  developed  to  teach  young  children 
programming  concepts  in  a fun  and  exciting  way.  The  interface  for  Microworlds  is  kid-friendly  and  allows  children 
to  not  only  program  but  also  draw  and  create  imaginative  projects  as  well. 

One  example  of  such  a project  is  an  autobiography.  Our  students  are  creating  an  interactive  program  in 
Microworlds  that  will  tell  their  stories.  In  contrast  to  a traditional  autobiography  that  would  solely  be  made  up  of 
writing,  our  students  are  drawing  pictures  and  weaving  those  pictures  into  their  stories.  This  allows  them  to  develop 
their  literacy,  technical,  and  creative  skills  in  one  setting  that  is  fun. 

Implications  and  Suggestions 

From  reading  the  information  that  has  been  published  about  the  digital  divide  and  from  my  experiences 
with  the  CROP  program  and  Jacob  Riis , it  has  become  abundantly  clear  that  we  are  entering  a time  in  our  society 
where  there  is  a new  type  of  inequality.  Computers  give  people  access  to  more  information  than  at  any  other  time  in 
our  history.  However,  a large  part  of  the  population  is  being  left  behind. 

No  matter  how  you  look  at  the  data,  poor  and  minority  students  are  at  a disadvantage  concerning  access  to 
new  technologies.  The  argument  I have  heard  many  times  is  that  we  cannot  expect  families  who  are  struggling  to 
survive  to  be  able  to  afford  a computer.  However,  what  we  should  be  more  concerned  with  is  how  to  engage  these 
families  through  the  use  of  technology. 

We  also  must  think  about  how  we  are  using  technology  with  minority  students.  When  I met  the  CROP 
students,  1 discussed  some  of  the  things  they  were  using  computers  for  at  school.  The  responses  I received  were 
quite  disheartening.  The  majority  of  these  students  stated  that  they  used  the  computer  to  play  games  or  take  tests. 
Their  home  use  was  not  helping  them  to  excel  either.  Most  of  the  students  in  the  CROP  program  own  computers 
(95%),  but  none  of  them  knew  how  to  search  for  pictures  on  the  Internet.  Students  who  do  not  have  access  to  high- 
quality  computer  experiences  at  home  or  school  are  not  being  provided  with  the  opportunities  they  need  to  be 
successful  in  society.  Is  this  the  way  we  want  to  prepare  students  for  the  future? 

This  issue  is  far  from  being  solved.  There  are  many  factors  that  must  be  taken  into  consideration  when 
discussing  the  impact  of  a new  technology  on  society,  and  computers  are  not  any  different.  There  are  always  those 
for  whom  it  is  a benefit  and  those  for  whom  it  is  a detriment.  The  first  step  in  bridging  the  gap  is  to  realize  that 
technology  has  not  benefited  everyone  equally,  and  begin  to  work  on  ways  to  level  the  playing  field.  One  possible 
solution  is  to  expose  students  to  innovative  projects,  such  as  the  ones  for  the  CROP  program  and  Jacob  Riis,  which 
allow  them  to  expand  their  views  about  how  computers  fit  into  their  lives  now  and  in  the  future. 
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Introduction 


The  accessibility  and  affordability  of  the  computer  and  the  rapid  expansion  of  the  Internet,  has  put 
information  literally  at  our  fingertips.  We  are  now  able  to  procure  volumes  of  data  with  just  the  stroke  of  a key.  The 
field  of  technology  is  rapidly  accelerating  to  allow  for  commonplace  interactive  audio,  video  and  computer 
conferencing.  Possibilities  for  communicating  and  exchanging  information  seem  limited  only  by  our  imaginations. 

Therefore,  in  order  to  prepare  students  for  this  swiftly  changing  Age  of  Information,  the  use  of  technology 
in  education  should  become  a critical  factor.  It  is  no  longer  enough  that  instruction  flows  from  teacher  to  student, 
nor  will  it  be  enough  to  only  expose  students  to  technology;  students  must  learn  to  become  active  participants  in  the 
process  of  their  education.  We  need  to  recognize  that  social,  political  and  economic  changes  are  occurring  and 
forcing  our  society  to  offer  alternatives  to  traditional  education.  Research  over  the  past  25  years  has  shown  that 
interaction  involving  a purposeful  cognitive  approach  by  the  student  is  an  essential  component  in  the  learning 
process.  Therefore,  education  courses  need  to  encourage  students  to  be  self-reflective,  self-corrective  and  self- 
constructive  learners.  Specifically,  technology  in  education,  with  its  "instant”  availability,  but  more  importantly, 
with  its  ability  to  allow  learner  control,  is  a viable  solution  to  this  need. 

Background  Information 

The  nature  of  the  Information/Communication  Age  in  which  we  live  demands  that  we  prepare  the  students 
of  today  for  the  challenges  of  tomorrow.  Change  is  occurring  faster  everyday  and  students  and  society  are  left 
economically  and  socially  at  risk  (Lane  & Cassidy,  1997).  As  a result,  students  will  need  to  not  only  be  prepared, 
but  empowered.  In  order  to  increase  global  awareness,  keep  pace  with  and  implement  the  rapidly  changing 
technology,  and  adapt  to  the  changes  in  the  work  force,  schools  need  to  adjust  to  meet  these  demands. 

Traditionally,  the  teacher  has  been  at  the  heart  of  the  educational  system  as  the  deliverer  of  knowledge  and 
the  center  of  learning.  However,  with  the  onslaught  of  available  information,  teachers  are  no  longer  able  to  know 
everything  of  value  to  teach  students,  but  are  now  learners  along  with  their  students.  No  longer  isolated  work  units 
within  a classroom,  the  teacher  of  today  needs  to  facilitate  the  learning  of  the  students  and  empower  them  to  explore 
and  learn  by  accessing  the  additional  expert  resources  in  the  community  and  throughout  the  world  (Guerra  & 
Alvarez-Buylla,  1995).  Methods  and  strategies  involved  in  facilitating  this  process  need  to  include  guided  practice, 
inquiry  learning,  and  teamwork.  Interaction  must  be  a part  of  the  learning  process;  however,  the  teacher  may  or  may 
not  be  physically  present. 

Since  educational  technology,  when  properly  used,  forces  the  role  of  the  teacher  to  shift,  it  is  a natural 
conclusion  that  the  role  of  the  student  must  also  change.  For  example,  research  shows  that  learners  presently 
enrolled  in  distance  education  courses  are  more  likely  to  be  active  listeners  and  are  able  to  work  independently 
(Trier,  1996).  While  motivation  is  a key  factor,  it  can  be  increased  if  students  feel  that  there  is  a certain  relevance  to 
the  medium  and  the  message  and  if  they  are  given  the  opportunity  to  create  a meaningful  project  from  the  material 
covered.  Students  will  be  given  more  initiative  and  have  more  control  over  what  they  learn.  Behaviors  expected  of 
students  will  include  being  self-directed,  responsible,  and  proactive. 

In  their  research,  Roblyer,  Edwards  and  Havriluk  (1997)  note  that  students  realize  several  unforeseen 
benefits.  Slower  students  blossomed,  less  popular  students  were  sought  out  for  advice  and  assistance,  and  formerly 
unmotivated  students  became  excited  and  involved.  Teachers  of  these  students  observed  improvements  in  student 
academic  performance,  self-esteem,  and  increased  acceptance  of  responsibility  (Roblyer,  Edwards  & Havriluk, 
1997). 

However,  when  any  classroom  is  transformed  because  of  the  appropriate  use  of  technology,  it  is  to  be 
expected  that  students  need  an  adjustment  period.  Changes  in  the  approaches  to  teaching  and  to  learning  will  result 
in  cognitive  dissonance.  Students  will  no  longer  be  able  to  expect  traditional  behaviors  from  teachers,  while 
simultaneously,  they  are  readjusting  to  their  new  responsibilities. 


394 


Further,  the  role  of  the  school  will  have  to  shift  in  order  to  more  accurately  reflect  the  work  environment. 
Work  environments,  and  the  world  in  general,  are  increasingly  becoming  global  in  their  operations.  Contemporary 
society  is  faced  not  only  with  the  problem  of  promoting  the  expansion  of  knowledge,  but  also  of  generating  a 
workforce  which  is  capable  of  adjusting  to  the  Information  Age.  Students  will  need  to  learn  more  effectively  and 
efficiently  than  ever  before  because  of  the  rapid  growth  of  information  and  because  of  the  escalation  of  knowledge 
and  skill  requirements  for  most  Jobs  (Wellbum,  1996). 

The  Contributions  of  Technology  to  Education 

Before  technology  will  be  adopted  into  the  educational  process,  teachers  must  first  be  convinced  that  there 
is  some  advantage  to  using  technology.  Findings  from  the  Apple  Classrooms  of  Tomorrow  (ACOT)  research 
suggest  that  students’  behaviors  and  attendance  improved,  along  with  attitude  toward  themselves  and  toward 
learning  (Ringstaff,  Yocam,  & Marsh,  1996).  Improved  student  performance  was  also  noted  in  several  ways:  (1)  test 
scores  indicated  that,  despite  time  spent  learning  to  use  technology,  students  were  performing  well,  and  some  were 
clearly  performing  better,  (2)  the  students  wrote  more,  more  effectively,  and  with  greater  fluidity,  and  (3)  some 
classes  finished  whole  units  of  study  far  more  quickly  than  in  past  years  (Ringstaff,  et  al,  1996).  Other  unintended 
outcomes  were  noted  which  included  students  becoming  socially  more  aware  and  more  confident,  students  working 
well  collaboratively,  and  students  exploring  and  representing  information  dynamically  and  in  many  forms 
(Ringstaff,  et  al,  1996). 

Educational  technology  makes  it  possible  to  create  learning  situations  in  which  students  can  be  engaged  in 
activities  that  they  find  interesting  and  exciting  for  their  own  reasons  and  which  accomplish  the  educational  goals  of 
teachers  (Riel,  1989).  Teachers  can  plan  various  activities  simultaneously,  and  students  can  learn  in  an  interactive, 
workshop-style  format  (Piper,  2000).  Computers  give  teachers  the  opportunity  to  expand  the  boundaries  of  the 
classroom  (Facemyer  & Peterson,  1996,  as  cited  in  Piper,  2000)  by  allowing  instant  access  to  information  and 
specialized  expertise.  In  addition,  educational  technology  can  create  new  avenues  for  social  exchange  and 
cooperative  learning  (Riel,  1989). 

Barriers  to  Change 

There  are  many  factors  responsible  for  the  non-adoption  of  educational  technology  among  teachers.  The 
National  Center  for  Education  Statistics  (NCES,  2000)  reports  that  in  1999,  one  of  the  barriers  most  frequently 
reported  by  public  school  teachers  was  the  lack  of  computers.  If  teachers  are  expected  to  use  computers  in  the 
classroom,  it  is  only  logical  to  assume  that  computers  will  be  provided.  However,  access  alone  doesn’t  ensure 
adoption  of  technology.  Location,  access,  and  current,  compatible  software  play  a part  in  determining  how  they  will 
be  used  (Sheingold,  1991).  Moreover,  teachers  at  schools  with  minority  students  accounting  for  more  than  50  per 
cent  of  enrollment  were  more  likely  to  cite  outdated,  unreliable,  or  incompatible  computers  as  barriers  to  use 
(NCES,  2000). 

Even  if  the  problem  of  access  is  solved,  teachers  must  still  be  taught  to  help  their  students  effectively  use 
Internet  resources  for  learning  purposes.  Evaluating  web-based  resources  is  an  essential  skill  because  the  Internet, 
besides  hosting  a wealth  of  valuable  educational  resources,  is  also  the  single  largest  source  of  misinformation  in  our 
society  (Maney,  1999). 

Another  barrier  reported  frequently  by  public  school  teachers  was  the  lack  of  release  time  for  teachers  to 
learn  how  to  use  computers  or  the  Internet  ^CES,  2000).  Franklin  (1999)  reports  that  time  was  the  most  often 
recorded  item  on  the  list  of  barriers  that  remain  in  place  to  prevent  technology  adoption.  She  further  states  that  “this 
coincides  with  the  President’s  Committee  of  Advisors  on  Science  and  Technology,  which  found  teachers  did  not 
have  enough  time  in  the  day  to  develop  new  lesson  plans  for  the  implementation  of  technology  when  technology 
was  newly  placed  in  a school”  (Franklin,  1999).  Chiero’s  (1997)  study  on  Teachers'  Perceptions  On  Factors 
Affecting  Computer  Use  lists  lack  of  time  as  the  single  most  important  barrier  to  technology  adoption.  Teachers  must 
be  allowed  adequate  time  to  learn  new  technologies  (Maney,  1999).  A lack  of  time  constrains  teachers  from  thinking 
about  new  ways  of  organizing  learning  in  their  classrooms  by  the  need  to  handle  day-to-day  issues,  surprises,  crises 
and  challenges  (Fullan,  1999).  Among  other  things,  this  daily  press  for  time  makes  teachers  dependent  upon  what 
they  already  know  and  prone  to  following  routines  (McKenzie,  1999). 

An  additional  barrier  reported  by  teachers  is  lack  of  time  in  the  schedule  for  students  to  use  computers  in 
class  (NCES,  2000).  Traditional  40  or  50 -minute  class  periods  do  not  allow  sufficient  time  for  students  to  be 
engaged  in  learning  through  use  of  technology. 
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If  teachers  are  going  to  adopt  and  use  technology,  there  must  be  a support  infrastructure  available  to  help 
them  troubleshoot  and  solve  technology -related  problems  (Maney,  1999).  This  includes  administrative  support, 
technical  support  or  advice,  and  support  regarding  ways  to  integrate  telecommunications  into  the  curriculum  (NCES, 
2000). 

Other  barriers  to  technology  use  as  reported  by  the  National  Center  on  Education  and  Statistics  (2000)  from 
teachers’  survey  responses  included  concern  about  student  access  to  inappropriate  materials  and  inadequate  training 
opportunities.  Chiero’s  (1997)  research  confirmed  that  lack  of  adequate  training  is  a resource  frequently  mentioned 
as  a major  barrier  to  computer  use.  Lack  of  training  was  the  second  most  often  reported  barrier  in  the  study  done  by 
Franklin  (1999).  The  need  for  professional  development  continues  to  be  a concern  for  public  schools. 

The  Adoption  Process 

The  adoption  of  technology  with  instruction  presents  a challenge  to  those  involved  in  the  change  business. 
Fullan  (1999)  describes  the  process  of  change  as  being  complex,  dynamic,  and  unpredictable.  Senge  (1990)  defines 
the  building  process  as  the  capacity  to  hold  a shared  common  picture  of  the  future  we  try  to  build  or  seek  to  create  A 
successful  change  agent  must  be  comfortable  dealing  with  ambiguities  and  with  failure  while  remaining  focused  on 
the  goal. 

"Situated'*  Change:  Failure  often  occurs  because  reform  is  often  packaged  as  a “one  size  fits  all”  deal. 
Miller  ( 1 996)  defineis  a situated  notion  of  school  reform  as  a reform  that  “conceptualizes  restructuring  of 
pedagogies,  curricula,  and  school  organizations  as  changing  in  purpose  and  form  across  differing  educational 
settings  and  circumstances"  (Miller,  1996).  She  stresses  that  the  difficulties  and  divisiveness  that  often  arise  occur 
because  reform  efforts  are  generalized  to  all  settings,  rather  than  situated  to  specific  setting. 

In  order  to  allow  change  to  be  situated,  the  process  of  adoption  should  first  begin  with  the  knowledge  stage, 
as  Rogers  (1995)  calls  the  initial  stage  of  the  adoption  process.  Crucial  information  needs  to  be  communicated  by 
the  change  agent  to  the  educational  system.  Specifically,  those  in  this  field  need  to  be  aware  that  the  use  of 
technology  in  education  is  more  than  just  a technological  system  or  a tool.  Instructional  technology  allows  for  its 
innovative  capabilities  to  interact  with  the  creative  talents  of  its  participants  (Chute,  Thompson  & Starin,  1996).  This 
informatbn  must  be  successfully  disseminated  to  the  users  by  the  change  agent  if  the  goal  of  integration  can  occur. 
The  challenge  of  integrating  technology  into  schools  and  classrooms  is  really  much  more  human  than  it  is 
technological  (U.S.  Office  of  Technology  Assessment,  1995). 

Educators  must  clearly  understand  that  technology  changes  the  context  in  which  education  takes  place 
(Lane  8l  Cassidy,  1997).  Teacher  roles,  student  roles,  school  structures,  and  related  issues  all  need  to  be  defined 
prior  to  implementation.  However,  in  order  for  any  school  reform  to  be  successful,  a goal  is  necessary,  and  because 
of  the  nature  of  the  school,  that  goal  needs  to  be  directly  related  to  observable  student  performance.  It  is  important  to 
remember,  however,  that  while  the  objective  of  change  is  the  same,  the  process  will  be  different  for  each  situation. 

Developing  a Vision:  The  next  step  toward  adopting  educational  technology  is  developing  a vision  of  how 
technology  should  influence  what  educators  do  (Costello,  1999).  The  educational  staff  must  contribute  to  the  vision. 
A team  approach  creates  ownership,  which  in  turn  will  promote  future  implementation  of  the  plan.  Ownership  of  the 
process  of  change  is  a powerful  motivational  strategy. 

The  key  to  effective  adoption  is  continued  dialogue  throughout  the  process  and  as  newer  technologies 
become  available.  The  premise  that  we  are  lifelong  learners  is  a fundamental  principle  of  distance  education. 
Educators  need  to  realistically  evaluate  the  educational  system  with  this  in  mind.  While  creating  the  vision  is  the 
second  step,  the  way  in  which  to  continually  promote  change  and  growth  is  by  maintaining  and  allowing  for 
modification  of  the  vision. 

Wellburn  (1996)  states  that  schools  which  spend  time  creating  an  instructional  vision  based  on  instructional 
goals  and  a shared  philosophy  have  been  most  successful  in  adopting  educational  technology.  His  findings  illustrate 
Rogers  (1995)  authoritative  type  of  Innovation-Decisions.  First,  Wellburn  (1996)  supports  the  fact  that  those  in 
authority  need  to  make  the  decision  to  adopt  the  innovation.  Once  the  decision  has  been  made,  those  whom  it  affects 
are  then  given  the  authority  to  determine  how  best  to  implement. 

Staff  Development:  In  order  for  the  educational  system  to  survive,  it  must  maintain  a competitive  edge 
(Senge,  1 990).  Historically  as  well  as  presently,  our  schools  have  been  slow  in  their  ability  to  learn  and  to  adjust  to 
change.  The  evidence  is  seen  in  the  rapid  acceleration  of  the  computer  industry  and  the  relatively  slow  rate  of 
adoption  of  its  technologies  into  the  classroom 

Teachers  need  training  and  they  need  to  be  involved  in  the  integration  process.  When  adults  see  themselves 
as  the  locus  of  causality  for  their  learning,  they  are  much  more  likely  to  be  intrinsically  and  positively  motivated  to 
change  (Wlodkowski,  1993). 
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According  to  McKenzie  (1998)  and  the  U.S.  Office  of  Technology  Assessment  (1995),  technology  training 
is  most  effective  if  presented  to  teachers  in  a ‘‘just  in  time”  fashion.  Teachers  will  learn  as  they  need  to  know. 
Moreover,  Gilmore  (1995)  stresses  that  staff  development  should  take  place  in  the  school.  An  advantage  to  this 
arrangement  is  that  learning  occurs  in  the  natural  environment.  Teachers  will  be  trained  with  the  actual  hardware 
and  software  that  they  are  expected  to  use. 

In  addition,  learning  along  with  one’s  peers  creates  a cooperative  atmosphere.  This  promotes  more 
exchange  of  information,  more  helping  and  sharing  of  resources  among  members,  more  peer  influence  toward 
productivity,  higher  incidence  of  creative  and  risk-taking  thinking,  higher  emotional  involvement  in  and 
commitment  to  productivity  by  more  members,  higher  acceptance  and  support  among  members,  more  of  a problem- 
solving orientation  to  conflict  management,  and  a lower  fear  of  failure  by  members  (Wlodkowski,  1993).  Change 
will  occur  most  quickly  in  environments  where  innovation  and  collegial  sharing  are  operating  simultaneously,  each 
promoting  the  other  (Sandholtz,  Ringstaff  & Dwyer,  1997). 

Certain  factors  are  needed  to  promote  the  change.  Specifically,  due  to  the  perceived  complexity  of  the  any 
technology,  an  agent  should  identify  needs  and  provide  for  leadership  and  support  for  the  innovation  with  the 
teachers  expected  to  adopt  (Hutton,  1994).  The  change  agent  should  also  encourage  partnerships  to  develop  which 
would  foster  the  adoption.  This  process  involves  identifying  the  categories  of  adopters,  which  in  this  case  would  be 
identifying  those  with  varying  degrees  of  computer  experience. 

Technical  Needs:  McKenzie  (1998)  emphatically  states  that  relying  on  only  a few  technology  specialists 
and  risking  the  development  of  dependency  relationships  might  actually  delay  the  progress  toward  technology 
integration.  In  order  to  utilize  technology  effectively,  teachers  need  immediate  assistance  when  equipment  fails. 
Unresolved  hardware  and  software  problems  will  create  frustration,  and  if  not  dealt  with  immediately,  teachers  will 
abandon  the  attempt  to  use  the  technology. 

If,  in  fact,  effective  technology  use  does  enhance  student  learning,  then  rapid  and  appropriate  incorporation 
of  technology  is  essential.  In  a 1995  report,  the  U.S.  Office  of  Technology  Assessment  (OTA)  asserts  that  helping 
teachers  use  technology  effectively  may  be  the  most  important  step  to  assuring  that  current  and  future  investments  in 
technology  are  realized.  Sheingold  believes  that  properly  trained  teachers  make  the  difference  between  the  success 
or  failure  of  an  integration  effort  (as  cited  in  Roblyer,  Edwards,  & Havriluk,  1997). 

Summary 

The  appropriate  use  of  technology  in  education  is  one  viable  solution  to  the  challenge  of  training  students 
for  this  Age  of  Information.  Relatively  few  schools  offer  such  a program  at  present.  Research,  although  still  sparse, 
has  indicated  that  student  achievement  is  comparable  to  traditional  student  achievement. 

Benefits  and  barriers  have  been  presented.  The  most  logical  benefits  are  improved  student  performance, 
behavior  and  an  efficient  manner  of  procuring  information.  An  additional  benefit  which  needs  to  be  mentioned  is  the 
global  awareness  and  education  it  easily  provides  for  learners. 

In  addition  to  barriers  referred  to  previously,  logistics  present  major  difficulties.  Administrators  and  staff 
would  need  to  thoroughly  and  elaborately  organize  and  prepare  for  the  implementation  of  technology  in  the 
classroom.  Many  hours  would  need  to  be  dedicated  to  studying  proper  implement  procedures.  Hardware  and 
software  would  need  to  be  bought  and  installed.  Space  is  a consideration.  An  ongoing  needs  assessment  would  be 
necessary  once  a program  is  in  place.  Technical  support  throughout  would  be  critical  to  its  survival. 

In  spite  of  these  potential  limitations,  educational  technology  can  work  effectively  with  our  school  systems 
to  properly  equip  our  students  for  the  future.  The  educational  system  has  to  be  willing  to  change  and  adapt  in  order 
to  not  only  survive,  but  to  empower  its  learners  to  be  proactive  participants  in  this  rapidly  shifting  world. 

Experts  in  the  use  of  instructional  technology  have  suggested  that  there  are  four  factors  necessary  to 
facilitate  the  adoption  of  technology.  First,  the  process  of  change  needs  to  be  situated  to  each  particular  school. 
Second,  establishing  a goal  is  critical.  Third  is  the  necessity  of  staff  development.  Finally,  technical  support  must  be 
provided. 
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Abstract 

Most  teachers  have  little  experience  integrating  technology  into  their  students ' learning  processes  and 
typically  do  not  have  models  on  which  to  build  their  own  visions  of  an  integrated  classroom  (Beichner, 
1993;  Cifuentes,  2001;  Kerr,  1996;  Schrum,  1999;  Studler  & Wetzel,  1999).  Teachers  participating  in 
technology  training  must  move  beyond  training  classrooms  to  apply  teaching  methods  that  facilitate 
technology  integration  in  their  classrooms. 

The  purpose  of  this  study  was  to  determine  to  what  extent  teachers  (a)  alter  their  teaching  methods  and  (b) 
integrate  technology  into  their  classroom  curriculum  during,  and  after  a technology  training  course 
designed  to  prepare  teachers  to  use  technologies  that  support  their  teaching  and  student  learning. 

The  eight  cases  studied  illuminated  the  processes  of  technology  integration  for  elementary  and  secondary 
teachers  possessing  low  and  high  levels  of  technology  skill  and  use.  Through  the  results  of  this  study, 
instructors  will  understand  better  how  to  facilitate  training  in  the  integration  of  technology  for  different 
types  of  teachers.  The  study  identified  implications  for  fiture  studies  involving  technology  integration 
training  and  processes  of  integrating  technology  in  the  curriculum. 

Theoretical  Framework 

In  order  to  investigate  the  processes  that  occur  during  and  after  technology  training  as  individuals  integrate 
technology  into  their  classroom  curriculum,  an  understanding  of  the  current  educational  system  as  it  relates  to 
technology  and  teaching  practices  must  be  established.  The  goals  of  restructuring  the  current  educational  system 
were  reviewed  to  gain  understanding  of  the  shift  from  the  traditional  role  of  teachers  as  purveyors  of  knowledge  to  a 
role  that  allowed  for  the  establishment  of  a more  learner-centered  environment  and  the  role  that  technology  would 
have  in  establishing  these  new  roles  and  supporting  the  new  learning  environments  (Goals  2000,  2000;  Technology 
Assessment  [OTA],  1995;  Texas  Long  Range  Plan,  1996).  The  theoretical  foundations  of  this  study  are  therefore 
based  on  a conceptual  understanding  of:  (a)  the  goals  of  restructuring;  (b)  the  role  of  technology  in  restructuring;  (c) 
the  changing  roles  of  teachers;  (d)  teacher  training;  (e)  barriers  to  technology  integration;  and  (f)  the  implementation 
process. 

Training  teachers  in  the  processes  of  integrating  technology  must  replace  current  practice  of  simply  training 
teachers  in  computer  applications  (Brownell,  1992;  Ertmer,  1999;  Roblyer,  Edwards,  & Havriluk,  2000;  Schrum, 
1999;  Simonson  & Thompson,  1997).  Teachers  determine  what  happens  in  the  classroom  and  how  innovations  are 
or  are  not  implemented  (Sandholtz,  Ringstaff,  & Dwyer,  1997).  Therefore,  if  teachers  are  to  move  beyond  the 
training  environment  to  implementation  in  their  classroom,  they  must  be  prepared  to  overcome  obstacles  that 
interfere  with  the  process.  Effective  training  programs  are  needed  to  provide  teachers  with  models  to  build  their  own 
visions  of  an  integrated  classroom  (Beichner,  1993;  Cifuentes,  2001;  Kerr,  1996;  Schrum,  1999;  Studler  & Wetzel, 
1999). 

Methods 


Case  study  methods  (Wiersma,  1995)  were  applied  to  gain  an  understanding  of  the  experiences  of  teachers 
as  they  moved  through  an  OL  or  F2F  course  designed  to  prepare  teachers  for  integrating  technology  into  the 
curriculum.  Complementary  data  collection  processes  (Shulman,  1986)  were  used  in  each  of  the  eight  cases  to 
provide  depth  and  breadth  in  identifying  and  analyzing  the  barriers  and  processes  affecting  the  impact  of  the  training 
course.  In  this  study,  the  integration  of  survey,  interview,  and  observational  approaches  offered  the  researcher  an 
opportunity  to  develop  a complete  analysis  of  participant  behavior  from  a holistic  perspective  (Gall,  Borg  & Gall, 
1996). 


Data  Sources 


INST  603 1: Applications  of  Technology,  a core  graduate-level  course  offered  by  the  School  of  Education  at 
the  University  of  Houston  - Clear  Lake,  introduced  students  to  the  tools  and  skills  necessary  to  understand  and 
operate  computers,  navigate  the  Internet  and  World  Wide  Web,  and  create  hypermedia  products.  The  course 
included  educational  applications  of  instructional  and  information  technologies  to  promote  the  integration  of 
technology  into  the  curriculum.  Emphasis  was  on  the  comprehensive  integration  and  implementation  of  the 
Technology  Applications  Texas  Essential  Knowledge  and  Skills  (Texas  Education  Agency,  2001),  Secretary’s 
Commission  on  Achieving  Necessary  Skills  (SCANS)  2000  report  (U.S.  Department  of  Labor),  and  those  tools  that 
have  important  implications  for  the  creation  of  products  with  the  technology.  The  course  was  offered  either  in  an  OL 
format  or  in  a traditional  F2F  setting.  Within  both  delivery  methods,  a combination  of  hands-on  lab  assignments  and 
content  material  was  offered  through  a student-centered  approach. 

The  PK-12  teachers  enrolled  in  all  sections  of  the  graduate  level  INST  6031: Applications  of  Technology 
course  at  University  of  Houston  - Clear  Lake  made  up  the  pool  of  30  participants.  A computer  use  survey  was 
administered  and  collected  prior  to  any  classroom  instruction.  The  survey  scores  were  based  on  a point  value 
associated  with  the  level  of  skills  selected  by  the  individual,  levels  1 - 4 were  given  point  values  of  1 - 4 
accordingly,  no  response  resulted  in  zero  points  for  that  question.  The  participants  were  divided  into  two  groups, 
which  were  representative  of  the  participants’  teaching  grade  levels  Elementary  (PK-6)  and  Secondary  (6-12).  The 
Elementary  and  Secondary  groups  were  then  divided  by  course  delivery  method.  The  two  course  delivery  methods 
under  investigation  were  OL  and  F2F.  Therefore,  the  cases  for  investigation  were  selected  from  a sample  of 
Elementary  - OL,  Elementary  - F2F,  Secondary  - OL  and  Secondary  - F2F.  A systematic  sample  using  circular 
lists,  rank  ordered  by  skill  level  from  highest  to  lowest  composite  score,  was  used  to  ensure  representation  of  varied 
technology  skill  levels  within  the  grade  levels. 

Results 


Initial  attempts  at  integrating  technology  indicated  all  of  the  teachers  used  technology  in  ways  that 
replicated  their  current  teaching  practices,  which  meant  six  of  the  eight  participants  created  teacher-directed 
technology -based  lessons.  Those  who  developed  learner-centered  integrated  lessons  followed  more  constructivist 
principles  in  the  classroom.  Their  classrooms  were  learner-centered  environments  that  focused  on  discover  and 
problem  solving. 

During  the  final  observation  all  teachers  used  technology  tools  that  were  learner-directed  in  application, 
although  many  of  the  activities  still  perpetuated  the  teacher-directed  environment.  Those  who  had  consistently  used 
constructivist  principles  in  their  teaching  continued  to  develop  learner-centered  activities  that  incorporated 
technology. 

During  Web-based  activities  students  were  responsible  for  obtaining,  synthesizing,  and  reporting  on 
information  obtained  from  the  Internet.  In  the  three  classrooms  that  used  a Web-based  activity  during  the  final 
observation,  student  engagement  increased,  discipline  problems  declined,  and  the  teachers’  role  changed  from  the 
sole  provider  of  knowledge  and  skills  to  one  of  facilitator  or  guide. 

The  researcher  found  several  intrinsic  and  extrinsic  factors  that  affected  the  integration  of  technology.  Lack 
of  technical  support  was  consistent  on  seven  of  the  eight  campuses.  The  campus  with  onsite  support  personnel  who 
assisted  the  teacher  in  locating  resources,  preparing  the  computer  for  student  use,  and  assisting  during  the 
technology -based  lesson,  moved  to  a higher  level  of  collaboration  among  the  teachers  on  campus  than  the  other 
observed  campuses.  Additional  barriers  beyond  the  control  of  the  teachers  or  the  researcher  were  created  by  (a)  an 
emphasis  on  a fixed  curriculum  that  supported  TAAS  and  End  of  Course  objectives,  (b)  a lack  of  technology 
training,  and  (c)  insufficient  resources. 

Teachers’  use  of  technology  was  directly  impacted  by  district  and  campus  level  policies.  The  teacher  in  the 
one-computer  classroom  was  hindered  by  policy  that  restricted  use  of  the  computer  to  the  teacher  only.  The 
restricted  use  affected  the  ability  of  the  teacher  to  move  beyond  teacher-directed  instructional  uses  of  the  computer. 
This  was  especially  true  of  Internet  access,  which  mandated  that  all  Internet-based  activities  conducted  in  the 
classroom  had  to  be  directed  by  the  teacher  at  all  times. 

There  were  no  noted  differences  between  the  growth  in  skill  levels  of  the  teachers  involved  in  the  OL 
section  of  the  course  and  those  in  the  F2F  section.  There  were  differences  between  the  OL  section  and  the  F2F 
section  in  their  identified  Stages  of  Concern.  Only  the  F2F  section  teachers  indicated  increased  concerns  about 
collaboration  (Stage  5).  Only  the  OL  section  teachers  indicated  increased  concerns  about  modifications  or 
alternatives  to  technology  integration. 

When  comparing  across  the  two  groups,  the  level  of  use  was  a greater  among  those  teachers  enrolled  in  the 
F2F  section  of  the  course  than  those  enrolled  in  the  OL  section.  This  could  indicate  that  the  F2F  population 
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benefited  from  the  ability  to  see  and  interact  with  technology  integration  in  an  environment  that  closely  resembled 
their  own  classroom  settings. 

The  teachers  in  the  F2F  section  used  cooperative  groups  and  demonstrations  more  frequently  than  the 
teachers  enrolled  in  the  OL  section.  This  is  consistent  with  research  on  the  influence  past  learning  experiences  have 
on  current  teaching  practices  because  the  F2F  group  had  increased  opportunity  for  cooperative  group  work  and  the 
OL  section  relied  on  demonstrations  for  a large  percentage  of  their  content  materials  (Dwyer,  Ringstaff,  & 

Sandholtz,  1990a,  1990b).  Classroom  management  for  the  F2F  and  OL  section  teachers  was  consistent  with  that  of 
the  other  groups.  Both  groups  indicated  improvements  in  classroom  management  with  the  use  of  technology. 

Elementary  and  Secondary  teachers  varied  in  their  growth  in  skills  and  knowledge  of  basic  and  advanced 
computer  use  but  were  similar  in  those  related  to  Internet  use.  The  two  groups  were  similar  in  their  identified  Stages 
of  Concern.  Both  groups  focused  early  on  informational  (Stage  1)  and  personal  (Stage  2)  concerns.  The  majority  of 
teachers  in  each  group  indicated  increased  levels  of  concern  over  time  for  the  consequences  of  using  technology 
with  their  students  (Stage  4).  The  Secondary  group  contained  a higher  percentage  of  teachers  with  time,  logistics, 
and  management  concerns  (Stage  3). 

The  levels  of  use  of  the  teachers  in  the  Elementary  group  increased  by  a higher  margin  than  that  found  in 
the  Secondary  group. 

Initially  the  two  groups  varied  in  their  preferred  methods  of  instructional  delivery.  By  the  end  of  the 
semester  the  two  groups  were  closely  aligned  in  their  uses  of  instructional  methods. 

The  teachers  in  both  groups  indicated  improvements  in  classroom  management  with  the  use  of  technology, 
citing  increased  student  engagement  with  technology.  The  need  for  discipline  was  reduced  in  both  groups.  Student 
assessment  was  more  manageable  when  the  students  were  interacting  with  the  technology. 

There  were  not  dramatic  differences  in  initial  skill  levels  between  the  High  Skill  and  Low  Skill  level 
groups.  The  biggest  difference  in  the  initial  scores  was  in  Teacher  Internet  Use  (Part  C).  While  both  groups  focused 
early  on  informational  (Stage  1)  and  personal  (Stage  2)  concerns,  only  the  Low  Skill  level  teachers  indicated 
increased  levels  of  concern  over  time  for  the  consequences  of  using  technology  with  their  students  (Stage  4).  Both 
groups  indicated  increased  concerns  about  collaboration  (Stage  5).  Both  groups  indicated  low  levels  of  concern 
related  to  time,  logistics,  and  management  concerns  (Stage  3). 

When  compared  across  the  two  groups  of  teachers,  it  appears  that  the  Level  of  Use  was  most  impacted  by 
initial  Level  of  Use  and  increased  skill  levels  over  time.  Teachers  at  the  Low  Skill  level  encountered  higher  levels  of 
technical  difficulties  that  required  assistance  from  the  researcher  to  resolve.  The  lack  of  onsite  assistance  directly 
impacted  their  ability  to  use  technology  without  additional  support. 

The  High  Skill  teachers  indicated  they  used  a wider  variety  of  instructional  methods  on  a regular  basis. 
After  training  the  two  groups  were  more  closely  aligned  with  the  Low  Skill  teachers  now  indicating  increased 
frequency  of  a wider  variety  of  methods.  Indicating  that  as  skills  and  knowledge  increase  the  individuals’  choice  of 
instructional  methods  were  expanded. 

In  summary,  the  results  of  this  study  indicate  that  the  individual  characteristics  of  the  teachers,  the  content 
area  they  teach,  their  previous  teaching  experience,  their  career  goals,  and  their  classroom  environment  have  an 
impact  on  how  and  to  what  degree  they  integrate  technology  into  their  classroom  curriculum  during  and  after 
training  in  the  processes  of  technology  integration.  A course  designed  to  train  teachers  in  technology  skills  and 
technology  integration  raises  the  skill  levels  of  the  teachers,  increases  the  use  of  technology  in  the  classroom  but  the 
course  in  this  study  did  not  alter  the  teachers’  established  teaching  methods.  This  researcher  hypothesizes  that  such 
alterations  require  a more  comprehensive  effort  than  one  class. 

Discussion 

The  teachers  in  this  study  possessed  unique  sets  of  skills  and  knowledge  related  to  computer  use,  Internet 
use  and  technology  integration.  Concerns  were  individualized  and  personal,  often  directly  linked  to  their  campus 
environment,  their  previous  educational  training,  their  prior  teaching  experience,  and  their  career  goals.  Levels  of 
use  reflected  skill  and  knowledge  growth  as  well  as  the  access  and  availability  of  technology  within  the  school 
environment.  Instructional  methods  were  used  that  met  the  needs  of  content,  students,  and  personal  preference. 
Classrooms  were  managed  in  ways  that  modeled  the  classrooms  of  the  individuals’  past  educational  experiences. 

During  the  course,  the  teachers  obtained  instruction  in  basic  use  and  technology  integration.  However,  their 
skill  growth  did  not  reflect  the  amount  of  training  received  but  instead  reflected  the  individuals’  initial  skill  levels 
and  opportunities  to  reinforce  the  new  knowledge  through  practice  and  application.  Therefore,  teacher  preparation 
programs  need  to  integrate  technology  throughout  the  program  and  not  restrict  technology  instruction  to  one  course; 
one  course  is  not  enough. 
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I observed  uses  of  technology  in  the  classroom  that  replicated  the  activities  in  the  course  and  sought  to 
mimic  the  existing  teaching  practices  of  the  individual  teacher.  The  teachers  who  adhered  to  teacher-directed 
principles  were  unable  to  create  a vision  for  technology  integration  in  their  classrooms,  while  those  who  followed  a 
more  constructivist  approach  to  instructional  delivery  were  able  to  envision  technology  as  a tool  that  would  enhance 
the  teaching  and  learning  process. 

Teachers  need  more  experiences  with  technical  skills  and  knowledge  if  they  are  to  develop  a vision  for 
technology  integration  within  their  own  individualized  environment.  They  need  models  of  effective  teaching 
practices  that  integrate  technology.  They  need  access  to  resources  that  promote  or  support  technology  integration  in 
the  curriculum.  They  need  to  experience  technology  integration  in  environments  that  closely  resemble  their  own 
classrooms;  and  they  need  opportunities  to  practice  technology  integration  in  their  classrooms  with  the  necessary 
levels  of  technical  and  administrative  support. 

If  teacher  educators  are  to  facilitate  technology  integration  for  different  types  of  teachers,  they  need  to 
design  and  implement  learning  environments  that  (a)  are  learner-centered,  (b)  encourage  collaboration,  (c)  promote 
discovery,  and  (d)  provide  activities  that  are  engaging  and  relevant  to  the  individual  needs  and  environments  of  the 
learners.  Teachers  will  develop  visions  of  technology  integration  based  on  their  own  educational  experiences. 
Therefore  training  programs  must  provide  rich  extended  experiences  in  technology  integration  and  model  effective 
practices  and  innovative  uses  of  technology  that  improve  teaching  and  learning. 
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In  2000,  the  Georgia  Legislature  mandated  that  all  certified  school  personnel  complete  the  Georgia 
Framework  for  INtegrating  TECHnology  in  the  Student-Centered  Classroom  (InTech)  or  approved  alternatives. 
InTech  is  a constructivist-based  technology-training  program.  It  was  developed  as  a statewide  staff  development 
initiative  by  the  Georgia  Department  of  Education  in  1997,  and  was  implemented  in  1998.  The  purpose  of  this 
paper  is  to  provide  information  about  this  initiative  to  improve  teaching  and  learning  through  technology.  Included 
are  a problem  statement,  brie  f summaries  of  three  dissertation  research  studies  investigating  the  influences  of 
InTech,  and  recommendations  regarding  similar  initiatives. 

Statement  of  the  Problem 

Since  1993,  the  state  of  Georgia  has  invested  over  $337  million  in  support  of  its  belief  that  providing 
educational  technology  for  classrooms  offers  effective  ways  to  improve  schools  and  to  help  students  learn  (Brackett, 
Henry,  & Weathersby,  1999).  It  has  been  reported  that  Georgia  schools  have  210,885  computers,  with  98  percent 
having  Internet  access  and  49  percent  equipped  with  local  area  networks  (Coley  et  al,  1997;  Georgia  Department  of 
Education,  2000).  The  FY  2001  Georgia  education  budget  included  $29,485,875  funding  for  computers  (Sherrod, 
2000). 

Availability  of  technology  is  one  issue;  its  use  is  another  (Byrom,  1998).  To  prepare  teachers  to  integrate 
technology,  student  learning,  and  academic  goals,  the  Georgia  Framework  for  INtegrating  TECHnology  in  the 
Student-Centered  Classroom  (InTech)  was  designed  as  staff  development  training  in  technology.  Its  overall  goal 
was  to  provide  a catalyst  for  fundamental  changes  in  the  teaching  and  learning  process.  Instruction  in  this  program  is 
constructivist-based  and  is  expected  to  lead  participants  to  development  of  constructivist  teaching  philosophies, 
characterized  by  a shift  to  more  student -centered  learning  (Holmes  et  al.,  1998).  InTech  activities  are  designed  to 
promote  active,  problem-based  learning  opportunities  that  can  be  transferred  to  participants*  classrooms. 

Since  its  introduction  in  1998,  numerous  Georgia  school  systems  voluntarily  have  implemented  InTech  and 
over  15,000  Georgia  educators  have  been  trained  through  the  program.  In  2000,  Governor  Roy  Barnes  signed  into 
law  Georgia  House  Bill  1187  (The  A Plus  Education  Reform  Act  of  2000)  which  mandates  that  all  Georgia  teachers 
and  administrators  seeking  certification  or  certificate  renewal  must  complete  successfully  the  InTech  training 
program  or  demonstrate  competency  in  ISTE  computer  skill  standards  through  alternatives  approved  by  Georgia’s 
Professional  Standards  Commission. 

This  state -sponsored  initiative  was  enacted  to  improve  Georgia’s  schools  and  to  enhance  student  learning; 
however,  decisions  for  the  ongoing  commitment  were  made  without  empirical  support.  To  address  the  specific  need 
for  research  examining  the  influences  of  InTech  and  the  general  need  for  research  examining  the  most  effective 
ways  to  implement  changes  necessary  for  integrating  technology  into  the  curriculum,  three  doctoral  candidates  at 
Valdosta  State  University  (VSU)  conducted  dissertation  studies  of  InTech.  Variables  of  the  three  studies  include: 
perceived  levels  of  technology  integration,  accessibility  to  technology,  administrative  support,  teacher  integration  of 
technology,  student  utilization  of  technology,  teacher  and  administrator  beliefs  related  to  technology  use. 

Summaries  are  given  below. 

InTech  Training,  Technology  Integration,  and  Administrative  Support  of  Technology:  Perceptions  of 
Elementary  School  Teachers. 

By  Lynn  Minor 
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InTech  has  been  offered  to  teachers  in  the  Valdosta  State  University  (VSU)  service  region  for  the  past  three 
years  through  the  VSU  Educational  Technology  Teaching  Center  (ETTC).  However,  the  extent  to  which  InTech 
training  at  the  VSU  ETTC  or  through  the  redelivery  program  affects  the  technology  integration  of  teachers  was 
largely  unknown.  Redelivery  is  a term  used  to  refer  to  a model  of  implementing  InTech  training  in  which  teachers 
receive  trainer-of-trainers  instruction  at  the  ETTC  and  return  to  implement  InTech  with  teachers  in  their  own  school. 
This  quantitative  research  study  was  conducted  to  evaluate  the  effectiveness  of  Phase  One  of  the  Elementary  InTech 
Professional  Development  Program  offered  at  Valdosta  State  University.  The  purpose  of  the  investigation  was  to 
examine  perceived  levels  of  technology  integration  of  elementary  school  teachers  as  a function  of  the  InTech 
training  model  received  and  to  determine  whether  a relationship  was  present  between  perceived  levels  of 
administrative  support  and  technology  integration.  Accessibility  to  technology  in  the  classrooms  of  the  participants 
was  also  investigated. 

Methods 


Data  for  this  study  were  collected  using  a questionnaire,  titled  Technology  Integration  Survey  for  Teachers, 
that  was  adapted  from  the  Student  Achievement  Survey  used  by  the  Bemie  School  District  (1999).  Twenty-six 
questions  concerning  teachers’  perceptions  of  technology  use  in  their  classrooms,  accessibility  of  technology  in  the 
classroom,  and  administrative  support  for  technology  integration  in  their  schools  were  included  on  the  instrument. 
Participants  were  selected  using  purposive  sampling  to  include  teachers  in  the  Valdosta  State  University  (VSU) 
Educational  Technology  Training  Center  (ETTC)  service  area  who  teach  at  elementary  schools  with  the  following 
three  groups  of  teachers:  teachers  who  received  InTech  training  at  the  VSU  ETTC,  teachers  who  received  InTech 
training  through  the  redelivery  model  at  their  own  school,  and  teachers  who  have  not  received  InTech  training. 
Twenty-six  elementary  schools,  representing  9 of  the  45  school  districts  in  the  VSU  ETTC  service  area,  met  the 
criteria  for  this  study.  Completed  surveys  were  received  from  630  of  the  993  possible  participants,  representing  a 
response  rate  of  63%. 

Results 


Responses  to  sixteen  Likert -type  survey  items  were  combined  to  calculate  the  perceived  level  of 
technology  integration.  A one-way  analysis  of  variance  (ANOVA)  was  conducted  to  determine  whether  an  overall 
difference  in  the  perceived  level  of  technology  integration  was  present  as  a function  of  the  InTech  training  model. 
Statistically  significant  differences  were  found  for  perceived  levels  of  technology  integration  with  a main  effect  of  F 
(2,  610)  = 66.28,  p < .0005.  Teachers  who  received  training  at  the  VSU  ETTC  (M  = 50.69)  perceived  their  level  of 
integration  as  significantly  higher  than  did  teachers  who  received  training  through  the  redelivery  model  (^=  46.15) 
and  teachers  with  no  InTech  training  (M  = 37.41).  Furthermore,  a statistically  significant  difference  was  also  found 
in  the  perceived  level  of  technology  integration  between  teachers  who  received  InTech  training  through  the 
redelivery  model  and  those  with  no  InTech  training.  Calculation  of  eta  squared  yielded  an  effect  size  of  .47. 
According  to  Cohen,  (1988),  a large  effect  is  represented  by  these  results. 

Additional  information  pertaining  to  teachers’  perceived  levels  of  technology  integration  was  gleaned  by 
analyzing  responses  to  survey  items  concerning  accessibility  to  various  types  of  technology  in  their  classrooms. 
Pearson  chi-square  tests  were  conducted  for  each  possible  response  to  determine  whether  statistically  significant 
differences  were  present  among  the  three  groups  of  teachers.  Statistically  significant  differences  in  access  to  a 
scanner,  more  than  one  multimedia  computer,  and  a scan  converter  and  television  were  found  with  more  VSU  ETTC 
InTech  teachers  reporting  access  to  these  than  redelivery  and  non-InTech  teachers.  In  addition,  more  redelivery 
teachers  reported  access  than  did  non-Intech  teachers.  However,  more  teachers  who  received  training  through  the 
redelivery  model  and  teachers  with  no  InTech  training  reported  having  one  multimedia  computer  than  did  teachers 
trained  at  the  VSU  ETTC.  Interestingly,  a higher  percentage  of  redelivery  InTech  teachers  reported  having  access  to 
a digital  camera  than  did  VSU  ETTC  InTech  teachers  and  non-InTech  teachers.  No  statistically  significant 
difference  was  found  among  the  three  training  groups  for  access  to  a laser  disk  player. 

Teachers’  responses  to  five  Likert-type  survey  items  were  combined  to  calculate  perceived  levels 
of  administrative  support.  A statistically  significant  relationship  was  found  between  perceived  administrative 
support  and  levels  of  technology  integration.  Use  of  a Pearson  r yielded  the  following:  i(613)  = .23,  p < .0005. 
Squaring  the  correlation  provided  evidence  that  5.29%  of  the  variance  was  shared  between  the  perceived  levels  of 
administrative  support  and  technology  integration.  According  to  Gay  and  Airasian  (2000),  this  coefficient  indicates 
a weak  relationship. 


405 


894 


Conclusions 


Phase  One  of  Elementary  InTech  training  has  had  favorable  effects  on  technology  integration  in  the 
classrooms  of  elementary  teachers  v^ho  received  this  training.  Teachers  who  received  InTech  training  through  the 
VSU  ETTC  indicated  higher  levels  of  technology  integration  than  did  teachers  who  received  InTech  training 
through  the  redelivery  model  and  teachers  with  no  InTech  training.  Furthermore,  a statistically  significant 
relationship  was  found  between  the  perceived  levels  of  administrative  support  and  levels  of  technology  integration. 
Although  this  finding  represented  a weak  relationship,  administrative  support  of  technology  professional 
development,  and  integration  is  recommended.  Administrators  can  provide  support  for  technology  integration  by 
arranging  for  teachers  to  participate  in  professional  development,  making  technology  accessible  in  classrooms,  and 
modeling  the  use  of  technology. 

Teachers’  and  Administrators’  Beliefs  Regarding  Constructivist-Based,  Exemplary  Practices  for  Technology 
Integration  in  Middle  School  Classrooms 
By  Fritzie  Sheumaker 

Introducing  technology  into  classrooms  as  key  components  in  the  teaching  and  learning  process  involves 
more  than  providing  hardware  and  software.  The  use  of  technology  must  coincide  with  how  students  learn  best 
(National  Council  for  Accreditation  of  Teacher  Education  Task  Force  on  Professional  Development  [NCATE], 

1 997).  Because  the  constructivist  use  of  technology  engages  students  in  the  learning  process  and  helps  them  build 
new  ideas  (Jonassen,  Peck,  & Wilson,  1999)  constructivist  philosophy  provides  the  foundation  for  effective 
strategies  for  teaching  with  technology  (Strommen  & Lincoln,  1992). 

InTech  was  designed  to  help  both  teachers  and  administrators  develop  constructivist-based  pedagogies  and 
use  technology  as  part  of  broader  educational  change  and  reform  efforts  (Holmes  et  al.,  1998).  While  teachers  make 
technology  use  work  in  classrooms  (Anderson  & Harris,  1997,  Becker,  1999,  2000;  Saye,  1998),  administrators  are 
essential  to  organizational  change  in  schools  (Bennett,  1996,  Lambert,  1998,  Stanley  et  al.,  1998).  The  building 
principal  was  found  to  be  the  change  agent  responsible  for  implementing  technology  integration  and  to  be  an 
important  factor  in  creating  momentum  toward  exemplary  technology  use,  constructivist  practice,  and  improved 
learning  for  students. 

Because  no  studies  were  found  on  the  effectiveness  of  Georgia’s  InTech  professional  development  model 
in  promoting  comparable  teacher  and  administrator  beliefs,  the  purpose  of  this  study  was  to  assess  the  effect  of 
participation  in  InTech  training  on  middle  school  teachers’  and  administrators’  beliefs  regarding  constructivist- 
based,  exemplary  uses  of  classroom  technology. 

Methods 


A causal-comparative  research  design  was  used,  and  participants  included  342  teachers  and  29 
administrators  from  10  Southwest  Georgia  Middle  Schools.  Data  were  gathered  from  each  participant’s  responses  on 
the  Technology  and  Teaching  Practices  Survey  (TTPS)  and  were  analyzed  through  analysis  of  variance  procedures. 
In  a process  labeled  as  qualitative  contrasting  case  analysis  (Onwuegbuzie  & Teddlie,  in  press),  qualitative  data 
were  collected  through  interviews  with  two  teachers  and  two  administrators  identified  on  the  basis  of  their 
quantitative  survey  scores.  The  selection  of  the  interview  participants  was  based  on  extreme  TTPS  scores  with  high 
constructivist  being  those  respondents  with  a mean  score  greater  than  5.0  and  low  constructivist  those  whose  mean 
scores  were  less  than  3.0. 

Results 


A factor  analysis  of  participant  responses  on  the  TTPS  revealed  three  factors  related  to  constructivist 
beliefs:  (a)  Nature  of  Classroom  Instruction  (NCI),  (b)  Nature  of  Classroom  Roles  (NCR),  and  (c)  Nature  of 
Knowledge  and  Evaluation  (NKE).  A score  was  also  calculated  from  two  additional  items  as  an  indication  of 
participants’  Comfort  with  Using  Computers  (CCI).  Based  on  the  view  of  Feng  (1995)  that  educators  may  adhere  to 
constructivist  beliefs  in  varying  degrees,  cut  points  for  the  total  TTPS  mean  were  established.  An  examination  of  the 
distribution  of  scores  revealed  a normal  curve,  and  the  means  were  then  coded  into  a new  variable  labeled  as 
Constructivist  Category.  A mean  score  between  2.00  - 3.00  indicated  a participant  could  be  termed  a 
“Traditionalist.”  A mean  score  of  3.01  - 4.99  or  5.00  - 6.00  identified  a participant  as  “Emergent-Constructivist”  or 
“Active-Constructivist”,  respectively. 
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A statistically  significant  relationship  was  found  between  respondents’  InTech  status  and  constructivist 
category.  With  regard  to  all  InTech-trained  teachers  and  administrators,  1 1%  scored  at  the  Active-Constructivist 
level  and  88%  as  Emergent-Constructivist.  When  only  teacher  respondents  were  considered,  there  was  also  a 
statistically  significant  relationship  between  InTech  status  and  Constructivist  Category  with  89  % of  InTech-trained 
teachers  ranked  as  Emergent-Constructivist  and  1 1%  as  Active-Constructivist.  There  was  no  statistically  significant 
relationship  between  administrators’  InTech  status  and  their  Constructivist  Category. 

Statistically  significant  differences  were  found  based  on  the  total  TTPS  mean  and  the  NCI,  NCR,  and  CCI 
subscale  scores  between  the  constructivist-based  beliefs  of  InTech  trained  teachers  and  administrators  and  non- 
InTech  trained  teachers  and  administrators.  InTech-trained  teachers  also  had  statistically  significantly  higher  scores 
than  their  non-InTech-trained  counterparts  on  the  total  TTPS  and  the  NCI  and  NCR  subscales.  However,  InTech 
training  did  not  appear  to  have  the  same  influence  on  the  constructivist-based  beliefs  of  participating  administrators. 
No  statistically  significant  differences  were  found  in  administrators’  scores  on  the  TTPS  total  score,  nor  the  NCI, 
NCR,  or  NKE  subscales.  InTech-trained  administrators  did  have  statistically  higher  scores  on  the  CCI  subscale  than 
non-InTech-trained  administrators. 

Analysis  of  the  interview  data  from  a Traditionalist  Teacher,  a Traditionalist  Administrator,  an  Active- 
Constructivist  Teacher,  and  an  Active-Constructivist  Administrator  revealed  that  a school-wide  shared  vision  for  the 
nature  of  technology  integration  appeared  to  be  the  most  influential  factor  contributing  to  teachers’  and 
administrators’  views  being  more  or  less  congruent  with  constructivist  philosophy.  Other  factors  identified  as 
contributing  to  a respondent’s  constructivist  views  were:  (a)  the  degree  of  emphasis  given  to  standardized  testing 
when  curricular  and  instructional  decisions  were  made;  (b)  the  degree  of  administrative  support  for  constructivist- 
based  instructional  practices;  (c)  the  level  of  comfort  a respondent  had  with  student-centered  classroom  roles,  (d)  the 
level  of  support  for  performance-based  classroom  assessments,  and  (e)  how  standardized  testing  demands  were 
balanced  with  technology  use.  Both  teachers  and  administrators  viewed  InTech  as  a factor  contributing  to  their 
comfort  with  using  computers  for  classroom  instruction. 

Conclusions 

The  results  of  this  study  are  an  indication  that  InTech  training  is  beneficial  to  middle  school  teachers  and 
administrators  who  complete  it  and  may  increase  the  likelihood  that  participating  teachers  and  administrators  will 
share  a vision  for  effective  technology  use.  Moreover,  constructivist-based  beliefs  are  present  among  teachers  and 
administrators  in  Southwest  Georgia  middle  schools  with  the  majority  of  study  participants  scoring  in  the  Emergent- 
Constructivist  Category.  However,  concerns  about  student  performance  on  standardized  tests  may  decrease  the 
support  for  and  use  of  constructivist-based  teaching  strategies.  Constructivism  is  not  an  end  to  itself;  its  value  lies  in 
its  capacity  to  have  a positive  influence  on  student  learning  (Feng,  1995).  To  strengthen  commitment  to 
constructivist-based,  exemplary  uses  of  classroom  technology,  improving  student  learning  outcomes  should  be  an 
essential  focus  of  the  skills  and  beliefs  promoted  by  InTech. 

The  Effectiveness  of  a Constructivist-Based  Professional  Development 
Technology  Integration  Program  in  Increasing  Technology  Utilization 
By  Rachelle  Fowler 

With  the  increasing  availability  of  technology  in  the  school  systems,  a key  issue  educators  must  address  is 
the  effective  integration  of  technology  into  the  curriculum.  Although  many  current  educators  completed  their 
professional  training  before  the  technological  age  in  education,  they  are  compelled  to  use  technology  that  most  find 
unfamiliar  and  intimidating  (Armstrong,  Davis,  & Young,  1996).  To  address  teachers’  needs  for  technology  skills, 
numerous  Georgia  school  systems  have  implemented  staff  development  training  in  technology  through  InTech,  a 
constructivist-based  professional  development  technology  integration  program.  Although  Georgia  has  taken  steps  to 
encourage  teacher  proficiency  in  the  use  of  technology,  such  proficiency  may  not  be  enough  to  affect  student  use  of 
technology  in  the  classroom.  The  purpose  of  this  study  was  to  investigate  both  teacher  and  student  utilization  of 
technology  in  the  academic  classrooms  of  teachers  who  have  or  have  not  been  trained  in  a constructivist-based 
professional  development  technology  training  program  such  as  the  state-sanctioned  InTech  model. 

Methods 


The  research  study  utilized  a mixed  methods  design.  Quantitative  data  were  collected  through  teacher  and 
student  surveys,  teacher  logs  of  technology  use,  and  computer  lab  and  media  center  sign-in  sheets.  Qualitative  data 
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were  collected  through  teacher/student  interviews  and  classroom  observations.  Subjects  for  the  study  included  65 
teachers  and  265  students  at  a comprehensive  high  school  located  in  rural  Southwest  Georgia. 

Results 


Both  quantitative  and  qualitative  analysis  of  data  indicated  an  increase  in  the  use  of  technology.  Teachers  in 
the  group  who  had  been  trained  through  the  InTech  model  utilized  technology  for  teacher-related  tasks  more 
frequently  than  they  did  prior  to  training.  Data  analysis  revealed  an  increase  in  the  use  of  technology  for  student 
management  of  grades,  student  information,  school  management,  word  processing,  databases  and  spreadsheets, 
desktop  publishing,  multimedia/authoring,  instructional  demonstration  and  tutoring,  information  retrieval,  Internet, 
web  page  development,  and  e-mail.  Trained  teachers  also  utilized  technology  more  frequently  for  teacher-related 
tasks  in  some  categories  than  untrained  teachers.  Data  analysis  revealed  an  increase  in  the  use  oftechnology  for 
spreadsheets  and  databases,  instructional  demonstrations,  word  processing,  and  Internet  access. 

Congruently,  students  enrolled  in  the  classes  of  trained  teachers  were  required  to  use  technology  more 
frequently  than  they  were  prior  to  teacher  training.  Data  analysis  revealed  an  increase  in  the  use  of  technology  for 
computer-assisted  instruction  for  simulations  and  educational  games,  word  processing,  information  retrieval, 
databases  and  spreadsheets,  Internet  access,  and  electronic  presentations.  Students  enrolled  in  classes  of  trained 
teachers  were  also  required  to  use  technology  more  than  students  in  the  classes  of  untrained  teachers.  Data  analysis 
indicated  a significant  increase  in  the  use  of  word  processing,  electronic  presentations,  and  Internet  access  for 
students  of  trained  teachers.  Teachers  from  both  groups,  who  required  students  to  use  technology,  employed 
constructivist-based  principles  in  their  assignments. 

Conclusions 

Based  on  the  results  of  analyses  of  quantitative  and  qualitative  data,  InTech,  a constructivist-based  model 
of  professional  development  for  technology  integration  training,  increases  both  teacher  and  student  utilization  of 
technology.  However,  many  factors  influence  the  degree  to  which  technology  is  implemented  in  a school  system. 
Because  of  the  importance  of  technology  in  our  highly  technological  society  and  global  economy,  educators  should 
continue  to  investigate  the  most  effective  ways  to  implement  changes  necessary  for  integrating  technology  into  the 
curriculum. 

Recommendations 

The  recommendation  for  more  research  is  a given  for  the  conclusion  of  any  review  of  research  literature. 
The  three  studies  described  in  this  paper,  although  all  investigated  InTech,  were  quite  diverse.  Three  school  levels, 
elementary,  middle  and  high  school  were  represented  in  the  studies.  Multiple  factors  in  relation  to  technology 
integration  training  were  researched,  including  delivery  models,  administrative  support  and  student  use  of 
technology.  The  InTech  program,  in  its  past  and  current  implementations,  is  rich  ground  for  further  empirical 
research,  for  ongoing  and  accurate  evaluation  research,  and  for  action  research  in  the  Educational  Technology 
Training  Centers  and  the  schools  where  InTech  lives  its  daily  life. 

More  research  is  not  the  only  need;  there  is  also  a need  and  a promise  for  more  InTech,  or  alternatives  to 
the  program.  As  the  state  supported  solution  to  legislatively  mandated  technology  competencies  for  educational 
personnel,  there  is  tremendous  pressure  to  make  InTech  available  to  very  large  numbers  of  educators  in  a relatively 
short  period  of  time.  This  pressure  on  individuals  and  organizations  has  generated  controversial  discussions  about 
the  suitability  of  InTech  training  as  the  ‘magic  bullet’  that  will  solve  the  technology  integration  ‘problem*.  Many 
school  systems,  regional  education  agencies  and  institutions  of  higher  education  have  designed  alternative  programs 
to  meet  their  local  needs.  But  InTech  remains  the  best  known,  most  widespread  and  well-developed  technology 
training  model  in  the  state. 

InTech  is  not  a static  program;  it  is  currently  being  extended  in  content  and  to  serve  additional  populations. 
InTech  has  integrated  experiences  with  assistive  technologies  into  the  basic  program.  A program  specifically  for 
media  specialists  has  been  developed.  Some  teacher  education  programs  have  used  a form  of  InTech  to  train  their 
pre -service  teachers.  Higher  education  faculty  have  participated  in  InTech  in  partnerships  with  pre -service  or  in- 
service  teachers.  A program  based  on  the  model  for  higher  education  faculty  is  in  the  planning  stages.  A second, 
more  advanced,  level  of  InTech  is  in  a pilot  phase  and  may  answer  criticism  that  the  basic  InTech  doesn’t  go  far 
enough  toward  true  integration  of  technology  into  teaching,  learning  and  the  curriculum. 


408 


897 


Whether  technology  integration  training  is  called  InTech  or  not,  there  is  a need  for  more  of  it.  But  such 
training  must  be  based  on  real  needs  of  real  people.  The  support  and  incentives  for  participation  and  implementation 
into  the  curriculum  must  be  in  place.  Research  can  provide  data  upon  which  to  base  these  programs,  and  can 
provide  rationales  for  continuation  or  expansion.  Support  and  incentives  must  be  provided  at  the  state  and  local 
levels  in  order  for  individuals  to  really  buy  into  the  potential  of  technology  to  improve  teaching  and  learning. 
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Teachers’  Perceptions  of  Technology:  Four  Categories  of  Concerns 


HeeKap  Lee 
Indiana  University 

Introduction 

Today,  many  educational  institutions  have  been  challenged  to  integrate  technology  into  their  work  settings. 
Technology  is  a mean  of  supporting  goals  related  to  increased  student  involvement  with  complex,  authentic  tasks 
within  classrooms  and  schools  (Scheingold,  1991).  However,  successful  technology  integrations  are  sometimes 
confronted  with  several  difficulties,  which  include  clients’  resistance  to  change  (Conner,  1992;  Collins,  1991)  or  the 
lack  of  cooperation  of  the  part  of  the  people  involved  in  that  change  (Kemp,  1996). 

Recently,  researchers  argue  that  an  innovation  without  considering  clients’  needs  or  concerns  usually 
resulted  in  resistance  to  change  (Ertmer,  1996;  Hall  & Hord,  1978;  Dormant,  1986).  The  reason  for  this  problem  is  a 
lack  of  attention  to  the  clients’  attitude,  perceptions,  and  concerns  that  people  form  toward  innovation.  These 
concerns  play  an  important  role  in  the  innovation  process  as  well  as  in  the  inherent  quality  of  the  proposed  change 
(Pershing,  An,  & Lee,  2000).  Hence,  identifying  and  addressing  concerns  and  perceptions  are  an  essential  task  of 
change  agents  during  the  whole  innovation  process.  In  this  article,  I will  introduce  a framework  for  analyzing 
clients’  concerns  and  perceptions  based  on  an  information  technology  project  that  has  been  implemented  in  a 
seminary  setting  during  the  last  five  years.  While  conducting  interviews  with  the  teaching  faculty  members  who 
participated  in  the  information  technology  project  at  a seminary,  1 identified  four  categories  of  concerns.  In  this 
article,  1 will  explain  the  characteristics  of  and  interventions  in  each  category  of  concern. 

Context  of  the  Case 

In  the  mid  1990s  a seminary  in  the  Midwestern  United  States  was  awarded  an  externally  funded  grant  for  a 
technology  initiative,  which  included  developing  instructional  computing  capabilities  throughout  the  school  (Saint 
Meinrad,  1995).  The  seminary  hired  two  instructional  interns  to  provide  computer  training  to  the  seminaiy  faculty 
and  staff.  At  the  beginning  phase  of  the  initiative,  the  interns  conducted  a training  needs  analysis.  The  main  focus  of 
the  analysis  was  to  gather  information  about  the  kinds  of  training  programs  faculty  and  staff  members  would  need. 
Through  the  analysis,  however,  several  concerns  surfaced  (Saint  Meinrad,  1998).  For  example,  the  faculty  members 
did  not  seem  to  think  that  computer  technology  was  a tool  useful  for  theology  education,  which  emphasizes  personal 
interactions  within  small  groups.  Administrators  of  the  initiative,  however,  did  not  pay  much  attention  to  this 
perceived  concern.  They  proceeded  on  the  assumption  that  faculty  members  always  complain  about  new  initiatives, 
and  they  viewed  such  concerns  as  natural.  They  assumed  that  faculty  memb  ers  would  eventually  accept  and  use 
computers  provided  the  faculty  members  received  the  proper  training.  With  these  assumptions  in  place,  the 
administrators  put  effort  into  collecting  and  addressing  training  needs  information  while  ignoring  their  concerns. 

After  the  interns  had  provided  in-service  training  for  one  year,  they  began  to  make  informal  visits  to  the 
classrooms,  computer  labs,  library,  and  the  faculty  resource  center.  They  found  that  many  faculty  members  were  not 
integrating  computers  into  their  teaching.  According  to  the  project  implementation  plan,  almost  all  faculty  members 
should  have  been  using  computers  in  their  instruction  after  one  year,  since  all  the  necessary  facilities  and  training 
had  been  provided. 

Faced  with  such  resistance,  the  administration  began  to  take  the  idea  of  concerns  seriously.  The 
administrators  of  the  seminary  learned  it  was  not  the  lack  of  facilities  or  training,  but  concerns  of  the  faculty  that 
affected  the  success  of  the  initiative.  With  this  realization,  they  asked  the  interns  to  conduct  a concerns  analysis. 
One-on-one  interviews  and  document  analyses  were  used  as  data  collection  methods.  All  twenty-six  teaching 
faculty  and  several  administrators  and  staff  of  the  seminary  were  interviewed  (Lee,  2001).  After  conducting  many 
rounds  of  card  sorting,  the  interns  identified  four  categories  of  concerns  as  below. 

Category  1:  Concerns  of  Individual  Incompatibility 

Faculty  perceived  that  the  project  was  not  compatible  with  their  theological  values  or  past  personal 
experiences.  These  concerns  had  a critical  influence  in  the  earlier  stages.  Some  faculty  who  understood  and  were 
well  informed  about  the  project,  however,  had  not  embrace  it  because  of  their  perception  of  technology  as  opposing 
theological  pedagogy.  If  they  we  re  not  persuaded  during  the  earlier  stages,  then  it  was  hard  to  accept  the  project. 
Hence,  this  area  of  concern  was  critically  important  in  the  earlier  stages  of  an  innovation  project,  but  its  importance 
declined  in  the  following  stages.  The  following  are  several  representatives  of  this  area  of  concerns. 
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Conflicts  of  Needs  between  Institution  and  Faculty  Regarding  Technology 

A definite  incongruence  existed  in  needs  between  individual  faculty  and  the  seminary  as  a whole.  The 
institution  had  emphasized  that  technology  was  the  only  tool  to  increase  learning  effectiveness  in  the  information 
age.  However,  to  the  individual  faculty,  using  technology  was  just  one  of  many  ways  to  improve  learning 
methodologies.  As  one  faculty  pointed  out,  to  accomplish  the  goal,  it  was  not  necessary  to  incorporate  technology, 
because  technology  was  not  the  only  way  to  accomplish  the  goals.  Furthermore,  overemphasis  on  technology  at  the 
beginning  stage  of  the  project  negatively  influenced  all  faculty  members. 

Skepticism  about  Technology 

Ten  faculty  members  among  the  26  were  very  skeptical  about  technology.  This  resulted  from  their  not 
being  aware  of  the  worth,  potential  benefits,  or  value  of  technology.  Also,  they  were  not  convinced  that  technology 
was  indispensable  for  their  academic  setting  or  their  personal  lives. 

Paradigm  Paralysis 

Fourteen  faculty  members  out  of  26  expressed  this  category  of  concern,  which  was  the  largest  barrier  to 
adopting  technology  among  the  faculty  members  of  the  seminary.  This  concern  can  be  broken  down  into  two  sub- 
categories: the  faculty’s  perception  of  technology  as  opposing  theological  pedagogy,  and  the  faculty’s  comfort  with 
current  teaching  styles.  Faculty  members  thought  that  technology  (or  the  information  technology  project)  was 
basically  incompatible  with  their  theological  context. 

Theology,  they  argued,  should  focus  on  nurturing  human  nature,  which  is  only  possible  with  human 
interactions  between  instructors  and  students.  In  this  point,  they  felt  that  their  particular  theological  discipline  could 
not  adapt  itself  to  technology  because  theological  educators  are  suspect  of  the  value  of  technology  beyond  the  basics 
of  classroom  pedagogy.  The  other  reason  is  that  they  believed  their  current  teaching  style  had  worked  well  for 
several  decades,  so  there  was  little  desire  to  take  the  time  to  change  it. 

Fear  of  Technology 

Nine  faculty  members  expressed  fear  of  technology  in  two  forms:  fear  of  the  unknown  and  fear  of  the  new. 
Some  faculty  were  afraid  something  would  go  wrong  with  the  computers.  Most  faculty  said  that  they  had  not  grown 
up  in  the  technology  culture.  To  them,  technology  was  a foreign  area.  Even  adopters  of  the  innovation  expressed 
concerns  that  technology  had  advanced  so  drastically  that  keeping  pace  with  the  advancement  of  technology  seemed 
impossible. 

Laggard  Syndrome 

Nine  faculty  members  perceived  themselves  as  being  far  behind  in  using  technology.  One  faculty  who  used 
technology  in  his  classroom  setting  even  expressed  that  he  was  at  the  knowledge  stage,  still  trying  to  find  out  the 
benefits  of  technology. 

False  Information/  Irrational  Belief 

Seven  faculty  members  sympathized  strongly  with  the  criticism  that  technology  is  not  a learning  tool 
proper  for  the  seminary.  Some  faculty  mentioned  that  technology  is  a deterrent  to  human  learning  and 
communication.  Those  arguments  were  not  based  on  scientifically  proven  facts  but  were  based  on  personal  feelings 
or  subjective  reflections.  However,  these  feelings  have  not  allowed  them  to  see  the  potential  benefits  of  the 
technology. 

What  effective  interventions  should  be  needed  to  address  this  category  of  concerns?  Rogers  (1995) 
indicated  that  person-to-person  communication  is  important  to  address  this  area  of  concerns.  Dormant  (1986)  also 
suggested  that  change  agents  should  be  counselors  who  draw  out  concerns,  and  listen  to  and  clarify  the  adoption 
units’  needs  and  interests.  Hence,  individual  persuasion  is  a useful  strategy  to  address  this  area  of  concerns  by 
providing  counseling  and  consultation  sessions.  The  seminary  realized  that  persuasion  on  an  individual  basis  was  the 
best  strategy  after  noticing  faculty’s  resistance  to  the  innovation. 
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The  seminary  recognized  that  a core  group  was  very  skeptical  about  technology  even  after  several  years 
had  passed  since  the  innovation  started.  To  identify  their  concerns,  the  seminary  conducted  one-on-one  interviews 
with  faculty  members  to  become  aware  of  the  many  issues  that  related  to  this  area.  The  seminary  stressed  that 
Instructional  Service  staff  were  not  attempting  to  change  faculty’s  teaching  styles,  but  to  enhance  their  teaching 
styles  with  the  use  of  technology.  Also,  the  seminary  published  a monthly  technology  newsletter,  both  in  print  and 
on  the  intranet,  featuring  articles  on  the  individual-incompatible  area  of  concerns.  Several  faculty  merrbers  wrote 
articles  mentioning  their  successful  experiences  with  technology  in  their  teaching  settings.  The  seminary  provided 
opportunities  for  faculty  members  to  visit  other  advanced  technology-driven  education  institutes  or  learning 
opportunities  to  familiarize  them  with  the  practical  applicability  of  technology  in  the  seminary  context.  Also,  more 
than  ten  faculty  members  attended  technology -related  seminars,  conferences,  and  workshops. 

Category  II;  Concerns  of  Unknown 

Even  when  the  value  of  an  innovation  is  compatible  with  the  target  audience’s  values,  the  individuals  of  the 
adoption  units  may  not  accept  the  proposed  innovation  as  planned  for  several  reasons,  including  fear  of  the  unknown 
and  lack  of  information  or  knowledge  required  to  implement  the  innovation.  In  the  earlier  stages,  the  individual 
faculty  usually  felt  fear  of  the  unknown  or  fear  of  lacking  required  knowledge  or  skills.  The  following  are  the  typical 
examples  of  this  category  of  concern. 

More  Work 

To  eight  faculty  members,  technology  was  one  more  burden  that  they  had  to  learn.  Technology  adds  or 
creates  another  task.  Even  faculty  who  used  technology  in  the  classroom  expressed  this  concern  most  often.  To 
learn  technology  was  becoming  increasingly  stressful  and  time  consuming  for  the  faculty  members. 

Lack  of  Detailed  Information  about  the  Project 

The  vision  of  the  project  was  not  address  well  to  all  faculty  members.  The  lack  of  vision  also  made  it 
difficult  to  set  up  the  details  for  diffusing  the  innovation.  Five  faculty  confessed  that  they  were  not  aware  of  the 
detailed  tasks  in  relation  to  the  innovation.  They  expressed  concerns  about  how  technological  innovations  were  to 
apply  to  the  particular  learning  environment. 

Teaching/Mentoring  Concerns 

Providing  individual  teaching  or  mentoring  was  an  effective  means  of  adopting  the  technology  by  faculty 
members.  Individual  training  was  preferred  over  group  sessions  by  the  school  faculty.  Several  reasons  were 
expressed.  The  difficulty  of  finding  a common  time  among  faculty  members  and  consideration  of  individual  pacing 
were  major  reasons. 

Time  Conflict 

Eight  faculty  mentioned  that  time  was  one  of  the  most  important  concerns  in  their  not  adopting  technology. 
They  said  that  technology  was  not  a priority  to  them,  for  their  primary  responsibility  was  to  prepare  a class  or 
preach.  Some  faculty  had  not  even  tried  to  learn  technology  because  they  worried  about  how  much  time  would  be 
spent. 

Students’  Unfavorable  Attitudes  toward  Technology 

Students’  unfavorable  attitudes  toward  technology  were  also  mentioned  by  two  faculty  members. 

Ironically,  faculty  members  who  had  unfavorable  attitudes  toward  technology  mentioned  that  students  showed  the 
same  phenomena.  Two  faculty  expressed  that  students  did  not  say  that  the  technology  helped  them.  That  made  some 
faculty  not  use  much  technology  in  classrooms. 

Lack  of  Information  about  Good  Applications 
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Four  faculty  members  said  that  it  was  hard  to  find  someone  who  had  applied  technology  very  well.  There 
was  no  easy  way  to  identify  other  faculty  members  on  campus  who  had  already  begun  to  use  technology  effectively 
in  their  teaching.  And  for  most  theological  disciplines  in  particular,  there  was  no  comprehensive,  easy-to-find  source 
of  information  about  relevant  instructional  applications  of  technology.  While  the  number  of  locally  successful 
models  of  educational  uses  of  technology  continued  to  increase,  access  to  good  descriptions  of  those  models, 
training  for  them,  and  reports  of  their  strengths  and  weaknesses  were  not  easy  to  find. 

The  major  strategy  for  addressing  this  category  of  concerns  is  learning,  because  usually  these  concerns  can 
be  overcome  by  providing  well-organized  training  programs,  job  aids,  and  consultation  programs.  Also,  providing 
correct  information  in  a timely  manner  is  another  useful  strategy  to  address  this  area  of  concern.  However,  the 
faculty’s  learning  focus  changed  from  general  and  basic  issues  of  technology  to  more  elaborate  and  complicated 
issues,  such  as  transferring  or  applying  the  technology  in  more  specific  contexts  in  this  case.  This  is  why  the 
learning  format  changed  over  time  from  the  general  group-based  to  the  individual  customized  format. 

To  identify  issues  in  this  area  and  set  up  learning  interventions,  the  seminary  conducted  a needs  analysis 
project  by  conducting  interviews  with  each  faculty  member  as  well  as  mailed  surveys  (Saint  Meinrad,  1998).  Based 
on  these  phenomena,  several  learning  interventions  were  arranged  in  the  seminary.  First  of  all,  an  individual  learning 
road  map  for  each  faculty  member  was  developed.  According  to  the  road  map,  the  well -organized  technology 
training  programs  were  provided  for  the  following  year.  The  interns  had  been  working  on-site  on  a weekly  basis. 
After  taking  these  programs  for  one  year,  the  faculty  improved  their  computer  competencies  from  2.5  out  of  5 on  the 
Likert  scale  to  3.1  in  the  same  survey  (Saint  Meinrad,  1998).  As  the  faculty  moved  deeper  into  the  innovation,  the 
focus  of  learning  shifted  to  more  individualized  consulting  and  one-to-one  training  sessions.  Also,  remote  consulting 
was  offered  by  using  electronic  communication  channels  between  the  faculty  and  the  outside  interns. 

To  address  time  concerns,  the  seminary  formed  a committee  to  reorganize  teaching  loads.  The 
recommendation  of  the  committee  was  that  the  eclectic  courses  could  be  cancelled  if  few  students  enrolled,  so  the 
faculty  could  be  learning  at  that  time  instead  of  teaching  the  course.  Also,  lack  of  time  to  learn  was  the  most  crucial 
factor  in  this  category  of  concerns  in  this  seminary.  To  address  this  concern,  the  seminary  developed  a training 
schedule  that  was  flexible,  meeting  at  different  hours  of  the  day,  even  evenings,  so  that  the  faculty  could  best  take 
advantage  of  the  offerings. 

Category  III:  Concerns  of  Organizational  Support 

The  organizational-compatible  concerns  were  salient  factors  to  be  considered  at  the  middle  stage  of  the 
project  at  the  seminary.  Faculty  who  understood  the  benefits  of  the  innovation  did  not  adopt  it  because  there  were  no 
organizational  encouragements  to  do  so.  Many  faculty  members  expressed  concerns  about  the  lack  of  organizational 
supporting  systems  and  motivational  systems.  The  following  remarks  are  the  typical  expressions  of  this  category  of 
concern.  The  following  are  several  representative  examples  of  this  category  of  concern. 

Equipment  and  Maintenance  Problems 

Several  concerns  were  expressed  in  this  category  by  seven  faculty  members.  First,  the  faculty  experienced 
difficulties  when  servers  went  down,  especially  after  hours  or  on  weekends.  Some  buildings  were  not  equipped  with 
technology.  Not  much  software  was  installed  in  the  Faculty  Resource  Center  (FRC)  or  the  Educational  Technology 
Center  (ETC),  which  made  programs  hard  to  access  when  needed.  Services  from  maintenance  persons  were  hard  to 
find  or  untimely  when  computing  problems  occurred.  Students  may  not  have  had  support  from  the  seminary  to  fix 
their  computers  if  the  computers  had  problems. 

Students’  Limited  Access  to  Equipment  and  Support  Services 

Three  faculty  members  of  the  26  mentioned  that  students’  opportunities  to  access  technology  were  limited. 
Not  all  the  students’  rooms  and  classrooms  were  wired.  Some  students  could  not  access  the  technology,  so  electronic 
communication  was  sometimes  impossible.  Also,  the  computers  in  the  student  production  center  were  so  old  that 
students  could  not  use  advanced  software. 

Students  not  Involved 

Three  faculty  members  mentioned  that  students  were  not  involved  in  the  innovation  from  the  start.  It  was 
directed  to  the  faculty  group  only  and  it  began  without  asking  how  students  would  learn  or  use  the  technology. 
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Lack  of  Organizational  Benefits 


Lack  of  organizational  benefits  and  motivational  factors  were  mentioned  by  seven  faculty  members  out  of 
26.  The  institution  did  not  recognize  the  adopters  of  the  innovation.  Two  faculty  mentioned  they  might  have  adopted 
it  if  the  institution  had  offered  some  motivation  or  incentives,  such  as  monetary  benefits.  Suggestions  of  non- 
monetary benefits  were  also  mentioned,  such  as  vacations  and  training  opportunities,  and  to  lessen  the  teaching 
burden. 

Distrust  and  Poor  Communication  Among  Stakeholders 

There  was  little  communication  and  coordination  among  stakeholders  during  the  innovation  diffusion 
process.  The  innovation  initiators  did  not  even  try  to  gather  ideas  from  the  three  constituent  groiqjs,  faculty 
members,  staff,  and  students,  in  order  to  make  the  most  effective  uses  of  technology,  new  approaches  to  teaching 
and  learning,  and  other  available  resources  in  the  seminary  setting.  Six  faculty  of  the  26  expressed  problems  with 
innovation  in  this  category. 

This  area  of  concerns  is  relatively  easy  to  measure  and  to  eliminate  if  addressed  carefully  and  in  a timely 
fashion  during  the  innovation  diffusion  process  (Fisher,  Wilmore,  & Howell,  1994).  Traditionally,  most  change 
scholars  have  overlooked  these  concerns  at  the  beginning.  However,  in  order  to  lead  a successful  innovation  project, 
the  plan  has  to  be  reviewed  regularly  during  the  innovation  process.  Raising  money,  allocating  resources,  and 
providing  technical  and  administrative  support,  including  incentives  or  motivational  systems,  are  essential  elements. 

These  concerns  can  be  eliminated  by  acquiring  resources  and  equipment,  providing  timely  technical  and 
administrative  support,  providing  incentives  or  benefit  systems,  and  maintaining  equipment.  Foa  (1993)  pointed  out 
that  incentives,  support,  and  reward  structures  are  needed  in  order  to  make  the  efforts  of  the  individuals  more 
widespread  and  their  results  used  more  comprehensively.  Major  problems  for  the  seminary  lay  in  the  institution’s 
failure  to  provide  motivation  or  incentives  to  encourage  faculty  members’  active  usage  of  the  innovation.  Many 
faculty  suggested  both  monetary  and  non-monetary  benefits  and  motivators,  such  as  vacations,  training 
opportunities,  and  a lessened  teaching  burden.  While  not  providing  any  monetary  benefits,  the  seminary  provided 
many  forms  of  non-monetary  benefits,  such  as  providing  training  programs  and  visiting  other  technologically 
advanced  schools.  The  director  of  the  Academic  Computing  Department  became  a member  of  the  president’s 
cabinet,  a group  of  advisors  to  the  president,  and  thus  was  directly  involved  in  developing  a new  master  plan,  which 
included  major  renovations  of  several  buildings  over  the  next  five  years. 

Category  IV:  Concerns  of  Organizational  Incompatibility 

Last  category  of  concerns  is  related  to  the  organizational  incompatibility.  Faculty  expressed  that  the 
innovation  was  not  compatible  with  the  seminary  culture.  The  seminary  culture  was  oriented  toward  more  human 
interaction,  and  focused  on  formation-building.  Furthermore,  the  seminary  was  isolated  geographically,  as  well  as 
divided  by  disciplines.  They  also  expressed  their  isolation  regarding  the  innovation.  The  innovation  was  initiated  in 
a top-down  manner.  They  did  not  receive  information  in  a timely  fashion.  Clear  goals  and  directions  for  the 
innovation  were  not  given  to  the  faculty.  Furthermore,  faculty  tended  to  work  individually  rather  than  in  teams. 

Every  faculty  member  understood  the  innovation  differently.  Hence,  they  perceived  that  two  incongruent  innovation 
diffusion  tracks  existed  in  the  seminary:  the  individual  faculty  track  and  that  of  the  institution.  This  concern  began  to 
increase  in  importance  after  addressing  the  individual-incompatible  concerns,  but  increased  strongly  in  importance 
at  the  implementation  stage  during  the  diffusion  process.  The  following  are  several  examples  of  this  category  of 
concerns. 

Isolated  Culture 

The  cultural  characteristics  of  the  seminary,  4 faculty  argued,  were  not  compatible  with  technology.  First, 
in  preparing  people  for  the  ministry,  the  top  priority  of  the  seminary  is  fundamentally  different  than  preparing 
people  to  teach  in  other  higher  educational  institutions.  The  use  of  technology  can  be  maximized  mainly  in  the  latter 
setting.  Religious  organizations  such  as  the  seminary  must  emphasize  the  value  of  forming  and  building 
relationships,  which  does  not  embrace  technology.  Hence,  some  faculty  mentioned  that  technology  was  not  a driving 
force  at  the  seminary.  Second,  faculty  had  not  grown  up  in  a technology  culture.  Some  faculty  mentioned  that  the 
European  learning  model,  which  mainly  uses  lecture  format  in  classrooms,  had  influenced  the  faculty  members  who 
had  studied  in  Europe,  who  were  the  majority  in  the  seminary.  Third,  the  individuality  of  faculty  members  was 
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another  cultural  characteristic.  Most  of  the  faculty  pursued  different  disciplines  and  different  areas  of  interest.  That 
was  the  major  reason  why  faculty  were  accustomed  to  working  individually  rather  than  in  teams,  which  the 
innovation  sometimes  required  them  to  do. 

Class  Characteristics 

Another  reason  for  incompatibility  originated  from  the  class  contexts  that  were  small-group  class  setting 
and  technology  was  not  related  the  course  content.  Five  faculty  members  argued  that  technology  could  not  make  an 
impact  in  a small  class.  Most  of  classes  were  populated  by  fewer  than  10  students.  In  this  situation,  technology  was 
ineffective  for  increasing  learning.  The  other  reason  why  the  faculty  did  not  utilize  technology  during  class  was  their 
perception  of  the  inability  of  the  course  to  embrace  technology. 

Sharing  and  Showing 

Learning  technology  was  one  of  the  biggest  concerns  of  the  faculty  members.  Nine  faculty  members  of  the 
26  expressed  this  type  of  concern.  Sharing  information  about,  or  experiences  with,  technology  among  faculty  was 
vital,  and  it  could  have  been  a strong  influence  on  the  faculty  as  a whole.  Partly  owing  to  a lack  of  vision  for  the 
innovation  and  to  a lack  of  concrete  examples  of  how  to  apply  technology  in  a seminary  setting,  the  faculty  wanted 
to  see  other  people's  experiences  and  or  knowledge. 

Not  Having  a Clear  Image  of  the  Project 

Eleven  faculty  out  of  26  expressed  a lack  of  vision  for  the  project  from  the  beginning.  This  area  of  concern 
was  the  second  largest  barrier  for  the  faculty.  They  argued  that  the  innovation  project  was  started  by  grant  money 
rather  than  a vision.  Without  serious  questions  about  why  this  innovation  was  needed  in  the  seminary,  the  institution 
started  the  innovation,  and  this  made  it  difficult  for  the  faculty  members  to  grasp  the  vision  or  purpose  of  the 
innovation. 

Fragmented  Technology  Planning 

Five  faculty  members  argued  that  the  innovation  was  started  without  considering  the  necessity  of 
information  technology  carefully  in  the  context  of  Catholic  pedagogy.  They  expressed  that  the  innovation  was 
focused  on  teaching  rather  than  learning,  and  focused  on  media  rather  than  methods.  Two  faculty  criticized  the 
innovation  for  starting  in  reverse  order,  selecting  media  (buying  computers)  without  considering  methods.  One 
faculty  mentioned  that  this  project  had  missed  one  critical  stage  in  the  beginning:  needs  assessment  or  values  . 


Collaboration  was  the  most  useful  strategy  to  address  this  area  of  concern  in  the  seminary.  To  address 
issues  of  this  area,  the  seminary’s  geographical  isolation,  diverse  faculty  disciplines,  and  a top-down  diffusion 
strategy,  collaborative  work  among  the  faculty  was  essential.  For  example,  creating  a vision  statement  and  sharing 
the  innovation-related  experiences  with  other  individuals  in  the  adoption  units  were  helpful  tasks  in  the  seminary. 

The  seminary  formed  an  ad  hoc  committee  to  set  up  a clear  vision  for  technology  and  teaching  at  the 
seminary.  The  committee  developed  a vision  with  consensus  from  all  faculty  members  and  reported  their  findings  to 
the  faculty.  Another  intervention  was  to  arrange  several  learning  events  in  order  to  facilitate  collaborative  work 
among  faculty  members  in  the  seminary.  Through  these  events,  faculty  members  shared  their  ideas  with  other 
faculty  members.  Sharing  among  faculty  was  the  key  activity  for  changing  the  seminary  culture.  These  events 
included  faculty  presentation  day,  faculty  learning  day,  small  group  interests,  brownbag  lunches  and  learning 
sessions.  Also,  through  the  funds  from  the  grant,  many  faculty  took  advantage  of  conference  opportunities  to  gain 
more  knowledge  about  the  appropriate  use  of  technology.  Furthermore,  the  seminary  developed  contacts  with  other 
schools  facing  the  same  issues  and  was  able  to  find  and  demonstrate  good  practices  in  technology  for  theological 
instruction. 

Concluding  remarks 

Information  technology  is  an  effective  means  of  increasing  teaching  and  learning  effectiveness  in  higher 
educational  settings  including  seminaries.  However,  it  must  be  well  planned  and  organized  before  the  project  begins. 
Identifying  clients’  concerns  and  taking  care  of  them  are  an  important  task  of  change  agents  during  an  innovation 


clarification. 
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process.  Setting  up  a vision  statement,  conducting  perception  analysis,  and  preparing  detailed  plans  for  the  project 
would  guarantee  a successful  implementation  of  an  information  technology  project  in  a higher  education  setting. 
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Abstract 

This  paper  looks  at  the  analysis  leading  to  and  design  of  a Web-based  ''hint  system. The  hints  are 
designed  to  help  middle-grades  teachers  participating  in  a mathematics  professional  development  effort 
called  InterMath  achieve  a high  degree  of  success  when  working  open-ended  mathematical  investigations. 
We  discuss  the  issues  and  processes  involved  with  designing  this  system,  paying  special  attention  to  the 
different  types  of  hints  and  the  limitations  of  the  system. 


Background 

InterMath  (http://www.intermath-uga.gatech.edu)  is  a web-based,  technology-intensive  professional 
development  experience  for  middle  school  mathematics  teachers.  The  professional  development  effort  is  aimed  at 
furthering  teachers’  mathematics  knowledge,  providing  experience  with  using  a variety  of  tools  to  promote  learning, 
and  supporting  meaningful  technology  integration  for  middle  school  mathematics  classrooms.  The  model  upon 
which  InterMath  is  based  assumes  that  teachers  who  experience  learning  in  a rich,  exploration-based  environment 
will  more  readily  transfer  these  kinds  of  experiences  to  their  classrooms.  The  centerpiece  of  the  InterMath 
experience  and  website  is  an  extensive  set  of  open-ended  mathematical  investigations  that  can  be  explored  using 
various  technologies  (e.g..  Excel  or  Geometer’s  Sketchpad).  These  investigations  form  the  central  experience  of  the 
fifleen-week  workshop  as  teachers  are  encouraged  to  explore  a particular  set  of  explorations  (e.g.,  triangles  or 
functions),  choose  problems  that  intrigue  them,  work  those  investigations,  and  write-up  the  solutions,  along  with 
extension  activities. 

Unlike  the  traditional  “make  and  take”  professional  development  experiences,  InterMath  does  not  aim  to 
provide  teachers  with  activities  they  can  take  back  to  their  classroom  and  use.  Instead,  InterMath  provides  teachers 
with  opportunities  to  hone  their  mathematics  skills,  reinforcing  both  their  understanding  of  mathematics  and  their 
ideas  about  what  it  means  to  learn  (and,  thus,  teach)  mathematics. 

InterMath  is  in  its  third  year  of  development  and  implementation.  During  the  pilot  workshop  offerings  in 
Year  2,  there  were  two  InterMath  workshops  conducted  by  two  different  faculty  members.  One  workshop  was 
offered  near  Atlanta  as  the  first  course  in  a cohort -based  graduate  degree  program  for  middle  school  teachers.  This 
workshop  was  taught  by  a mathematics  education  professor  who  is  also  one  of  the  developers  of  the  InterMath 
program.  The  other  workshop  was  offered  at  the  University  of  Georgia  campus  by  a professor  of  mathematics.  The 
participants  in  the  UGA  workshop  received  staff  development  credits  for  their  experience,  as  well  as  a stipend. 

The  Problem 

Our  initial  work  with  teachers  indicated  that  one  of  the  challenges  of  InterMath  is  that  the  teachers  are 
uncomfortable  engaging  in  problems  that  they  do  not  feel  confident  in  their  ability  to  finish.  The  teachers  tend  to 
work  the  easier  problems  first  - and  may  not  ever  work  the  problems  they  perceive  as  being  the  most  challenging. 
However,  those  challenging  problems  may  be  the  ones  that  provide  the  greatest  learning  experience.  Our  interviews 
with  teachers  and  classroom  interactions  indicated  that  there  were  multiple  reasons  for  these  problem  selection 
behaviors.  The  most  troubling  reason,  however,  was  an  avoidance  of  mathematics  that  was  viewed  as  difficult. 

To  further  complicate  the  situation,  teachers  in  our  pilot  workshops  were  considerably  more  challenged  by 
the  variety  and  uses  of  technology  required  for  success  in  the  workshop  than  we  had  anticipated.  We  expected 
teachers  to  be  able,  by  the  end  of  the  workshop,  to  use  Geometer’s  Sketchpad,  NuCalc,  and  Excel,  as  well  as  to  be 
able  to  create  and  publish  a webpage  using  a WYSIWYG  editor  and  FTP  software.  While  most  of  the  teachers  had 
already  completed  a basic  technology  workshop  series,  they  had  little  to  no  experience  using  technology  to  support 
learning  in  a particular  content  area.  Further,  many  had  little  hands-on  experience  in  using  the  computers  with  their 
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students  to  support  mathematics  of  any  kind.  In  short,  InterMath  requires  teachers  to  be  comfortable  with  many 
pieces  of  technology,  as  well  as  to  develop  a concrete  image  of  what  technology  can  help  them  accomplish  in  their 
content  area. 

It  was  determined  that  these  problems  were  serious  enough  to  warrant  additional  support  structures  being 
created  to  support  teachers  in  successfully  expanding  their  own  content  knowledge  as  well  as  their  pedagogical 
content  knowledge.  To  this  end,  it  was  decided  that  some  form  of  online  help  or  hint  system  needed  to  be  developed 
for  the  InterMath  problem  set  that  would  allow  teachers  to  be  more  successful  in  their  interactions  with  our 
materials. 

This  paper  describes  the  design  process  to  date  and  discusses  the  implications  of  our  analysis  phase  on  the 
kind  of  hint  systems  we  are  working  on.  To  date,  we  have  completed  the  analysis  system  and  have  implemented  a 
rapid  prototyping  (Tripp  & Bichelmeyer,  1990)  approach  to  help  explore  the  potential  of  each  option.  Further,  we 
have  completed  an  initial  paper-based  design  of  the  “constructopedia,”  a separate  aspect  of  the  InterMath 
scaffolding  system  that  will  be  discussed  later  in  this  paper 

An  Analysis  of  the  Problem 

The  analysis  portion  of  the  hint  system  project  occurred  during  Winter  and  Spring  2001 . Data  were 
collected  in  the  form  of  conversations  - both  live  and  via  email  - with  a variety  of  subject-matter  experts  who  are 
participants  in  the  InterMath  project,  as  well  as  observations  and  informal  interviews  of  teachers  involved  in  the 
pilot  workshop.  The  discussions  among  team  members  were  a rich  source  of  data  for  thinking  about  what  we  value 
in  the  learning  process  and  what  matters  in  how  we  interact  with  the  learners. 

Learner  Analysis 

Information  was  collected  during  each  InterMath  session  through  both  formal  and  informal  means. 
InterMath  instructors  and  facilitators  informally  observed  activities  and  comments  while  teaching  and  while 
circulating  to  check  on  participants'  progress  and  to  provide  them  with  technological  and  mathematical  assistance 
when  needed.  Formally,  at  least  one  person  was  designated  as  the  class  observer  each  session  and  took  detailed 
notes  regarding  events  that  occurred  in  class,  including  participants'  actions,  comments,  technical  problems, 
successes,  and  mathematical  difficulties.  In  addition,  many  participants  were  interviewed  at  the  conclusion  of  the 
workshops.  The  informal  and  formal  observations,  along  with  transcripts  of  the  interviews,  provided  a wealth  of 
information  regarding  the  difficulties  the  workshop  participants  had  experienced  and  how  the  InterMath  experience 
could  be  changed  to  better  benefit  teachers  in  future  InterMath  workshops. 

As  anticipated,  an  analysis  of  the  data  provided  much  insight  into  the  InterMath  experience  from  the 
perspective  of  the  teachers  enrolled  in  the  class.  While  every  participant  said  that  he  or  she  would  recommend  or 
have  already  recommended  InterMath  to  other  teachers,  their  comments  provided  valuable  infonnation  about 
problems  they  experienced  during  the  course.  Some  teachers  noted  how  time  -consuming  problems  can  be  and  how 
many  problems  on  the  InterMath  website  cannot  be  used  with  their  classes  without  modifications.  However,  two 
other  issues  were  mentioned  repeatedly  as  the  predominate  difficulties  teachers  faced  in  the  workshop  - difficulties 
with  the  technologies  being  used  and  problems  with  the  mathematics  itself.  This  paper  addresses  the  second  of  these 
two  issues. 

A number  of  the  participants  expressed  frustration  over  the  difficulty  of  many  InterMath  problem  sets. 

Some  of  the  participants  noted  that  they  do  not  have  strong  backgrounds  in  mathematics,  while  others  described  the 
mathematical  rustiness  that  has  occurred  while  teaching  middle  grades  math. 

'‘There's  some  of  the  math  that,  right  now,  I don  7 think  I have  down  pat.  ” 

'T  don't  have  a very  strong  math  background,  so  I struggle  with  a lot  of  the  problems....  I mean  it's  been  a 
long  time  since  I've  been  in  a math  class.  I know  this  is  a graduate  class,  but  I've  only  been  teaching  7^ 
grade  math  and  there's  only  so  much  that  we  do-  so  the  vocabulary  and  all  is  unfamiliar  to  me.  ” 

"A  few  of  them  it's  going  to  take  some  brushing  up  on  - because  as  a grade  teacher,  there  are  a few 
things  that  we  don't  hit  in  7^*  grade.  ” 

"That's  the  hardest  part.  Sort  of  bringing  out  the  true  math  that's  behind  it.  Because  / don 't  know  if  I know 
enough  to  do  it.  " 
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Many  participants  noted  in  their  interviews  how  helpful  the  assistance  from  course  facilitators  was  when 
technological  or  mathematical  impasses  were  experienced. 

Brian  and  Shannon  have  been  wonderful  They've  talked  me  through  so  many  [problems].  ” 

'7  think  the  very  best  part  is  probably  the  instructors  - the  people  that  are  helping  out....  Because  they're 
so  understanding  and  they  don 't  act  like  you're  stupid  when  you  don 't  know  how  to  do  something.  ** 

These  comments  alone  give  sufficient  cause  to  consider  creating  some  type  of  mathematical  scaffolding 
system  for  the  InterMath  workshop  InterMath  workshop,  as  InterMath  personnel  are  not  always  on  hand  when  the 
teachers  are  working  on  their  problem  sets. 

In  describing  how  their  ideal  math  classrooms  would  operate  and  how  they  believe  students  learn,  some 
teachers  revealed  the  importance  they  place  on  scaffolding  their  own  students,  further  prompting  InterMath 
designers  to  consider  the  implementation  of  a hint  system. 

“I  can  steer  them  and  give  them  hints.  And  it  also  helps  if  they're  going  in  a completely  wrong  direction.  I 
can  hear  that  too  and  steer  them  the  right  way.  ” 

“I  teach  6^^  grade  — so  some  are  way  over  their  heads  and  some  are  more  appropriate.  So  I kind  of  have  to 
give  them  a little  bit  of  guidance  and  they  know  what  they're  looking  for.  If  you  just  let  them  go  blindly, 
then  they're  gonna,  they're  gonna  get  lost. 

Based  on  the  information  gathered  throughout  the  course  of  the  InterMath  workshop  and  post-workshop 
interviews  with  the  participants,  it  seemed  obvious  that  a hint  system  would  be  a beneficial  addition  to  this 
professional  development  endeavor.  The  goals  of  this  system  would  be  to  encourage  teachers  to  select  problems  that 
are  more  difficult  than  those  hey  might  otherwise  attempt,  assist  them  in  furthering  their  mathematical  thinking  and 
understanding,  and  decrease  the  frustration  that  can  occur  when  they  come  to  a point  where  no  more  progress  can  be 
made  without  some  type  of  assistance. 

Hint  System 

Because  the  teachers  needed  more  support  than  could  be  offered  in  the  workshop  and  because  our  team 
could  not  be  available  all  the  time,  we  decided  that  an  online  system  to  support  teacher  learning  anytime,  anywhere 
was  a critical  element  of  the  support  system.  To  this  end,  we  determined  that  developing  some  form  of  scaffolding 
system  would  be  appropriate.  By  adding  a “hint”  to  the  more  complicated  problems,  we  may  be  able  to  aid  teachers 
in  persevering  to  become  more  successful  problem-solvers.  These  hints  may  focus  on  the  mathematics  or  on  the  use 
of  technology  to  support  the  mathematics.  We  are  striving  to  balance  between  providing  too  little  and  too  much 
support  to  the  teachers. 

One  of  our  primary  concerns  in  considering  this  support  system  has  been  that  it  may  inadvertently  limit  the 
cognitive  growth  potential  for  users  of  InterMath.  Based  on  our  initial  field  test  of  the  hints,  this  concern  is  well- 
founded.  Teachers  in  our  test  group  clicked  on  the  hint  before  they  attempted  to  solve  the  problem  on  their  own.  The 
static  hint  system  cannot  provide  the  cooperative  problem-solving  effort  between  the  learner  and  a more 
knowledgeable  other  that  defines  scaffolding  (Collins,  Brown,  & Newman,  1989),  nor  can  it  anticipate  exactly  when 
the  learner  may  require  assistance.  Learning  in  this  environment  comes  from  engagement  in  the  problem-solving 
process,  therefore  the  learners  need  to  be  actively  involved  in  the  process  in  order  to  benefit.  If  they  click  on  the  hint 
before  struggling  on  their  own,  the  potential  of  the  learning  experience  may  be  significantly  weakened.  Further,  the 
hint  system  needed  to  push  thinking  forward  and  promote  metacognitive  engagement  rather  than  provide  a crutch 
that  eases  the  cognitive  involvement  of  the  learners  and  results  in  over-reliance  in  the  system  over  time  (Kao  & 
Lehman,  1997). 

Another  issue  that  arose  in  the  analysis  phase  was  the  purpose  of  the  hints.  We  identified  three  distinctly 
different  purposes  hints  might  serve.  The  team  also  explored  some  of  the  arguments  against  developing  an  online 
scaffolding  system.  Because  of  these  concerns,  we  were  working  toward  developing  a system  that  attempted  to  use 
only  questions  or  tools  that  the  learners  will  be  able  to  use  on  other  problems  and  in  other  ways  (Polya,  1981).  In 
this  way,  we  aimed  to  develop  a tool  that,  while  static,  might  support  the  development  of  transferable  knowledge  for 
the  learner. 
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The  arguments  against  the  hint  system  arose  from  our  beliefs  about  learning  and  our  previous  work  with 
teachers  supporting  students  in  computer-based  environments  (e.g.,  Hawley  & Duffy,  1998).  In  addition  to  the 
concern  that  the  hint  system  might  become  a crutch,  we  were  further  concerned  about  the  ability  of  an  online 
scaffolding  system  to  provide  the  kind  of  help  the  learner  needed  at  the  time  he  needed  it  - his  zone  of  proximal 
development  (ZPD)  (Vygotsky,  1978).  After  all,  the  online  system  cannot  sense  where  the  learner  is  within  a 
problem,  what  the  learner’s  mathematical  knowledge  is,  or  whether  the  problem  is  a technical  one  (e.g.,  not  knowing 
how  to  construct  a shape  or  write  a formula)  or  a mathematical  one  (e.g.,  not  knowing  how  to  approach  a problem). 

It  was  the  “inside  help”  that  we  felt  the  system  could  not  provide  (Polya,  1981).  That  is,  the  computer  is  not 
sensitive  or  “intelligent”  enough  to  provide  the  learner  with  the  kinds  of  questions  or  suggestions  that  may  have 
occurred  to  the  student  - and  those  are  the  questions  that  move  the  student  to  new  levels.  This  kind  of  help  assumes 
that  there  are  some  general  ideas  that  can  guide  the  learner  from  one  problem  to  another.  This  is  in  sharp  contrast  to 
“outside  help”  which  should  be  treated  as  a last  resort  (Polya,  1981). 

Additionally,  the  use  of  a static  hints  system  in  an  online  setting  requires  the  student  to  be  responsible  for 
“fading”  the  scaffolding.  It  is  unclear  that  learners  can  effectively  self-monitor  to  determine  where  they  are  in  their 
own  ZPD.  It  is  clear,  however,  that  fading  is  a critical  part  of  the  scaffolding  process  (Kao  & Lehman,  1997;  Oliver 
& Hannafm,  2000).  Therefore,  finding  ways  to  provide  support  without  over-supporting  became  a theme  in  our 
development  process. 

Finally,  the  static  hints  system  is  problematic  in  that  scaffolding  is  often  considered  an  interactional 
approach  (Driscoll,  1994).  It,  ideally,  provides  for  an  interaction  between  two  people  that  can  evolve  and/or  fade 
over  time.  The  reciprocal  teaching  research  (e.g.,  Palincsar  & Brown,  1984;  Brown  & Palincsar,  1982)  provides  a 
compelling  description  of  a scaffolding  system  that  exemplifies  this  approach.  Our  design  team  considered  how  or  if 
the  hint  system  could  serve  as  a scaffolding  device  without  this  kind  of  teacher-learner  interaction. 

While  compelling,  the  arguments  against  the  hints  system  did  not  provide  alternatives  for  supporting 
teachers  in  attempting  or  completing  the  problems  that  offered  the  most  promise  for  their  mathematical  development 
- the  “hard”  math  problems.  Using  Bruner’s  view  of  scaffolding,  we  wanted  to  protect  our  learners  from  frustration 
(Bruner,  1981).  Therefore,  three  different  approaches  and  views  of  the  content-oriented  hints  were  considered:  hints 
that  aimed  to  keep  the  teachers  motivated,  hints  that  aimed  to  provide  generalizable  strategies  for  the  learners,  and 
hints  that  modeled  expert  thinking  for  the  teachers.  The  first  kind  of  hint  sought  to  provide  scaffolding  that 
supported  the  teachers  through  the  rough  spots  and  helped  them  feel  successful.  The  second  and  third  kinds  of  hints 
served  the  purpose  of  trying  to  help  the  teachers  become  more  expert-like  in  their  approaches  to  mathematical 
problem  solving. 

Motivational  Hints 

Occasionally  teachers  in  the  InterMath  course  became  frustrated  after  working  on  a problem  for  an 
extended  period  of  time  without  making  any  headway.  Sometimes  the  teachers  would  give  up  on  problems  such  as 
these  and  move  on  to  less  challenging  ones.  As  InterMath  is  a professional  development  environment,  one  in  which 
teachers  are  encouraged  and  expected  to  develop  and  expand  their  mathematical  understanding,  this  "avoidance 
behavior"  played  a large  role  in  our  decision  to  develop  the  hint  system.  We  believed  that  such  a system  would 
motivate  the  teachers  to  choose  and  persevere  with  more  challenging  problem  sets. 

In  describing  general  principles  for  motivating  people.  Ford  (1992)  describes  the  importance  of  providing 
indirect  facilitation  for  goal  attainment,  rather  than  controlling  learners'  actions.  In  our  case,  the  hint  system  would 
be  that  facilitation,  which,  for  some  teachers,  would  be  the  determining  factor  for  whether  they  would  choose  a more 
challenging  problem  over  a simpler  one.  This  is  particularly  relevant  because  problems  are  noiassigned  to  teachers 
in  the  InterMath  course;  teachers  choose  which  problems  they  wish  to  explore,  giving  them  more  ownership  of  the 
process. 

Along  with  the  notion  of  learner  control  comes  the  idea  of  self-efficacy,  or  learners' judgments  about 
whether  or  not  they  can  succeed  at  specified  tasks  (e.g.,  Bandura,  1989;  Pajares,  1996).  It  was  our  hope  that 
providing  a hint  system  as  scaffolding  would  increase  learner's  feelings  of  self-efficacy,  increasing  their  likelihood 
of  choosing  more  complex  problems  and  expanding  their  mathematical  horizons. 

Self-efficacy  is  closely  tied  to  the  concept  of  personal  agency  beliefs  (Ford,  1992).  Ford  describes  these 
beliefs  as  the  evaluative  thoughts  comparing  a desired  outcome  (successful  completion  of  a problem)  to  an 
anticipated  outcome  (successfully  and  easily  completing  a simple  problem,  or  possibly  failing  to  correctly  complete 
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a more  difficult  one).  Ford  believes  that  in  addition  to  being  able  to  achieve  goals,  learners  need  to  believe  in  their 
abilities  to  achieve  them. 

Personal  agency  beliefs  are  often  more  fundamental  than  the  actual  skills  and  circumstances  they  represent 
in  the  sense  that  they  can  motivate  people  to  create  opportunities  and  acquire  capabilities  they  do  not  yet 
possess,  (page  251) 

We  hoped  that  providing  the  hint  system  as  scaffolding  would  increase  learners’  estimations  of  their  own 
abilities  to  solve  difficult  mathematics  problems,  improve  their  success  rate  when  they  do  choose  these  problems, 
and  provide  a "safety  net"  in  case  they  do  become  stuck  on  the  problems.  We  believe  that  a motivation-guided 
scaffolding  system  will  ultimately  help  teachers  to  achieve  the  ultimate  goal  of  InterMath  - improving  their 
understanding  of  mathematics. 

Table  1:  Motivational  Hints  for  **Penningfor  Pony*’  Problem 

Problem:  To  make  a pen  for  his  new  pony,  Ted  will  use  an  existing  fence  as  one  side  of  the  pen.  If  he 
has  ninety -six  meters  of  fencing,  what  are  the  dimensions  of  the  largest  rectangular  pen  he  can  make? 

(Source:  Mathematics  Teaching  in  the  Middle  School,  Nov-Decl994). 

Hint  1 : Is  it  possible  to  relate  the  amount  of  fencing  in  terms  of  the  side  length  of  one  side  of  fence? 

Hint  2:  What  equations  can  be  written  to  relate  the  side  lengths  created  by  the  fencing? 


Generalizable  Hints 

Because  of  our  team  discussions,  we  decided  to  explore  multiple  forms  of  hints  for  this  system.  To  this  end, 
Polya  (1957)  provided  the  basis  for  our  thinking  in  the  development  of  the  second  kind  of  hints-  those  that  support 
the  development  of  transferable,  generalizable  problem-solving  strategies.  Polya  identified  a four-step  problem- 
solving process  that  includes  a number  of  substeps.  The  four  key  steps  are:  (1)  Understand  the  problem;  (2)  Devise  a 
plan;  (3)  Carry  out  the  plan;  and  (4)  Look  back.  In  reviewing  the  process  and  reflecting  on  the  kinds  of  challenges 
the  teachers  we  had  worked  with  thus  far  faced,  we  determined  that  providing  hints  that  helped  the  learners  work  in 
phases  one  and  two  of  Polya’s  work  would  be  most  beneficial.  To  this  end,  a set  of  hints  that  provided  both  the 
strategy  (e.g.,  “Can  you  restate  the  problem?”)  and  a specific  hint  for  the  problem  of  interest  (e.g.,  “Before  you 
answer  the  given  problem,  think  about  the  definitions  and  formulas  of  circumference  and  area.  On  what  part  of  each 
circle  do  you  really  need  to  focus?  How  do  these  parts  relate  to  each  other?”).  The  goal  was  to  support  the  learner 
through  a combination  of  macro-  and  micro -level  scaffolding  (Guzdial,  1994). 

The  goal  of  transfer  in  this  static  environment  is  a difficult  one  to  achieve.  Research  has  indicated  that  the 
greatest  opportunity  for  transfer  involves  an  active  system  in  which  the  hints  become  more  specific  as  needed  (e.g., 
Bransford,  Brown,  & Cocking,  1999;  Campione  & Brown,  1987).  This  graduated  prompting  provides  a way  to  more 
actively  engage  the  learner  as  a partner  in  the  scaffolding  system.  Due  to  the  limitations  of  a static  hints  system,  we 
could  not  offer  the  progressive  drilling  into  an  area  that  a teacher  could,  though  we  did  explore  that  option.  Instead, 
we  chose  to  provide  a set  of  hints  rather  than  one  that  provided  the  learner  with  both  the  general  hint  and  the 
specific.  Our  intention  would  be  for  the  learner  to  be  self-regulating  in  choosing  to  use  both  the  hints  system  itself 
and  the  specific  hint. 

Table  2:  Generalizable  Hints  for  “Penning  for  Pony  " Problem 

Problem:  To  make  a pen  for  his  new  pony,  Ted  will  use  an  existing  fence  as  one  side  of  the  pen.  If  he 
has  ninety -six  meters  of  fencing,  what  are  the  dimensions  of  the  largest  rectangular  pen  he  can  make? 

(Source:  Mathematics  Teaching  in  the  Middle  School,  Nov-Decl994). 

Strategy  1:  What  are  the  data?  What  is  the  condition? 

Hint  1 : What  does  the  ninety-six  meters  of  fence  tell  you  about  the  rectangular  pen? 

Strategy  2:  Go  back  to  the  definitions. 

Hint  2:  How  do  you  find  the  area  of  a rectangle?  How  do  you  find  the  perimeter  of  a rectangle?  How 
are  the  area  and  perimeter  of  a given  rectangle  related  to  each  other? 


421 


910 


Model  Hints 


Wood,  Bruner,  and  Ross  (1976)  define  the  expert’s  role  in  scaffolding  as  directing  and  maintaining  the 
learner’s  attention,  while  also  modeling  the  task  and  highlighting  the  critical  features  of  that  task.  Our  third  approach 
to  hints  builds  from  that  notion.  It  is  well-documented  that  there  are  fundamental  differences  in  the  way  experts 
organize  knowledge  and  the  way  novices  organize  that  same  knowledge  (e.g.,  Bransford,  Brown  & Cocking,  1999). 
Experts  tend  to  organize  thinking  and  strategies  around  core  concepts  whereas,  more  novice  thinkers  tend  to  exhibit 
signs  of  more  linear  and  procedural  understandings.  Further,  experts  are  better  able  to  identify  appropriate  instances 
for  applying  theories  and  procedures  to  problem-solving  instances.  Therefore,  if  the  scaffolding  system  were  to 
promote  the  development  of  more  expert  thinking  about  mathematics,  one  possible  approach  might  be  to  provide  a 
rich  set  of  problems  for  the  learners  to  solve  and  provide  models  of  expert  approaches  to  solving  those  problems. 

Modeling  has  been  shown  to  be  an  effective  means  for  supporting  learning  (e.g.,  Palincsar  & Brown,  1984; 
Schoenfeld,  1991).  It  provides  an  example  for  learners  to  take  and  adapt  for  their  own  purposes.  The  limitation  of 
the  modeling  method  is  that  it,  too,  relies  on  learner  self-regulation.  If  the  learner  does  not  engage  in  reflective 
adoption  and  adaptation  of  the  modeled  example,  that  learner  has  learned  only  a prescriptive  task  rather  than  a 
generalizable  approach.  With  a hint  system  of  this  kind,  our  goal  would  be  to  support  the  learner  to  reflect  on  what 
was  important  in  a situation.  The  instructor  may  need  to  spend  time  engaging  learners  in  reflective  activities  to  gain 
the  skill  necessary  to  do  this.  However,  this  may  be  a powerful  way  to  help  move  middle  grades  teachers, 
notoriously  underprepared  in  their  content  areas  (e.g.,  SREB,  1998),  toward  developing  more  expert  levels  of 
content  understanding. 

Table  3:  Model  Hints  for  ‘‘Penning  for  Pony”  Problem 

Problem:  To  make  a pen  for  his  new  pony,  Ted  will  use  an  existing  fence  as  one  side  of  the  pen.  If  he 
has  ninety -six  meters  of  fencing,  what  are  the  dimensions  of  the  largest  rectangular  pen  he  can  make? 

(Source:  Mathematics  Teaching  in  the  Middle  School,  Nov-Decl994). 

Hint  1:  What  if  we  built  a similar  figure  on  each  side  of  the  fixed  wall? 


Hint  2:  Use  the  algebraic  mean  - geometric  mean  inequality  to  find  the  greatest  possible  area,  then 
use  that  information  to  determine  the  length  of  each  side. 

(Hint  2 also  includes  working  of  the  mathematical  equations.) 
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The  Design  Process 


Our  design  process  ran  parallel  to  our  evaluation  process.  We  began  by  looking  at  existing  systems  to  see 
how  they  provided  support  for  learners.  These  systems  included  the  support  system  in  the  Knowledge  Integration 
Environment  (KIE)  (Slotta  & Linn,  2000),  as  well  as  EMILE  (Guzdial,  1994)  and  CSILE  (Scardamalia  et  al.,  1992). 
These  tools  provided  very  different  approaches  to  scaffolding  learning,  ranging  from  static  to  very  dynamic  and 
including  various  levels  of  teacher/knowledgeable  other  support  within  the  system. 

The  tools  that  we  reviewed  ranged,  also,  from  technologically  complex  to  those  that  were  relatively  simple. 
In  the  end,  the  decisbn  to  follow  a simpler  path  was  tied  to  two  issues:  budget  and  time.  The  enormous  undertaking 
of  developing  a dynamic  support  system  was  outside  the  scope  of  our  efforts  and  was  not  included  in  the  budget. 
Further,  InterMath  is  a multifaceted  system  that  includes  communication  tools,  a dictionary,  materials  for 
instructors,  and,  of  course,  the  investigations.  Because  each  of  these  pieces  requires  support,  the  need  to  keep  the 
scaffolding  system  to  a manageable  size  was  considerable.  Further,  the  InterMath  team  recognized  that  teachers  in 
need  of  more  specialized  support  could  contact  an  InterMath  team  member  or  could  use  the  communications  tools  to 
pose  questions  to  other  members  of  the  community.  The  trade-off  with  this  was  the  loss  of  momentum  caused  by  the 
delayed  communication  in  an  asynchronous  environment.  However,  if  we  paired  a just-in-time  scaffolding  system 
with  these  other  tools,  we  felt  we  could  offer  support  for  the  learners  that  would  be  meaningful. 

The  most  critical  decision  made  as  a result  of  our  process  was  the  decision  to  split  the  hints  system  into  two 
distinct  pieces.  One  piece  would  be  the  hints  as  discussed  in  the  analysis  section.  The  other  part  would  be  the 
“constructopedia,”  which  is  a system  to  support  teachers  in  completing  mathematical  constructions  using 
Geometer’s  Sketchpad.  While  making  constructions  in  this  program  is  still  a mathematical  process,  it  is  the  point 
where  mathematical  concepts  intertwine  with  technology.  The  support  that  teachers  struggling  with  the  technology 
needed  was  not  the  same  kind  of  support  they  needed  to  conceptually  understand  the  problem.  The  constructopedia 
borrows  heavily  from  the  Lego  approach  to  instruction  - teachers  are  provided  with  pictures  that  show  the 
construction  being  put  together.  They  are  provided  with  minimal  text  as  well  because  our  rapid  prototyping  process 
indicated  that  the  images  alone  were  too  confusing.  The  goal  of  the  constructopedia  is  to  provide  teachers  with  the 
support  they  need,  while  forcing  them  to  remain  engaged  in  the  mathematics  of  the  construction. 

A final  portion  of  the  design  phase  has  been  the  identification  of  which  investigations  need  hints  and  what 
kinds  of  hints  they  might  need.  This  process  was  begun  by  having  a new  graduate  student  whose  mathematical 
background  was  similar  to  our  target  audience’s  work  several  of  the  investigations  to  identify  places  where  they 
became  difficult  as  well  as  ways  she  was  able  to  overcome  those  difficulties.  This  provided  a beginning  guide  for 
the  hints  system.  However,  her  activity  on  the  project  team  and  in  graduate-level  mathematics  education  courses 
quickly  moved  her  beyond  the  mathematical  content  knowledge  of  our  target  audience.  Therefore,  further  work  to 
identify  investigations  in  need  of  support  will  need  to  be  done  a different  way. 

The  Development  Process  & Future  Work 

The  analysis  and  design  phases  employed  three  instructional  design  strategies:  1)  having  a person  similar  to 
our  target  audience  work  through  investigations  to  identify  potential  problems  and  discuss  how  those  complications 
affect  the  successful  completion  of  the  problem;  2)  reviewing  existing  computer-based  scaffolding  systems  to 
explore  the  characteristics  and  possibilities  of  effective  systems;  and  3)  working  with  SMEs  on  our  team  to 
determine  how  to  translate  face-to-face  questioning  strategies  to  static  online  strategies.  The  development  phase 
introduces  a fourth  key  strategy  - rapid  prototyping. 

Both  the  constructopedia  and  the  hints  system  itself  have  gone  through  multiple  iterations.  During  the  pilot 
courses,  we  were  able  to  bring  in  different  kinds  of  hints  and  test  them  out  with  members  of  our  target  audience. 

This  provided  extremely  valuable  feedback  as  we  were  able  to  put  the  hints  system  to  work,  watch  the  users  interact 
with  it,  and  talk  to  them  about  their  reactions  to  it.  Interestingly,  in  the  audition  of  the  hints  strategies,  the 
participants  claimed  that  the  hints  were  not  useful,  despite  having  relied  on  them  to  complete  their  problems. 

We  are  currently  involved  with  the  development  stages.  There  are  still  obstacles  to  overcome,  including 
making  the  final  decisions  about  which  approach  to  the  hints  system  to  adopt.  There  are  very  practical 
considerations  that  must  be  weighed  in  this  decision.  For  example,  we  must  choose  a hints  system  that  members  of 
our  team  can  create  and  refine  with  little  SME  intervention.  From  this  perspective,  the  generalizable  hints  are  a 
strong  contender.  We  must  also  choose  the  system  that  offers  the  greatest  level  of  promise  and  that  is  most 
consistent  with  our  overall  approach.  From  this  perspective,  the  model  hints  are  more  appropriate.  At  this  point,  the 
motivational  hints  have  been  left  behind  because  our  initial  evaluation  showed  that  participants  found  them  less 
helpful  than  the  others. 
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Another  critical  obstacle  is  determining  which  investigations  need  hints  and  what  kinds  of  problems 
teachers  might  have  with  them.  This  is  problematic  because  all  of  our  team  members  have  either  extremely  high 
mathematical  ability,  or  mathematical  ability  that  is  too  limited.  To  address  this  problem,  we  have  considered  a 
number  of  approaches  including  conducting  more  teacher  observations  during  our  next  workshop  offering  and  using 
our  best  guesses  to  find  problematic  investigations. 

Conclusion 

In  short,  we  see  the  use  of  a web-based  hints  system  as  being  a potentially  valuable  tool  for  supporting 
teachers  in  tackling  complex  math  problems.  We  recognize  that  there  are  limitations  with  this  plan,  but  see 
combining  the  just-in-time  help  of  a static  system  with  the  asynchronous  support  of  a wider  community  as  a viable 
option  for  promoting  the  development  of  mathematical  thinking.  We  are  currently  involved  in  a rapid-protoyping 
approach  to  find  appropriate  ways  for  supporting  teacher  development. 
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Abstract 

The  paper  conceptually  elaborates  the  traditional  concept  of  expert  performance  by  Glaser  et.  al,  and 
discusses  a series  of  attributes  for  supporting  or  nurturing  an  environment  for  expertise.  Then  an 
alternative  point  of  view  from  a standpoint  of  activity  theory  (social  constructive  theory)  will  be  taken  to 
see  how  one  (instructional  designer)  might  support  an  expertise  in  the  areas  of  designing  technology-based 
performance  support  systems. 

Introduction 

With  the  increased  demand  and  interest  in  electronic  performance  support  systems  (EPSS),  particularly  for 
supporting  knowledge-based  problem  solving  expertise  in  the  information  age  (Gustafson,  2000;  Dickelman,  2000; 
Kasvi  & Vartiainen,  2000),  instructional  designers  are  facing  a new  challenge  designing  a system  that  could  deliver 
(or  transfer)  needed  knowledge  (expertise  in  a particular  knowledge  domain)  to  leamers/users,  because  we  do  not 
really  know  what  it  is  that  learners  need  to  learn  to  become  effective  problems  solvers  within  the  context  that 
learning  (or  performance  require)  occurs  (Backler,  1988).  The  questions  include  “What  skilled/experienced  problem 
solvers  (or  expert  performer)  go  about  solving  a problem  (a  task)?”  and  “What  kinds  of  knowledge  they  make  use  of 
when  doing  so?”  In  short,  we  do  not  really  know  what  expertise  is  required  for  individual  learners  to  perform 
particular  tasks  within  certain  conditions,  and  if  we  do  not  know  this  then  how  do  we  design  an  effective  system  to 
delivery  them. 

However,  an  alternate  view  of  cognition  has  been  developing  over  the  past  three  decades  or  so:  expertise 
researches  from  cognition  psychology.  Other  researches  that  influence  instructional  design  are  so  called:  situated 
cognition,  distributed  cognition,  activity  theory,  and  other  dimensions  relating  to  social-cultural  perspectives  have 
been  brought  into  attention  by  socio -culture  psychology.  Fundamentally,  these  notions  stress  the  individual  minds 
(expertise  and  mental  model),  the  environment  (or  authentic  contexts),  and  the  tight  interrelations  among  them 
(Hung,  Koh,  & Chua,  2000).  In  this  paper,  the  author  will  focus  on  activity  theory  only. 

Instructional  designers  are  designing  instructional  environments  both  to  understand  the  improvement  of 
performance  and  to  further  define  theories  applicable  to  the  design  of  conditions  for  learning.  Therefore,  the 
objective  for  the  EPSS  designer  is  for  more  people  (novices  and  intermediate  learners)  to  attain  competence  at 
higher  levels  (expert)  than  ever  before.  Given  the  notion  that  different  performance  may  require  different  support, 
the  EPSS  designer  needs  to  understand  what  knowledge  expertise  is  required,  how  expertise  (knowledge;  skills)  was 
acquired,  for  different  individuals  and  in  what  conditions.  To  reach  their  goal,  EPSS  designers  began  to 
accommodate  changes  in  theory  and  practice  and  did  so  in  a way  that  added  value  to  the  discipline.  Two  disciplines 
that  increasingly  influenced  instructional  design  are  researches  on  expertise  from  cognitive  psychology  and  activity 
theory  from  socio -culture  psychology. 

In  this  paper  on  the  literature,  the  author  will  first  give  an  overview  of  EPSS  then  review  two  areas  of 
studies  related  to  (advanced)  EPSS  design.  One  is  expertise  from  cognitive  psychology;  the  other  is  activity  theory 
proposed  by  socio -culture  psychology. 

What  is  EPSS  and  why  should  instructional  designers  be  interested  in  it? 

What  is  an  Electronic  Performance  Support  System  (EPSS)?  “EPSS”  is  still  a relatively  new  term  in  the 
field  of  corporate  training  and  instructional  design,  having  been  first  introduced  by  Gery  in  1989.  An  EPSS  refers  to 
any  electronic  integrated  system  or  infrastructure  that  can  provide  access  to  information  and  tools  to  enable 
individuals  to  achieve  a high  level  of  performance  in  a short  amount  of  time  and  with  a minimum  of  support  from 
other  people.  According  to  EPSS  InfoSite,  an  EPSS  can  also  be  described  as  any  computer  software  program  or 
component  that  improves  worker  performance  by 
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1.  reducing  the  complexity  or  number  of  steps  required  to  perform  a task, 

2.  providing  the  performance  information  an  employee  needs  to  perform  a task,  or 

3.  providing  a decision  support  system  that  enables  a worker  to  identify  the  action  that  is  appropriate 
for  a particular  set  of  conditions. 

Currently,  corporations  are  benefiting  from  the  implementation  and  usage  of  integrated  electronic  systems 
(Raybould,  1995;  Kasvi,  2000;  Gustafson,  2000).  For  example,  a large  corporation  may  combine  its  many  electronic 
tools  (such  as  databases,  word  processing,  e-mail,  and  calendars)  into  an  EPSS  to  facilitate  ease  of  access  and  usage. 
By  using  an  EPSS,  information  or  tools  are  more  easily  accessible,  thus  increasing  the  workers’  efficiency  (Gery, 
1991;  Kavat,  1997;  Raybould,  1990). 

When  the  workplace  becomes  increasingly  complex,  quick,  and  accurate  the  speed  and  accuracy  of 
information  processing  becomes  a competitive  necessity,  especially  in  the  technology-rich-rapidly  changing 
information  age.  In  the  information  age,  even  experts  may  have  to  struggle  to  maintain  their  level  of  performance 
(Winslow  and  Bramer,  1994).  With  advances  in  technology,  it  becomes  possible  to  provide  a variety  of  new  forms 
of  support  to  aid  worker  performance,  promote  workers’  satisfaction  (confidence),  shorten  the  process  of 
transforming  a novice  into  an  expert,  and  potentially  reducing  the  amount  of  training  they  require. 

The  definition  of  EPSS  varies.  Among  other  terms  used  are:  performance  support,  online  performance 
support,  performance  support  tool,  performance  support  engineering,  performance  centered  design,  and  Embedded 
Support  (Gustafson,  2000).  The  definition  of  EPSS  here  tends  to  be  broad  and  encompassing.  Thus,  it  includes 
everything  from  the  simple  database  that  provides  information  to  workers  (for  example  item  price  and  customer 
information)  to  EPSSs  for  the  complex  tasks  that  involve  complex  cognitive  skills  (such  as  air  traffic  control 
systems  based  on  expert  systems  and  artificial  intelligence). 

The  elements  and  characters  of  what  an  EPSS  constitutes  also  vary.  Gery  (1991)  included  four  elements:  an 
information  base,  advisor,  tutorials,  and  tools  to  assist  the  user.  In  the  past  decades,  EPSS  design  has  matured;  it  is 
not  limited  to  Gery’s  four  elements.  “Information  bases  may  include  multiple  knowledge  sources;  advising  may  take 
on  features  of  expert  systems  or  artificial  intelligence;  tutorials  may  be  extensive  and  contain  contextual  multimedia 
instruction;  and  the  tools  have  become  more  sophisticated  (Gustafson,  2000;  p.  37).” 

However,  as  an  instructional  designer,  we  have  every  reason  to  believe  these  trends  will  continue,  the 
demand  of  creating/developing  EPSSs  will  grow,  and  the  design  of  EPSS  will  not  become  less  complex  and  less 
challenging  as  the  rate  of  change  keeps  accelerating. 

What  are  these  studies  on  expertise  about  and  why  should  instructional  designers  be  interested  in  the  theory 
of  expertise? 

“It  is  likely  true  that  all  humans  have  some  form  of  mental  model  or  conceptualization  of  the  operation  and 
function  of  any  piece  of  technical  equipment  (Johnson,  1988).”  Whether  accurate  or  not,  understanding  how  humans 
learn  and  process  knowledge  (expertise)  helps  instructional  designers  in  developing  effective  leaming/support 
systems.  As  mentioned  before,  EPSS  designers  are  interested  in  knowing  what  expertise  is  required  for  individual 
learners  to  perform  particular  tasks  within  what  conditions  and  then  how  do  we  delivery  them  as  instructional 
designers.  In  the  past  few  decades,  cognitive  psychology  has  been  studying  and  developing  this. 

Chase  and  Simon  (1973)  and  deGroot  (1965)  were  among  the  first  to  explore  the  development  of  expertise 
in  master  chess  players  and  they  proposed  an  information-processing  perspective  (theory)  as  the  analogy  of  the 
human  processor.  Since  then,  a great  deal  of  research  has  been  carried  out  in  the  area  of  expertise  in  problem-solving 
areas,  such  as  chess,  bridge  (Chamess,  1979),  physics  (Chi,  Feltovich,  & Glaser,  1981),  mathematics  (Schoenfield 
and  Herrmann,  1982),  and  medical  diagnosis  (Patel  & Groen,  1986).  More  recently,  research  has  expanded  to  the 
instructional  design  field  (Rowland,  1992;  Perez  & Emery,  1995;  Perez,  Johnson,  & Emery,  1995;  Le  Maistre,  1998; 
Julian,  Larsen,  & Kinzie,  1999),  the  sport  domain,  and  parallels  between  sport  and  the  more  traditional  cognitive 
tasks  have  been  evidenced.  However,  with  the  studies  to  date,  emphasis  is  the  relationship  between  expertise  and 
human  competence  in  general.  What  we  learn  about  expert  performance  and  its  acquisition  is  applicable  to 
understanding  and  improving  competence  in  the  skills  and  knowledge  learned  in  school  and  in  the  workplace. 

1.  Expert  behaviors  are  different  in  terms  of  cognition  structure  (mental  model).  Experts  have 

organized  their  knowledge  into  complex  structures  that  link  the  abstract  conceptualizations  of  the 
domain  with  the  surface  features  of  the  system  (Chi,  Glaser,  & Rees,  1982;  Chi,  Feltovich,  & 
Glaser,  1981).  The  development  of  knowledge/expertise  is  acquired  in  such  a way  that  it  is  highly 
connected  and  articulated,  so  that  inference  and  reasoning  are  enabled,  as  is  access  to  procedural 
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actions.  In  short,  those  organizations  of  knowledge  provide  a schema  for  thinking  and  cognitive 
activity. 

2.  The  structures  of  knowledge  expertise/skills  are  developed.  The  work  of  Laufer  & Click  (1996) 
proved  that  the  four  major  interactive  skill  components  that  expert  performers  coordinate  when 
troubleshooting:  system,  procedural,  strategic  knowledge  (macro-level  cognitive  skills 
architecture)  and  strategic  decision  factors  (inform  the  decision-making  process  of  strategic 
knowledge). 

3.  Leaming/expertise  is  a continuum.  The  progression  can  be  described  in  terms  of  three  interactive 
phases:  (a)  external  support;  (b)  transition  (c)  self-regulation  (Glaser,  1996).  At  each  different 
phase,  the  use  of  external  support  with  the  performer  calling  on  competitors,  performance 
situations,  and  the  advice  of  coaches  as  particularly  needed  is  very  selective.  In  this  regard,  we 
should  design  improved  and  supportive  environments  for  different  kinds  of  learning  (performance) 
(accordingly) 

Putting  it  all  together,  we  believe  (a)  cognitive  performance  models  believed  to  be  detailed  enough  to 
provide  the  criterion  performance  for  an  instructional  system;  (b)  a developmental  trajectory  depicting  the  skill 
acquisition  path  from  novice  to  expert;  and  (c)  individual  learning  and  performance  differences  that  result  in 
impasses  or  barriers  along  the  skill  acquisition  trajectory  (Gott  8l  Lesgold,  2000)  are  important  in 
leaming/perfo nuance.  Therefore,  EPSS  designers  will  be  able  to  identify  critical  learning  and  support  elements 
during  the  design  process. 

EPSS  designers  are  interested  in  understanding  how  an  integrated,  un-simplified  form  of  real-world 
competence  evolves  over  time  as  well  as  in  searching  for  detailed  cognitive  performance  models  that  could  drive 
effective  instruction  and  then  integrate  them  into  learning  systems  (for  effective  learning),  because  real-world 
learning  and  problems  are  unpredictable,  dynamic  and  complex. 

Knowledge  gained  fromthese  studies  contributes  to  the  design  of  optimal  instructional  processes  for 
acquiring  expertise,  or  at  least  facilitating  its  development.  Most  of  the  research  cited  above,  however,  is  quite  basic 
and  was  not  guided  by  a desire  to  improve  the  process  of  designing  EPSSs.  What  we  would  like  to  do  here  is 
describe  an  additional  approach.  Activity  Theory,  that  has  been  emphasized  in  several  recent  research  studies  (Laufer 
& Glick,  1996;  Jonassen  & Rohrer-Murphy,  1999;  Hung,  1999;  Hung,  Koh,  & Chua,  2000;  Hung  & Wong,  2000) 

Activity  Theory 

According  to  Hung  & Chen  (2000)  the  basic  structure  of  an  activity  consists  of  the  (a)  intended  object  to  be 
achieved  by  (b)  subjects  involved  within  the  context  of  (c)  a community  where  work  is  mediated  by  (d)  tools,  (e) 
rules  of  the  practice,  and  (f)  division  of  labor  (Cole  & Engestrom,  1991;  Jonassen  & Rohrer-Murphy,  1999;  Kuutti, 
1996).  Thus  Activity  cannot  be  understood  or  analyzed  outside  the  context  in  which  it  occurs. 

Activity  theory  originated  within  the  cultural-historical  school  of  Soviet  Psychology  and  classical  German 
philosophy  (Wertsch,  1981;  Hung,  1999;  Jonassen  & Rohrer-Murphy,  1999;  Hung  & Wang,  2000).  Following  that, 
it  was  subsequently  followed  up  by  current  researchers  in  Social-Cultural  Psychology,  such  as  situated  cognition, 
distributed  cognition,  activity  theory,  and  other  dimensions  relating  to  social-cultural  perspectives  (Hung,  Koh,  and 
Chua,  2000).  The  fundamental  notion  is  that  there  are  close  interrelations  between  the  individual  mind  with  others 
and  the  environment  or  authentic  context(s).  As  an  EPSS  designer,  we  need  to  be  more  concerned  with  the  context 
in  which  learning  and  performance  occurs,  as  well  as  the  design  process  itself. 

Knowledge  is  socially  constructed  based  on  the  processes  of  intemationality,  history,  culture,  and  tool 
mediation.  The  production  of  any  activity  involves  individuals,  the  object  of  the  activity,  the  tools  that  are  used  in 
the  activity,  and  the  actions  and  operations  that  affect  an  outcome  (Nardi,  1996;  Hung,  1999).  Rather  than  focusing 
on  knowledge  states  and  representations,  the  work  of  Laufer  & Glick  (1996)  focuses  on  the  activities  in  which 
people  are  engaged,  the  nature  of  their  tools,  the  social  relationships,  the  contextual  factors,  and  the  goals  and 
outcomes  of  activity.  As  a result,  “learning  occurs  only  in  the  context  of  meaningful  activity,  it  is  important  to 
analyze  the  activity  and  the  contexts  as  part  of  instructional  design  process  (Jonassen  & Rohrer-Murphy,  1999).” 

Implementations  To  EPSS  Design 

Instructional  designers  should  focus  not  only  on  the  products  or  outcomes  of  learning  but  also  on  the 
historical  and  genetic  processes  that  lead  to  the  resultant  phenomena.  Hung,  Koh,  & Chua  (2000)  concluded  two 
fundamental  reasons  for  this. 
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1.  Culture,  though  man-made,  both  forms  and  makes  possible  the  workings  of  the  human  mind;  thus, 
learning  and  thinking  are  always  situated  in  cultural  settings  and  always  depended  upon  the 
utilization  of  cultural  resources. 

2.  Because  human  action  and  learning  are  primarily  socio-culturally  situated,  even  when  the 
individual  sits  in  solitude  and  contemplates  something,  he  or  she  is  socio-culturally  situated  by 
virtue  of  the  mediational  means  he/she  employs 

In  addition,  jobs  are  placing  greater  cognitive  demands  on  workers.  In  Job  environments  such  as  aviation, 
operating  rooms,  and  command  and  control  centers,  especially  where  time-constrained  decisions  are  critical  to 
overall  performance,  there  is  a growing  demand  for  cognitive  analysis  to  support  the  design  of  Job  aids  and  training 
systems.  Within  these  operational  environments,  cognitive  analysis  is  evolving  from  research  and  development 
projects  to  applied  cognitive  analyses  integrated  with  traditional  instructional  design  processes  (Redding  & 

Seamster,  1994).  Activity  approach  has  been  proposed  in  Laufer  & Clicks*  study  (1996)  for  investigating  an 
everyday  work  task  by  novices  and  experts. 

Activity  is  not  merely  external  behavior;  rather,  it  is  inextricably  linked  with  consciousness.  It  is  the  key  to 
understanding  the  relationship  between  consciousness  and  the  objective  world.  Hence,  conscious  learning  emerges 
from  activity  or  performance,  not  a precursor  of  it.  For  this  reason,  activity  theory  has  been  introduced  and  used  in 
the  field  of  human-computer  interaction  (Bodker,  1991),  constructivist  learning  environments  (Jonassen  & Rohrer- 
Murphy,  1999)  and  instructional  design  (Wilson,  1999)  in  order  to  provide  a clear  operational  framework  for 
designing  complex  leaming/support  systems. 

These  studies  all  emphasize  the  need  to  consider  the  larger  context.  They  place  an  emphasis  on  social 
interaction  within  an  activity  context  and  the  processes  of  internalization  that  take  place  through  interaction  and 
mediation.  In  this  regard,  EPSS  designers  need  to  understand  the  performer  culture  (organization  culture)  and  work 
environment  (learning  context)  to  at  least  some  extent  in  order  to  negotiate  the  needs  (based  on  the  situations)  and 
design  effective  products  (Wilson,  1999;  Jonassen  & Rohrer-Murphy,  1999;  Kara  & Schwen,  1999;  and  Hung, 

1999) 

The  main  contribution  of  this  activity  framework  is  that  it  proposes  a different  perspective  for  analyzing 
(examining  of)  work  practices,  as  performed  by  people  within  natural  settings.  The  framework  is  most  useful  in  both 
ill-defined  and  well-defined  task  areas  where  routing  and  non-normal  tasks  have  been  specified  through  a task 
analysis  or  the  ISD  process.  Recognizing  the  wide  range  of  cognitive  analysis  methods,  this  framework  (approach), 
in  addition  to  the  methods  in  ISD  process,  allows  for  studying  different  forms  of  human  practices,  factoring  in  the 
processes  of  context  as  developmental  process,  both  at  the  individual  and  social  levels  at  the  same  time  (Kuutti, 

1996;  Hung  & Wong,  2000). 

Activity  theory’s  approach  to  instructional  design  is  clearly  based  on  distinctly  different  epistemic  and 
pedagogical  assumptions  than  classical  approaches  to  instructional  design.  According  to  Jonassen  & Rohrer-Murphy 
(1999),  activity  provides  an  alternative  perspective  to  the  mental  istic  and  idealist  views  of  human  knowledge  that 
claim  that  learning  must  precede  activity.  Activity  theory  posits  that  conscious  learning  emerges  from  activity 
(performance),  not  as  a precursor  to  it.  So  activity  theory  provides  us  with  an  alternative  way  of  viewing  human 
thinking  and  activity.  Activity  theory  is  also  a powerful  socio -cultural  and  socio -historical  lens  through  which  we 
can  analyze  most  forms  of  human  activity. 

In  short,  activity  theory  provides  an  alternative  framework  for  designing  effective  systems  by 
understanding  the  expertise  of  particular  task/knowledge/performance,  expert  [as  well  as  novice]  behaviors  (mental 
model,  goals,  rules,  intention,  motivation,  social  interaction),  and  learning  context  (culture,  tools,  objects, 
environments). 

Conclusion 

It  is  exciting  that  the  development  of  technology  provides  many  opportunities  to  enhance  performance  (and 
learning)  that  will  involve  EPSSs  and  other  forms  of  environmental  modification.  In  addition  to  employing  classic 
instructional  design  methods,  EPSS  designers  are  aware  of  alternative  approaches  for  designing  and  developing 
good  instructional  systems,  such  as  rapid  prototyping  methodology  (Jones  & Richey,  2000)  and  concurrent 
engineering  (Gustafson,  2000).  At  another  level,  the  design  of  these  modalities  will  als  o require  alternative  (or 
mixed/combination)  approaches  (such  as  activity  approach)  to  understand  change  from  a socio -culture  perspective 
of  focus  from  performance  outcome  to  performance  activity  and  the  relationship  of  learning  with  their  environment 
where  learning  and  activities  take  places. 

We  need  to  design  more  effective  EPSSs  as  the  demand  increases.  It  is  important  to  know  the  difference 
between  experts  and  novices  because  we  can  know  how  knowledge  is  transferred  and  how  novices  become  experts 
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as  well  as  how  instructional  designers  can  support  these  people  effectively  based  on  the  notion  that  different 
performance  requires  different  support.  However,  to  reach  this  goal,  particular  attention  should  be  given  to  the 
question  of  how  we  design  the  environment  (support  system)  of  people  and  things  about  them,  and  use  the  situations 
they  encounter  to  improve  their  performance.  It  would  also  be  informative  for  EPSS  designers  to  understand  the 
properties  of  different  disciplines  and  different  situations  of  performance  that  are  more  or  less  amenable  to  designing 
conditions  for  improvement,  and  that  require  various  kinds  of  participatory  experiences  and  assisting  devices  for 
supporting  performance  in  the  course  of  acquiring  competence. 

Equipped  with  all  the  understanding  and  knowledge  described  above  of  how  technical  expertise  appears  to 
grow  in  the  wild,  the  initial  state  of  the  learner  and  an  explication  of  the  process  of  learning  (i.e.,  the  transition  from 
initial  state  to  a desired  state  in  an  instructional  setting),  learners*  individual  differences,  and  the  assumption  of 
learning  mechanisms  (internalization,  assimilation,  and  restructuring;  Gott  & Lesgold,  2000),  we  are  ready  to  turn 
our  attention  to  designing  an  instructional  environment  that  could  effectively  reproduce  such  expert -like 
performance,  but  through  systematic  learning  events,  accelerating  the  lengthy  process  that  occurred  naturally  (Gott 
& Lesgold,  2000). 

With  these  theories,  techniques  and  knowledge  that  make  the  knowledge  structures  and  cognitive 
performance  of  competence  explicit,  knowable,  and  leamable,  we  believe  a properly  designed  instructional 
environment  could  shorten  the  skill  acquisition  process,  resulting  in  accelerated  skill  development  for  a group  of 
novice  learners. 
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Abstract 

The  proposed  study  is  to  investigate  the  design  and  development  process  of  EPSSs.  The  primary  purpose  of 
this  study  is  to  help  the  EPSS  designers  by  proposing  a more  effective  and  productive  EPSS  design  and 
development  model.  By  analyzing  EPSS  products  and  observing  EPSS  design  projects,  the  researcher  will 
propose  a design  model  for  future  EPSS  designers. 

Introduction 

The  recent  developments  of  technologies  have  been  changing  the  way  we  live.  Many  organizations,  such  as 
business,  educational  or  governmental  settings  have  been  challenged  to  integrate  information  technologies  into  their 
work  settings.  Parallel  to  this  change,  in  the  past  decade,  there  has  been  a significant  shift  from  traditional  training 
and  instructional  systems  to  performance-based,  individualized  and  just-in-time  support. 

As  stated  by  Hung  (1998),  today’s  organizations  convey  the  ineffectiveness  of  traditional  performance 
support  interventions,  represented  by  both  traditional  training  and  traditional  performance  support  technologies. 
Traditional  interventions  have  helped  a little,  but  not  significantly,  to  move  the  organization  successfully  into  the 
performance  zone.  Moreover,  according  to  Rosenberg  (1995),  for  a performance-based  approach  the  concepts  of 
students,  courses,  curricula  and  instruction  have  little  meaning.  Rosenberg,  Coscurelli,  and  Hutchinson  (1999)  note 
that: 

The  overwhelming  amount  of  complex  information  required  to  perform  work  at  a competent  level 
has  placed  considerable  strain  on  traditional  education  and  training  systems.  This  situation  has  led 
to  the  development  of  job  aids,  computer  databases,  and  electronic  training  systems  as  well  as  of 
structured  text  design.  (p.30) 

Therefore,  especially  because  of  the  wide  availability  of  new  information  technologies,  knowledge-based 
organizations,  such  as  educational  institutions  and  corporations,  have  started  to  implement  and  use  Electronic 
Performance  Support  Systems  (EPSS)  to  increase  both  performance  and  productivity. 

Laffey  (1995)  indicates  the  goal  of  an  EPSS  as  providing  whatever  is  necessary  to  ensure  performance  and 
learning  at  the  moment  of  need.  It  is  agreed  upon  by  several  researchers  (Gery,  1989;  Gery,  1995;  Raybould,  1995; 
Lawton,  1999;  Gustafson,  2000)  that,  in  order  to  improve  performance  and  reduce  the  time  spent  on  task,  the 
ultimate  goal  of  an  EPSS  is  to  provide  performers  with  the  right  information,  in  the  right  quantity  and  detail,  at  the 
right  time.  In  those  respects,  EPSS  is  different  from  traditional  training  systems. 

With  the  help  of  such  systems,  performers  receive  support  during  performance,  rather  than  before  they 
perform  their  jobs.  Laffey  (1995)  adds  that  an  EPSS  is  not  only  a vehicle  for  delivering  static  information,  but  also  a 
reconceptualization  of  the  work  environment  that  is  grounded  in  the  fluid  nature  of  support  in  the  work  environment. 
This  is  a major  shift  from  the  traditional  approach  to  training. 

As  pointed  out  by  Raybould  (2000),  the  major  question  facing  organizations  today  is  not  whether  to  do 
performance-centered  design,  but  how  to  get  it  done.  Since  the  ultimate  goal  of  EPSSs  is  to  reduce  the  cost  of 
training  while  increasing  productivity  and  performance,  they  have  been  developed  and  used  by  many  organizations 
during  the  last  decade. 

As  stated  above,  EPSSs  are  different  from  traditional  training  and  instructional  systems,  because  their 
primary  purpose  is  to  support  performers  while  doing  their  actual  job.  According  to  Rosenberg  (1995),  EPSS  is  a 
fundamental  paradigm  shift  in  thinking,  and  it  requires  a broader  perspective  about  what  is  possible  in  improving 
human  performance.  Rosenberg  criticizes  many  educators  because  they  are  locked  in  a linear  cultural  model  that 
focuses  on  learning  as  an  end  and  instruction  as  a means,  both  of  which  contrast  with  the  means  and  ends  of  EPSS. 
This  difference  also  makes  the  EPSS  design  and  development  process  different.  Since  the  overall  purpose  of  an 
EPSS  is  different  from  the  traditional  purpose  of  instruction,  several  researchers  (Milheim,  1997;  Northrup,  1999) 
state  that  the  use  of  traditional  instructional  system  development  models  is  inadequate  for  EPSS  development. 
Northrup  ( 1 999)  summarizes  this  difference  as  “instructional  design  is  the  process  of  designing  and  developing 
instruction  to  achieve  specific  learning  outcomes,  where  EPSS  is  more  focused  on  producing  task  performance”  (p. 
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1).  Rosenberg  (1995)  adds  that  involvement  in  building  and  implementing  EPSSs  requires  a fundamental  rethinking 
of  the  relationship  between  learning  and  performance.  Parallel  to  this,  the  design  and  development  of  more  effective 
EPSSs  is  becoming  a critical  issue  that  needs  to  be  focused  on  by  researchers.  Therefore,  since  EPSS  has  significant 
differences  from  instruction,  then  the  approach  to  its  design/development  process  should  also  be  different. 

Significance  of  the  Study 

Several  researchers  state  the  need  for  a clear  methodology  for  building  EPSSs  (Fisher  & Horn,  1997; 
Gustafson,  2000;  Milheim,  1997).  Cote  (2000)  states  that  the  focus  on  the  "work  process"  makes  the  design  and 
development  of  an  EPSS  quite  different  than  the  traditional  models  of  instructional  design.  If  the  primary  purpose  of 
an  EPSS  is  to  support  performance,  learning  is  of  secondary  importance;  and  if  an  EPSS  is  different  than  instruction, 
how  should  it  be  designed? 

Despite  a growing  number  of  success  stories,  the  EPSS  remains  a relatively  new  concept  and  little  is  known 
about  the  different  aspects  of  EPSS  design.  This  study  will  create  a better  understanding  of  the  EPSS  design  process. 
By  observing  and  analyzing  EPSS  design  projects  in  different  phases  of  the  production  path,  the  researcher  will 
propose  a design  model  for  future  EPSS  developers. 

For  this  study,  rather  than  having  in-depth  analysis  of  individual  steps,  the  focus  will  be  on  the  generic 
design  process,  with  the  major  components  that  reflect  the  patterns  of  design  activities  carried  out  by  EPSS 
designers  in  the  field.  Identifying  the  underlying  detailed  steps  is  beyond  the  scope  of  this  study.  They  can  be 
addressed  in  future  studies. 

Literature  Review 

It  is  generally  agreed  upon  that  Gloria  Gery  is  the  first  person  who  introduced  the  term  EPSS  at  the  end  of 
the  1980s  (Gery,  1989).  Gery  (1989)  and  Brown  (1996)  define  an  EPSS  as  a self-contained,  online  system  which  is 
designed  to  integrate  a knowledge  base,  expert  advice,  learning  experiences,  and  guidance  with  the  goal  of  provid  ing 
individuals  with  the  ability  to  perform  at  a higher  level  in  the  workplace  and  requires  minimal  support  and 
intervention  by  others.  Another  EPSS  pioneer  Raybould  (1995)  defines  EPSS  as 

...the  electronic  infrastructure  that  captures,  stores,  and  distributes  individual  and  corporate 
knowledge  assets  throughout  an  organization,  to  enable  individuals  to  achieve  required  levels  of 
performance  in  the  fastest  possible  time  and  with  a minimum  of  support  from  other  people,  (p.  11) 

Later,  Gustafson  broadens  those  dimensions  by  including  multiple  knowledge  sources,  expert  systems  or 
artificial  intelligence,  contextual  multimedia  instruction  and  more  sophisticated  tools  (Gustafson,  2000). 

As  seen  from  the  literature,  EPSS  is  still  a young  and  evolving  field  (Moore,  1998).  Therefore,  even  though 
the  term  EPSS  has  been  widely  used  recently,  there  is  no  generally  agreed-upon  definition  (Cole,  Fischer  & 

Saltzman,  1997;  Villachica  & Stone,  1999;  Hannafin,  Hill  & McCarthy,  2000).  According  to  Brown  (1996),  the  best 
use  of  an  EPSS  happens  under  the  following  circumstances:  if  there  is  a profusion  of  information  required  to 
perform  work  at  a competent  level  and  technology  explosions  exist,  experts  are  not  available,  expectations  for 
performers  are  high,  and  performers  have  a self-directed  learning  style. 

Similar  to  the  case  of  different  definitions  of  EPSS,  there  are  also  different  views  about  the  components  and 
characteristics  of  EPSS.  For  example,  Schwen  et  al.  (1993)  argue  that  an  EPSS  has  four  characteristics:  information 
management,  collaboration  management,  productivity  through  embedded  guidance  and  work  metaphor,  and  finally  a 
problem  solving  environment  that  integrates  basic  tools,  information  management,  collaboration  and  productivity 
tools  in  a seamless  environment. 

Hannafin  et  al.  (2000)  list  the  four  core  components  of  EPSS  as  resources,  contexts,  tools,  and  scaffolds. 

The  ways  in  which  the  varied  elements  within  the  components  are  combined  will  vary  depending  upon  the  goals, 
context,  and  participants. 

Reigeluth  (1999)  defines  an  EPSS  as  a computer  program  that  provides  support  for  the  performance  of  a 
task.  According  to  him,  an  EPSS  usually  has  four  major  components:  a database,  an  expert  system,  an  instructional 
system  and  tools.  These  components  are  explained  below: 

• A database.  A store  of  information  that  can  be  accessed  by  keywords  or  topical  menus  during  the 
performance  of  a task.  In  addition,  the  database  could  provide  access  to  other  tools  or  software  to  help 
the  worker  do  his  or  her  job,  such  as  spreadsheet  and  word  processing  software.  “The  EPSS  would 
integrate  tutorials  or  instructional  systems  and  expert  systems  into  the  database,  both  in  order  to  make 
them  context-sensitive,  and  to  allow  them  to  share  data”  (Sleight,  1993,  p.7).  The  type  of  data  in  this 
database  may  be  textual,  numeric,  visual  (photographs  and  video),  or  audio  (conversations,  speeches,  and 


music).  The  information  would  be  linked  with  related  information  via  non-linear  hypertext  links, 
providing  fast  access  to  information,  and  allowing  for  different  levels  of  knowledge  in  users.  The 
effectiveness  of  an  EPSS  strongly  depends  on  how  well  the  database  matches  with  the  user’s  task  and 
environment. 

• An  expert  system.  An  expert  system  is  a computer-based  system,  which  emulates  the  decision-making 
ability  of  a human  expert  (Jonassen  8c  Reeves,  1996).  It  can  be  accessed  as  an  expert  is  accessed — by 
asking  questions  or  receiving  unsolicited  suggestions — during  performance  of  a task.  It  may  suggest  the 
most  appropriate  procedure  or  step  to  do  next.  Today,  the  terms  expert  system,  knowledge-based  system, 
and  knowledge-based  expert  system  are  used  synonymously. 

• An  instructional  system.  A set  of  methods  (such  as  providing  hints,  suggestions,  and  guidance  to  move 
the  student  along)  of  instruction  to  help  performers  just  in  time  for  performance.  It  may  be  a list  of  steps 
to  take,  a motion  video  showing  a procedure,  or  a simulation  of  a task  that  allows  the  user  to  practice. 

• Tools.  Programs  that  a performer  uses  to  perform  specific  tasks,  such  as  grade  book  programs , electronic 
notebooks,  spreadsheets,  statistical  analysis  packages,  and  even  e-mail  programs  and  Web  browsers. 

Reigeluth  states  that  the  Human-Computer-Interface  (HCI)  aspect  of  EPSS  covers  all  these  components  and 
can  be  thought  of  as  an  umbrella  under  which  all  of  the  other  components  are  located. 

What  Makes  EPSS  Different? 

As  stated  in  the  previous  pages,  it  is  generally  agreed  among  EPSS  experts  that  an  EPSS  is  different  from 
an  instructional  system  (IS)  in  many  respects.  One  of  the  most  important  differences  is  its  main  focus,  which  is 
performing  rather  than  learning.  In  their  article,  Witt  and  Wager  (1994)  compare  EPSSs  with  instructional  systems 
to  highlight  the  differences  between  them.  They  argue  that  an  instructional  system  is  “a  product  that  contributes  to 
the  achie  vement  of  some  type  of  learning  outcome,”  and  instruction  has  methods  “for  bringing  about  desired 
changes  in  student  knowledge  and  skills  for  a specific  course  content  and  a specific  student  population”  (p.  20).  In 
contrast,  while  learning  may  occur  during  the  use  of  electronic  performance  support,  its  primary  purpose  is  to 
facilitate  performance  of  a task;  learning  is  of  secondary  importance”  (p.  20). 

The  second  important  difference  between  IS  and  EPSS  is  using  EPSS  while  doing  the  actual  job,  not 
beforehand.  On  this  aspect.  Cole  et  al.  (1997)  state  that  EPSS  is  actually  a paradigm  shift  for  training  organizations 
because  in  EPSS  knowledge  delivery  takes  place  soon  enough  that  is  applied  to  the  appropriate  situation,  and  late 
enough  that  the  user  does  not  have  to  go  through  training  or  information  overload”  (p.  50). 

The  third  difference  reported  is  that  people  do  not  need  to  follow  a predetermined  sequence  while  using  the 
EPSS.  A novice  and  an  expert  may  use  it  differently.  In  addition  to  this,  in  an  instructional  system  it  is  assumed  that 
the  terminal  requirements  of  an  instructional  product  are  predetermined  (Witt  & Wager,  1994).  According  to  Ryder 
and  Wilson  ( 1 996),  if  the  content  is  well-defined  and  stable,  and  it  is  based  on  algorithms  and  rules,  an  instructional 
system  becomes  the  best  solution.  However,  an  EPSS  provides  contextually  relevant  information  for  a dynamic 
environment  (Laffey,  1995)* 

As  seen  until  this  point,  even  though  there  are  some  similarities  between  Instructional  Systems  and  EPSSs, 
the  differences  are  important.  For  certain  kinds  of  well-defined  content  within  stable  training  environments, 
instructional  systems  may  work  satisfactorily.  However,  as  stated  by  Wilson,  Jonassen,  and  Cole  (1993),  the 
limitations  of  this  approach  become  apparent  when  working  in  ill-defined  content  domains,  which  requires  self- 
pacing and  creativity  or  when  working  with  highly  diverse  populations. 

Need  for  an  EPSS  Design/Development  Approach 

As  seen  in  the  previous  section,  EPSSs  are  significantly  different  from  instructional  systems.  Therefore,  as 
stated  by  Witt  and  Wager  (1994),  the  fundamental  differences  of  purpose  between  the  two  products  indicate  that 
different  methodologies  need  to  be  used  to  create  them.  Laffey  (1995)  emphasized  that  EPSSs  will  no  longer  be 
simply  extensions  of  what  we  know  about  instructional  design  and  the  design  of  databases.  The  same  point  is  made 
by  Gustafson  (2000)  that  the  design  of  EPSSs  requires  alternative  (or  mixed)  methods.  Finally,  the  lack  of  well- 
established  design  models  is  obvious  from  the  literature.  For  example,  Gery  (1995)  states. 

Few  [EPSSs]  are  guided  by  a set  of  integrated  and  fully  articulated  design  principles.  Many 
innovations  are  the  result  of  individual  or  team  creativity  and  iterative  design  employing  rapid 
prototyping  coupled  with  ongoing  usability  and  performance  testing  (p.  48) 

Current  Approaches  to  EPSS  Design/Development  and  Their  Problems 
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There  is  a broad  consensus  that  there  is  a lack  of  well-defined  EPSS  design  and  development  models 
(Gustafson,  1993;  Gustafson,  2000;  Milheim,  1997;  Laffey,  1995;  Rosenberg,  Coscarelli  & Hutchison,  1999),  and 
information  that  describes  how  people  have  actually  designed  and  developed  EPSSs  is  also  insufficient  (Gery, 
1989b).  As  stated  by  Dickelman  (1996)  even  though  the  definition  of  EPSS  is  clear  for  many  people,  they  cannot  tell 
you  “how  to  go  about  doing  it”. 

According  to  Gustafson  (2000),  there  are  three  main  reasons  for  this  lack  of  information.  First,  since  many 
EPSSs  are  developed  by  commercial  organizations  and  are  their  property,  they  do  not  share  this  information  with 
others.  Winer  et  al.  (1999)  also  support  this  argument.  They  state  that  because  of  competitive  advantage  and  strict 
confidentiality  regulations,  it  is  difficult  to  obtain  data  about  specific  EPSS  developments.  Second,  the  history  of 
EPSS  is  no  more  than  10-12  years.  Therefore,  procedures  have  not  been  well  developed  and  tested.  Gery  (1995a) 
supports  the  view  that  since  EPSS  is  relatively  new,  it  is  difficult  to  define  a detailed  development  methodology  for 
EPSS.  Finally,  Gustafson  reports  the  third  reason,  as  “..  some  EPSS  designers  may  be  reluctant  to  talk  about  what 
they  have  done,  since  they  are  unable  to  clearly  articulate  specific  and  replicable  procedures.”  (p.  42). 

In  the  literature,  EPSS  designers  have  used  classical  instructional  design  approaches  (e.g.  ADDIE)  (Benko 
& Webster,  1997;  Graham  & Sheu,  2000),  a modification  of  ED'^  (an  instructional  design  and  development 
methodology  from  Digital  Equipment  Corporation)  (Brown,  1996),  rapid  prototyping  (Tripp  & Bichelmeyer,  1990; 
Law,  Okey  & Carter,  1995),  prototyping  and  layers  of  necessity  (Wedman  & Tessmer,  1991;  Northrup  et  al.,  1998), 
or  combination  of  them  (Gustafson,  2000).  Witt  and  Wager  (1994)  report  that  large  companies  modify  the  ISD 
models  “to  meet  the  unique  challenges  and  complexities  of  EPSS  solutions.”  (p.  20) 

However,  as  stated  by  several  researchers  (Wilson,  1999;  Rosenberg,  Coscarelli  & Hutchison,  1999),  ID 
models  have  several  limitations  in  performance  support  systems  development.  The  most  significant  problem  with 
them  is  that  they  generally  analyze  the  job  tasks  to  identify  whether  someone  can  perform  the  task  or  not. 
Performance  support  is  concerned  with  providing  information  and  assistance  when  the  employee  needs  it  and  how 
they  need  it  (Witt  & Wager,  1994).  Rosenberg  et  al.  (1999)  report  the  necessity  of  a paradigm  change  to 
performance  technology  systems  development  models. 

Habelow  (2000)  also  raises  the  issue  of  the  difference  between  the  EPSS  design  process  and  ISD,  and  she 
discusses  the  need  for  involvement  of  other  disciplines.  She  states 

...  most  organizations  have  their  own  customized  version  of  an  ISD  process  that  is  used  as  an 
approach  to  the  design  of  EPSS  applications.  With  such  a striking  difference  in  perspective, 
learning  versus  performance,  a new  model  of  EPSS  design  and  development  that  combines  the 
instructional  value  of  education  focused  models  and  incorporates  a more  technological  perspective 
of  computer  and  information  systems  models  may  be  warranted,  (p.76) 

Raybould  (1995)  suggests  that  the  methodology  for  developing  this  electronic  infrastructure  must  have  a 
wider  scope  than  other  existing  methodologies.  He  emphasizes  the  need  for  involving  different  elements  in  EPSS 
design  methodology,  namely  human  performance  technology,  instructional  systems  development,  computer  based 
training,  information  engineering,  business  process  reengineering,  knowledge  engineering  and  technical  writing  and 
interface  design  (Raybould,  1995;  Raybould,  2000). 

Help  from  Other  Fields 

As  stated  above,  an  EPSS  is  composed  of  several  modules  or  components.  The  implementation  process 
needs  to  be  based  on  multiple  theories  and  approaches.  Therefore,  the  process  of  building  those  components  in  an 
integrated  manner  has  to  be  a multidisciplinary  approach.  Raybould  (2000)  calls  this  approach  Performance  Support 
Engineering  (PSE). 

For  a successful  EPSS  design/development  process,  Hannafm,  Hill  and  McCarthy  (2000)  emphasize  the 
importance  of  welcoming  other  approaches  that  emerged  outside  of  our  traditional  community,  in  other  related 
disciplines.  Similar  to  other  researchers,  they  also  agree  on  the  multidisciplinary  nature  of  EPSS  design,  stating  that: 
EPSS  design  practices  represent  a convergence  among  several  related  fields  and  specialties, 
including  human  performance  technology,  computer-supported  collaborative  work,  technical 
communications,  electronic  publishing,  instructional  design,  and  workplace  training,  (p.  3) 

Similarly,  Gustafson  (2000)  affirms  this  view  and  aigues  that  EPSS  design/development  is  an  immature 
technology,  so  exploring  other  disciplines  and  applying  their  related  methodologies  will  definitely  make  a 
significant  contribution  to  the  work  in  EPSS  design/development. 

It  is  obvious  that  EPSS  design  and  development  is  a complex  and  multidisciplinary  process.  Among  the 
several  methodological  approaches  that  I have  found  in  the  literature,  there  are  two  approaches  that  look  very 
promising  for  building  a model  for  the  EPSS  design  and  development  process.  Multi  view  is  one  of  them,  which 
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comes  from  Information  Science,  one  of  the  Soft  System  Methodologies  (SSM).  The  second  one  is  Chaos  Theory, 
which  is  related  to  SSM  and  ISD.  Since  the  EPSS  design  and  development  process  is  so  complex,  Chaos  Theory 
might  provide  us  with  a new  perspective  to  better  understand  this  process.  It  is  this  researcher’s  initial  intuition  that 
an  EPSS  has  similar  characteristics  to  a chaos  system. 

Information  Systems  Design  and  EPSS 

According  to  Villachica  and  Stone  (1999),  Information  Systems  (IS)  development  models  offer  detailed 
strategies  that  can  be  applied  to  EPSS  creation.  Cole  et  al.  (1997)  also  agree  on  this  view  and  argue  that  EPSSs 
should  adhere  to  general  software  engineering  methods.  Furthermore,  Gustafson  (2000a)  strongly  encourages  the 
researchers  in  1ST  to  look  more  at  approaches  in  Information  Science  discipline.  Foshay  et  al.  (1999)  capture  the 
essence  of  building  human  performance  theory  from  IS  by  stating  that  IS  principles  “define  a common  ground 
between  HPT  [Human  Performance  Technology]  and  information  science”  (p.  899).  According  to  them  three 
branches  of  IS  support  HPT:  Information  Technology,  Information  Systems  and  Information  Management. 

Complex  projects  and  unsuccessful  attempts  to  realize  them  is  a common  problem  for  many 
disciplines/fields.  Recently,  a systemic  approach  to  this  kind  of  problem-solving  has  been  provided  in  a 
methodology  developed  by  Peter  Checkland,  Professor  of  Systems  at  Lancaster  University,  UK.  This  is  known  as 
the  Soft  Systems  Methodology  (SSM)  (Checkland  & Scholes,  1990).  It  was  developed  to  overcome  the  limitations 
of  the  conventional  systems  analysis  approaches.  Such  approaches  are  prescriptive  and  emphasize  the  “hard”  aspects 
of  the  problem  domain,  which  is  the  certain  and  the  precis  e.  However,  Soft  Systems  Methodology  is  designed  to 
allow  the  human  element  of  such  systems  to  be  incorporated  into  system  design  work,  and  both  technical  and  non- 
technical issues  are  explicitly  addressed.  It  is  most  appropriate  for  systems  that  are  complex  and  ill-structured 
(Finegan,  1994;  Bennetts  & Wood-Harper,  2000). 

A well-known  design  framework  from  IS,  which  seems  a promising  approach  for  building  an  EPSS 
design/development  model,  is  Multiview  (Avison,  1996).  Multiview  contrasts  with  traditional  IS  design 
methodologies,  where  "the  steps  are  prescribed  in  great  detail  and  are  expected  to  be  followed  rigorously  in  all 
situations"  (Avison  & Wood-Harper,  1990,  p.l3).  Multiview  is  perhaps  the  most  famous  attempt  at  combining  hard 
systems  and  soft  systems  philosophies. 

As  stated  by  Raybould  (2000),  a very  important  aspect  of  performance  support  engineering  is  its  focus  on 
human  elements  as  well  as  computer  elements.  Therefore,  I believe  several  aspects  of  Multiview,  specifically  SSM 
and  STD,  will  best  fit  for  the  development  of  an  EPSS  design  model. 

From  my  practical  experiences,  both  in  IS  and  ISD,  I see  that  traditional  development  methods  are  only 
used  in  theory.  As  stated  by  Raccoon  (1995a),  the  practical  development  of  such  systems  has  always  followed 
complex  life  cycles.  Rather  than  phases  being  followed  rigidly,  they  come  and  go  as  the  project  evolves.  This  is  the 
same  pattern  that  nature  follows.  The  weather,  political  systems,  economy,  society  and  all  other  complex  systems  do 
not  follow  a predetermined  path.  Rather,  they  show  a chaotic  path.  Therefore,  I believe  that,  while  exploring  the 
EPSS  design  and  development  process,  it  is  worthwhile  looking  at  Chaos  Theory.  In  the  following  section,  this 
approach  will  be  explored  briefly. 

Chaos  Theory  (Dynamic  Systems  or  Adaptive  Development) 

Several  researchers  agree  that  the  traditional  systems  approach  to  problem  solving  has  a reductionist  nature 
and  it  tends  to  solve  a problem  by  fragmentation,  one  stage  at  a time  (Finegan,  1 994;  You,  1 993;  Jonassen,  1 990). 
This  approach  may  work  for  some  small  scale  and  well-defined  situations.  However,  the  systems  associated  with 
human  activity  are  complex  and  not  well-defined.  According  to  Jonassen  (1990)  “simple  systems  behave  in  simple 
ways,  and  complex  systems  behave  in  complex  and  less  predictive  ways.  The  behavior  of  a system  cannot  be 
examined  accurately  by  analyzing  its  components”  (p.  34).  As  an  alternative  to  a linear,  reductionist  and 
deterministic  approach.  Chaos  or  the  dynamical  systems  approach  is  proposed.  In  a complex  system  “the 
components  are  related  and  interlock  with  one  another  such  that  a change  in  one  component  invariably  affects 
another  part  of  the  system,  or  eventually  even  the  entire  system”  (Mumare,  cited  in  Chieuw,  1991,  p.25).  Gordon 
and  Greenspan  explain  Chaos  as  the  study  of  disorder,  and  it  appears  in  non-linear  systems  (as  cited  in  King,  1991). 
Since  Chaos  deals  with  non-linear  and  disorderly  systems,  many  disciplines,  including  technological,  social  and 
economic,  are  appropriate  for  applying  its  principles.  As  stated  by  Highsmith  (2000)  “from  physics  to  biology  to 
chemistry  to  evolution,  complex  adaptive  systems  theory  began  to  help  explain  occurrences  in  the  real  world  that  the 
linear  approximations  of  the  old  science  could  not”  (p.  10).  According  to  King  (1991),  for  many  different 
disciplines.  Chaos  gives  new  data,  suggests  innovative  approaches  to  old  ideas,  and  reaffirms  certain  approaches. 
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As  pointed  out  by  Hannafin,  Hill  and  McCarthy  (2000),  “EPSS  systems  have  evolved  into  larger  and  more 
complex  environments”  (p.  32).  They  do  not  have  fixed  features  and  components.  In  addition  to  this,  especially  in 
complex  domains,  “an  effective  performance  requires  expertise  beyond  the  skill  level”  (Sinitsa,  2000,  p.l8)  and 
performers  follow  unpredictable  patterns  that  are  discontinuous  and  complex  (Paulson  & Paulson,  cited  in  You, 
1993,  p.  24).  As  a result,  such  factors  make  the  design  process  itself  less  systematic  and  more  situated  (Sherry  & 
Wilson,  1996).  EPSS  development  does  not  operate  in  isolation,  and  not  done  in  a linear  manner  (Ho  & Hara,  2001). 
However,  this  life  cycle  is  interdependent  and  connected. 

To  summarize,  the  EPSS  design/development  process  has  a complex  nature.  To  the  extent  that  it  is  similar 
to  IS  development,  EPSS  development  is  at  the  edge  of  technology,  and  it  requires  a mixture  of  chaos-based  top- 
down  and  bottonvup  processes  (Raccoon,  1995b). 
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Abstract 

The  number  of  educational  sites  on  the  World  Wide  Web  grows  daily  and  teachers  may  have  difficulty 
identifying  sites  well  matched  to  their  intended  outcomes  for  learners.  This  study  describes  the  development 
of  an  instrument  to  categorize  educational  Websites.  One  hundred  ninety-five  (195)  "exemplary”  or 
award-winning  educational  Websites  were  examined  and  evaluated  by  the  researchers.  During  the 
evaluation,  thirteen  general  site  categories  emerged,  and  the  sites  were  sorted  into  these  categories.  This 
paper  describes  the  categories  of  the  instrument,  the  process  by  which  the  instrument  was  developed,  and 
the  result  of  this  categorization. 

The  use  of  the  World  Wide  Web  (the  Web)  appears  to  be  increasing  rapidly  in  education,  and  may  have 
already  become  an  important  resource  for  teaching  and  learning.  However,  the  sheer  number  of  educational  sites  and 
variety  of  sources  can  be  overwhelming  for  educators  who  want  to  incorporate  the  Web  in  instruction.  It  is  difficult 
to  know  which  sites  are  good  and  which  are  not.  In  fact,  this  proliferation  and  the  speed  with  which  new  sites 
appear  makes  it  near  impossible  for  any  one  teacher  to  examine  enough  sites  to  sift  the  wheat  from  the  chaff. 

To  assist  teachers  and  parents,  some  organizations  and  agencies  have  recognized  educational  sites  they 
judge  to  be  “exemplary.”  For  example,  the  International  Academy  of  Digital  Arts  and  Sciences  selects  the  best  of 
the  Web  for  the  annual  Webby  Awards  fhttp://www. webbvawards.com/nominees/index.html ).  Similarly,  each  year 
the  Education  Source  names  their  Top  100  Educational  Websites 

fhttD://www.edusource.com/articles/topl  00  99/default. asp).  PC  Magazine  also  names  its  annual  Top  100  Websites, 
including  an  “Education  and  Family”  category 

nittp://www.zdnet.com/pcmag/stories/reviews/0.6755. 239445 3 >00.htmn.  Some  agencies  identify  “exemplary” 
Websites  more  often;  for  example,  each  month  the  Eisenhower  National  Clearinghouse  for  Mathematics  and  Science 
Education  (ENC)  recognizes  thirteen  exemplary  Websites  in  science  and  mathematics 
nittp://www.cnc.org/classroom/dd/nf  ddcrit.htm). 

Examining  these  recognized  educational  Websites  reveals  a wide  variety  of  purposes,  strategies,  audiences, 
and  content.  Our  purpose  in  this  study  was  to  examine  such  recognized  educational  Websites  and  develop  an 
instrument  to  categorize  them  based  on  emergent  categories.  This,  in  turn,  should  help  show  which  types  of  sites 
seem  most  likely  to  be  named  as  “exemplary.” 

Methods 


A total  of  218  sites  were  compiled  from  sites  recognized  in  2000  by  the  above-named  recognizing  groups 
(Webby’s,  ENC,  Top  100  Ed  Websites,  PC  Magazine).  When  we  examined  those  sites,  seven  were  duplicates  and 
sixteen  were  non-functional,  yielding  195  functional  “exemplary”  educational  sites. 

The  195  sites  examined  were  developed  for  various  purposes  and  were  diverse  in  target  audience  and 
content.  This  meant  that  any  instrument  we  designed  had  to  encompass  what  we  found  on  the  sites.  Beginning  with  a 
proposed  categorization  instrument,  two  researchers  reviewed  and  categorized  each  of  the  195  sites.  They  then 
compared  results,  and  discussed  and  debated  at  length  on  sites  they  categorized.  A third  researcher  served  as  a 
mediator  to  resolve  inter-rater  differences.  Each  site  was  analyzed  until  all  three  members  agreed  on  the 
categorization  of  every  site.  The  instrument  and  operational  definition  of  each  category  evolved  during  the  process 
of  evaluating  the  sites.  That  is,  as  we  analyzed  the  sites  and  resolved  our  disagreements,  the  categories  on  our 
instrument  were  modified  or  confirmed. 

Below  we  list  13  possible  categories  for  Websites  that  emerged  as  the  researchers  evaluated  the  sample  of 
“exemplary”  or  award-winning  sites.  Note,  however,  that  not  all  sites  can  easily  be  placed  in  a single  category. 

Some  sites  exhibit  properties  of  two  or  more  of  the  following  categories.  When  classifying  these  sites,  we  identified 
which  classification  seemed  most  representative  of  the  site  as  a whole.  Sites  that  were  equally  distributed  across  two 
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or  more  types  were  noted  and  included  in  the  discussion  of  results.  The  categories  are  presented  as  follows  with 
definitions  and  examples  of  each 

Instructional 

To  be  classified  as  instructional,  a site  must  include  (1)  an  intended  learning  outcome,  (2)  instructional 
strategies,  (3)  learning  materials  and  activities,  and  may  include  (4)  learner  assessment  and/or  feedback.  An 
instructional  strategy  is  a pedagogical  technique  designed  to  facilitate  learning  through  a combination  of  teacher  and 
learner  activity.  We  observed  that  these  elements  may  be  explicitly  stated  and  demonstrated  or  they  may  be 
embedded  within  site  content. 

Instructional  Websites  with  a constructivist-oriented  design  and/or  self-exploratory  navigation  strategy 
usually  don’t  include  formal  stated  learning  tasks,  however.  Instead,  these  sites  usually  provide  learning  activities 
that  require  learners  to  do  something  on  the  Web,  including  reading  posted  information;  completing  practice 
activities  and/or  playing  games;  exploring  related  resources;  using  a site  search  engine;  communicating  and/or 
sharing  with  peers,  teachers,  and/or  experts;  exploring,  articulating,  or  reflecting  in  a problem-based  learning;  and 
creating  content.  Note  that  the  learning  activities  in  a site  must  go  beyond  just  reading  posted  information  in  order  to 
be  categorized  as  an  instructional  site. 

Examples  of  instructional  sites  include:  Learn  Physics  Today  fhttp://! ibrarY.advanced.org/lQ796)  and 
Conflict  Yellowstone  Wolves  rhttP://powavusd.sdcoe.kl2.ca.us/mtr/ConflictYellowstoneWolf.htm).  Learn  Physics 
Today  is  an  online  physics  tutorial.  Conflict  Yellowstone  Wolves  require  learners  to  analyze  the  Rocky  Mountain 
Gray  Wolf  problem  and  draw  their  own  conclusion  to  the  question:  Should  the  wolves  in  Yellowstone  National 
Park  be  removed? 

Sometimes  a site  will  appear  to  be  instructional  when  it  is  not.  For  example,  some  sites  provide  online 
quizzes  and  games  (one  form  of  learning  activity).  However,  their  mere  presence  does  not  guarantee  that  a site  is 
instructional.  When  quizzes  and  games  serve  no  instructional  purpose  and  are  designed  primarily  for  entertainment, 
we  did  not  classify  the  site  as  instructional.  Another  way  in  which  a site  can  erroneously  appear  to  be  instructional 
is  when  it  uses  instructional  labels  like  “tutorial”  or  “problem-based  learning  project”  in  referring  to  itself,  but 
actually  contains  no  instructional  strategies.  In  other  words,  it  is  not  enough  simply  to  claim  to  be  instructional.  The 
key  is  whether  the  site  includes  the  three  essential  instructional  elements. 

Content  Collection 

A content  collection  site  is  a collection  of  information  about  a specific  content  area  (such  as  genetics  or 
insects)  that  is  informative  and  might  be  used  in  learning  but  is  not  instructional.  Content  collection  sites  differ  from 
instructional  sites  in  that  they  don’t  provide  useis  with  learning  activities,  goals,  strategies,  or  assessment.  They 
may,  however,  include  informative  readings,  illustrations  and  other  rich  content.  A typical  content  collection  site  on 
a specific  subject  is  Common  Cold  nittp://www.commoncold.org/).  a comprehensive  source  of  information  about  the 
common  cold.  Carol  Hurst's  Children's  Literature  Site  (http://www.carolhurst.com/)  is  an  example  of  content 
collection  site  on  a general  topic.  The  site  is  a collection  of  reviews  of  great  books  for  kids,  ideas  of  ways  to  use 
them  in  the  classroom  and  collections  of  books  and  activities  about  particular  subjects,  curricular  areas,  and 
professional  topics. 

Archive/Database/Reference 

An  Archive/Database/Reference  (ADR)  is  a collection  of  information  that  is  organized  as  an  archive.  These 
sites  may  be  indexed  chronologically,  alphabetically,  or  topically.  Such  sites  are  designed  as  information  and 
reference  tools.  Online  dictionaries,  encyclopedias,  reference  books,  and  question-answer  services  comprise  this 
category.  These  differ  from  Content  Collections  in  that  they  may  address  multiple  topics  and  are  oriented  toward 
searching  for  information  rather  than  examining  a topic.  One  example  of  an  ADR  sites  is  American  Memory 
(http://memorv.loc.gov).  a site  that  offers  more  than  seven  million  digital  items  from  more  than  100  historical 
collections  relating  to  the  history  and  culture  of  the  United  States.  Ask  Jeeves  for  Kids  (http://www.aikids.com/ ) and 
The  Last  Word  Science  Questions  and  Answers  (http://www.last-word.com/ ) are  representative  of  questions-answers 
service  ADR  sites. 

Compilation  of  Online  Learning  Activities/Games 
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These  sites  are  collections  of  individual  learning  activities  and/or  games  to  be  completed  online  by  learners. 
ExploreMath.com  (Tittp://w ww.exploremath.com/index.cfm)  is  a collection  of  interactive  math  activities  in  13 
categories.  As  suggested  by  the  site,  these  individual  learning  activities  may  be  incorporated  into  the  mathematics 
classroom,  lab,  or  distance  learning  curricula,  but  such  compilations  fail  to  exhibit  the  essential  three  components  of 
an  instructional  site. 

Collection  of  Links 

Sites  in  this  category  feature  listings  of  categorized  external  hyperlinks  to  online  resources  on  a subject. 
They  contain  little  or  no  content  of  their  own  and  serve  principally  as  portals  to  external  content.  European 
Renaissance  (http://www.execpc.com/~dboals/rena.html)  lists  67  related  external  links.  It  should  be  noted  that  if  a 
site’s  collection  of  links  are  mostly  internal  (or  most  of  the  actual  content  is  within  the  site),  it  should  be  classified  as 
Content  Collection. 

Online  Exhibit 

Online  exhibits  are  focused  collections  of  media  (for  example,  images,  recordings,  animations,  videos, 
VRMLs)  related  to  some  particular  content  or  event.  Displays  of  museum  or  organization  holdings  online  are  typical 
of  this  category.  Such  sites  are  designed  for  exhibition  and  information  purposes.  These  virtual  exhibits  are  often 
representations  of  actual  museum  exhibits  in  the  physical  world.  For  example.  Ocean  Planet 
(http://seawifs.gsfc.nasa.gov/ocean  planet.htm)  is  an  online  version  of  than  actual  exhibit  from  the  Smithsonian 
National  Museum  of  Natural  History. 

Teacher  Resource 

Teacher  resource  sites  are  designed  to  provide  teachers  with  such  things  as  lesson  plans,  classroom 
activities,  teacher  guides,  curricula,  state  and  national  standards,  and  professional-development  resources.  These 
materials  may  be  housed  within  the  site  or  available  as  downloadable  files.  While  such  sites  may  also  include 
subject  matter  links  and  learner  activities,  the  main  focus  of  the  site  is  assisting  teachers. 

For  example.  One  Sky  Many  Voices  (http://www.oneskv.umich.edu/)  lists  inquiry -based  K-12  science 
curricula  for  use  by  teachers,  and  Illuminations  (http://illuminations.nctm.org/)  is  designed  as  a teacher  resource 
with  the  following  components:  online  multimedia  math  investigations,  classroom  video  vignettes,  standards-based 
lesson  plans,  links  to  reviewed  Websites,  and  access  to  the  National  Council  of  Teachers  of  Mathematics  Principles 
and  Standards. 

Vicarious  Participation 

Vicarious  participation  sites  provide  learners  with  opportunities  to  participate  online  in  an  ongoing 
educational  or  research  activity,  or  an  expedition.  Virtual  field  trips  are  included  in  this  classification.  These  sites 
attempt  to  give  the  learner  a sense  of  participation  in  activities  not  available  to  the  typical  classroom  student.  Such 
sites  may  archive  the  materials  from  past  explorations  for  examination  any  time  after  the  activity  is  concluded.  The 
Jason  Project  (http://www.lasonproiect.org/)  is  an  examp  le  of  an  ongoing  educational  activity.  In  The  Jason  Project, 
learners  can  participate  in  a current  learning  adventure  and  visit  Jason’s  past  expeditions.  In  Extreme  2000 
(http://www.ocean.udel.edu/deePsea/).  a deep-sea  research  expedition,  learners  may  view  video  clips  of  the 
expedition,  read  the  dive  log  and  daily  journal,  and  listen  to  live  audio  from  the  research  team. 

Research/Curriculum  Project 

These  sites  are  designed  to  inform  the  visitor  about  a particular  ongoing  research  or  curriculum  project. 
Content  may  include  news  and  upcoming  events,  research  results  and  publications,  individuals  associated  with  the 
project,  funding,  and  other  related  information.  For  example.  Rural  and  Urban  Images 

(http://www.ael.org/nsf/voices/index.htm)  is  a three-year  project  seeking  to  help  girls  excel  and  feel  confident  in 
science,  math,  and  technology. 

Communication  Community 

Communication  community  sites  facilitate  discussion,  interaction,  entertainment,  and  other  information- 
sharing. Users  can  play  interactive  games  and  share  thoughts  and  ideas  with  each  other  via  e-mail,  chat,  and  message 
boards.  Such  sites  usually  do  not  include  learning  activities,  although  they  may  be  “educational”  in  the  broader  sense 
of  encouraging  discussion  of  current  events  and  issues  of  importance  to  learners.  Example  sites  include  Cyberkids 
(http://www.cvberkids.com/ ) and  Kids  Space  (http://www.kids-space.org/). 
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Place  with  Public  Mission 

This  Website  classification  refers  to  actual  places  such  as  museums,  historic  sites  (like  Mystic  Seaport  or 
Colonial  Williamsburg),  zoos,  gardens,  aquariums,  parks,  and  the  like.  Such  sites  typically  address  visitor 
information,  schedules,  exhibits,  “how  to  get  here,”  membership,  and  special  events.  Many  of  these  sites  include  an 
education  section  with  learning  materials  and  activities  for  teachers  and  students  and  are  often  referred  to  under  the 
umbrella  of  Informal  Education.  Exploratorium  fhttp:// www.exploratorium.edu/index.htmn  and  Colonial 
Williamsburg  fhttp://www. history. ore)  are  two  examples  of  such  sites. 

Academic  or  Research  Organization 

These  sites  represent  a particular  non-commercial  research  organization  or  academic  unit.  They  typically 
address  such  things  as  the  nature  and  purpose  of  the  organization,  purpose,  its  current  and  past  projects,  recent 
results,  pertinent  news  and  events,  and  related  educational  materials  and  activities.  NASA  (http://www.nasa.gov/) 
and  the  NASA  Jet  Propulsion  Laboratory  (http://www.ipl.nasa.gov)  are  two  examples  of  such  sites. 

Commercial 

Commercial  sites  are  primarily  intended  to  promote  and  sell  products  or  services.  Many  commercial  sites 
have  ancillary  components  designed  to  inform,  educate,  or  entertain.  For  example,  a company  selling  cold  and  flu 
medicine  may  include  information  or  instruction  about  the  causes  and  treatments  of  colds  and  flu.  It  is  sometimes 
necessary  to  explore  a site  extensively  to  identify  whether  it  is  commercial  or  not.  Simply  having  commercial 
sponsorship  (as  might  be  indicated  by  advertising  banners)  is  not  enough.  The  site  must  have  a goal  of  helping  to 
promote  a company’s  products.  One  way  in  which  some  commercial  sites  promote  their  products  or  services  subtly 
is  by  targeting  young  audiences  using  cartoon-like  animation  and  games  on  their  Websites.  Once  again,  one  needs 
to  look  for  the  commercial  connection.  Usually  it  becomes  clear  on  close  examination.  Examples  include  CNN 
Interactive  (http://www.cnn.com).  National  Geographic  Society  (http://www.nationalgeographic.com/ ).  SurJMonkey 
Kids  Channel  (http://www.surfmonkev.com/ ) and  US  Space  Camp  (http://www.spacecamp.com). 

Table  1 presents  the  distribution  of  “exemplary”  sites  across  our  classification  scheme.  The  three 
categories  most  likely  to  be  recognized,  in  order  of  distribution  in  our  study,  were  Content  Collection  (26.2%), 
Instructional  (20.5%),  and  Teacher  Resource  (1 1 .8%).  These  three  types  of  sites  account  for  approximately  six  out 
of  every  ten  sites  recognized.  This  suggests  how  the  recognizing  groups  defined  “exemplary”  and  perhaps  hints 
more  directly  at  how  they  viewed  educational  use  of  the  Web. 

Table  L Distribution  of ‘'Exemplary  ” Educational  Websites  Across  Categories 


Categories 

# of  Sites 

% 

Instructional 

40 

20.5% 

Content  collection 

51 

26.2% 

Archive/Database/Reference 

14 

7.2% 

Compilation  of  individual  online  learning  activities/games 

13 

6.2% 

Collection  of  links 

13 

6.2% 

Online  exhibit 

7 

3.6% 

Teacher  resource 

23 

11.8% 

Vicarious  participation 

9 

4.6% 

Research/Curriculum  project 

2 

1.0% 

Communication  community 

2 

1.0% 

Places  with  public  mission 

11 

5.6% 

Academic  or  research  organization 

6 

3.1% 

Commercial 

10 

5.1% 

Total: 

195 

100% 
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Conclusions 


We  learned  quickly  that  some  sites  cannot  be  neatly  classified  into  a single  category.  We  analyzed  sites  that 
could  be  assigned  to  two,  three,  or  more  categories.  In  dealing  with  such  sites,  we  chose  to  make  a holistic 
assignment  based  on  what  we  deemed  to  be  the  site’s  principal  orientation  in  terms  of  our  classification  scheme.  For 
example,  NASA  Is  My  Playground  nittp://kids.msfc.nasa.govA  and  Merriam-Webster  Word  Central 
nittp://www.wordcentral.corril  contain  both  instruction  and  teacher  resources.  Since  the  majority  of  content  was 
instruction,  the  sites  were  categorized  as  instructional  sites.  Ask  Jeeves  for  Kids  (http://www.aikids.corril  fits  into  at 
least  three  categories  — ADR,  Teacher  Resource,  and  Communication  Community.  Since  its  primary  activity  was 
searching,  it  was  classified  as  an  ADR  site. 

Although  Reeves  and  Reeves  (1998)  and  Wilson  (1997)  have  suggested  that  constructivist  design  may  be 
promising  in  Web-based  learning  environment,  the  “exemplary”  sites  examined  in  this  study  were  almost 
exclusively  objectivist  in  design  orientation.  Our  results  are  consistent  with  those  in  a similar  study  conducted  by 
Mioduser,  Nachmias,  Lahav,  and  Oren  in  1998.  Their  study  investigated  436  educational  Websites  in  an  effort  to 
determine  the  current  state  and  emerging  trend  in  Web-based  learning  environment. 

We  found  a number  of  Websites  that  perhaps  overstated  the  extent  to  which  they  were  instructional  sites. 
For  example.  Embryo  Images  (bttp://www. med.unc.edu/embrvo  images/1  described  itself  as  an  instructional 
tutorial,  but  is  simply  is  a slide  show  of  images  with  text  descriptions.  Other  sites  included  descriptions  of 
“exploration”  or  “problem-based”  instructional  strategies,  but  turned  out  to  involve  nothing  more  than  reading 
posted  information. 

The  next  step  in  our  research  will  be  to  complete  an  analysis  of  the  instructional  design  of  the  40 
instructional  sites  we  identified.  Our  goal  is  to  determine  the  major  common  characteristics  of  such  recognized 
instructional  Websites.  As  part  of  this  analysis,  we  plan  to  refine  and  expand  our  current  instrument’s  categories. 
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Abstract 

PreKCentral  is  a web  site  designed  to  provide  preschool  teachers,  daycare  providers  and  families  with 
tools  to  be  used  for  the  positive  stimulation,  development  and  education  of  young  children  through  the 
available  technology.  This  web  site  uses  a database  to  hold  the  activities  and  links  and  web  site  visitors  use 
either  a predefined  simple  search  or  tailor  the  search  for  their  interests.  This  session  would  provide  an 
overview  of  the  development  of  the  site,  the  design  considerations/constraints,  and  an  overall  look  at  the 
site.  Other  features  such  as  a chat  room,  forum,  kid  art  and  links  will  be  examined. 

Description  of  the  site 

PreK  Central  is  a web  site  designed  for  preschool  teachers,  parents  of  preschoolers,  and  caregivers.  This 
site  is  based  on  the  mission,  philosophy  and  practices  of  the  National  Association  of  Education  of  the  Young  Child 
(NAEYC).  NAEYC  supports  the  planning  and  practice  of  guiding  preschool  children  with  activities  that  are 
developmentally  age  appropriate  By  focusing  on  the  needs  of  the  child,  NAEYC  believes  children  begin  to  build  a 
strong  foundation  to  lifelong  learning.  The  NAEYC  web  site  can  be  accessed  at  http://www.naeyc.org. 

PreK  Central  strives  to  provide  many  services  to  its  visitors.  Site  visitors  can  search  for  age  appropriate 
activities  as  well  as  participate  in  chats,  forums,  find  outside  links  to  other  sites  or  explore  our  preschool  art  gallery. 
Early  in  the  planning  stages  of  the  site,  it  was  apparent  that  an  underlying  database  was  needed  to  provide  a dynamic 
site  incorporating  structure,  organization  and  to  anticipate  site  growth.  Each  section  of  this  site  was  designed  with 
organization,  content,  usability  and  potential  site  growth  in  mind. 

The  development  of  the  PreK  Central  (PreK)  web  site  began  in  1997  after  the  collaboration  on  two  web 
sites.  As  graduate  students  we  redesigned  the  web  site  for  college  of  Human  Resources  and  Education  at  Virginia 
Tech  and  were  instructional  designers  for  PE  Central  (http://pe. central. vt.edu. J As  parents  of  preschoolers,  we  both 
found  few  sites  geared  toward  preschool  education  and  even  fewer  that  offered  developmentally  appropriate 
activities  for  children.  As  instructional  designers,  we  found  that  many  very  good  sites  were  available  on  the  Net,  but 
they  lacked  three  key  elements:  organization  /content,  a scheme  for  age  appropriateness  and  appropriate  use  of 
technology.  Sites  from  the  search  lacked  the  organization  and  content  needed  to  support  the  site  visitor.  Also 
missing  from  preschool  education  sites  was  a scheme  to  determine  an  activity’s  appropriateness  for  preschoolers. 
Another  missing  component  from  available  sites  was  the  appropriate  and  effective  use  of  available  technology. 

These  three  areas  were  the  main  focus  in  our  site  design  and  will  be  elaborated  on  through  the  site’s  design  process. 

Planning  and  constructing  the  site 
Organization  and  content 

Pre  K Central  was  envisioned  with  these  multiple  sections:  a database  of  activities,  a chat  room,  a 
forum,  a page  of  preschooler-related  links,  and  the  preschool  art  gallery.  Access  to  these  sections  is  designed  with 
the  audience  in  mind.  The  elements  that  make  up  the  site  (the  navigation,  layout,  graphics,  links  and  other 
information)  should  be  easily  understandable  by  any  visitor  to  the  site.  Our  target  audience  is  a diverse  group  of 
visitors.  The  site  is  for  teachers,  parents  of  preschoolers  and  other  preschool  caregivers.  This  audience  can  be 
broken  into  two  major  groups:  those  who  have  formal  training  (i.e.  preschool  teachers)  and  those  with  no  formal 
training.  Sites  with  a diverse  audience  such  as  PreK  Central  need  to  be  flexible  and  provide  several  avenues  to 
obtain  information.  One  way  to  design  to  these  audiences  is  to  begin  by  asking  questions  like,  “How  will  different 
visitors  use  the  site?”  or  “What  information  will  visitors  be  looking  for?”  This  technique  becomes  one  of  the  driving 
forces  in  the  design  of  the  site.  A detail  of  how  those  with  formal  training  and  those  with  informal  training  are 
addressed  in  our  scheme  for  appropriate  practices. 

Site  Content 
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The  database  would  need  to  hold  blocks  of  text,  but  could  also  be  used  for  other  forms  of  web  media  such 
as  images  (GIFs  and  JPEGs),  sound  files  (MPEGs,  WAV  files  or  QuickTime  movies).  There  is  also  a need  to  try  to 
foresee  other  forms  of  media  that  may  be  included  since  web  design  is  still  in  its  infancy.  Designers  for  PE  Central 
failed  to  accurately  predict  how  the  site  would  grow.  By  planning  for  growth,  hopefully  Pre  K Central  could  easily 
be  expanded  to  site  visitor’s  future  needs. 

After  designing  the  database  , usability  testing  was  done  by  a technique  we  termed  a ‘mental  walkthrough’ 
which  was  a helpful  tool  to  determine  whether  the  logic  used  in  the  design  process  was  sound.  This  also  helps  later 
when  deciding  whether  of  not  to  use  certain  elements.  “Mental-walkthroughs”  put  designers  in  place  of  the  site 
visitor  and  mentally  walk  through  a web  site.  These  mental  walkthroughs  help  us,  as  designers,  add  consistency  to 
the  site.  They  also  force  designers  to  “ think  outside  of  the  box”  and  really  consider  the  site  visitor.  Many  times 
web  designers  design  a site  as  they  would  use  it  and  force  the  visitor  to  “think  like  me”  to  get  an  around  a site. 

Some  questions  asked  for  this  site  (with  our  possible  solutions): 

What  information  might  1 look  for  if  1 was  a parent? 

• Activities  grouped  in  understandable  categories 

• Variety  of  outside  links 

What  information  might  1 look  for  if  1 was  a teacher? 

• Activities  grouped  by  skills  to  build 

• Variety  of  outside  links 

Is  the  viewer  and  potential  viewer  being  considered? 

• Assume  the  role  of  a potential  viewer 

Is  an  experienced  visitor  bored  or  annoyed  by  the  searches? 

• Build  with  enough  search  options 

• Layout  the  search  page  so  that  it  is  easy  to  navigate 

• Layout  the  site  so  the  search  is  easy  to  find 

Could  1 use  this  site  if  I had  little  experience  using  the  Web? 

• Design  easy,  obvious  site  navigation 

• Provide  many  ways  to  find  information 

What  terms  are  common  enough  for  a layperson  such  as  a parent  to  recognize? 

• Choose  terms  that  are  intuitive  as  to  their  meaning 

• Use  images  that  help  describe  terms 

What  terms  would  a teacher  or  trained  educator  likely  to  recognize? 

• Use  NAEYC’s  terminology 

How  could  both  these  groups  of  visitors  get  to  the  same  information,  easily  and  quickly? 

•Provide  many  ways  to  find  information 

Scheme  for  appropriate  practices 

PreK  Central  has  a diverse  audience:  for  preschool  teachers,  parents  of  preschoolers,  and  caregivers.  As 
state  before,  the  audience  is  made  up  of  two  major  groups:  those  who  have  formal  training  (i.e.  preschool  teachers) 
and  those  with  no  formal  training.  Searching  for  information  was  a crucial  part  of  the  design  of  the  PreK  Central 
web  site.  On  the  surface  there  appears  to  be  categories  of  activities  and  a searchable  feature.  These  are  really  one 
the  same  search.  The  categories  of  activities  are  really  defined  fields  in  the  database,  a narrowed  search. 

Creating  two  different  ways  to  search  the  same  database  solved  the  solution  to  this.  For  teachers  and  other 
child  care  professionals,  a search  of  “kid  skills”  is  offered.  From  NAEYC’s  standards  of  appropriate  practices 
(Bredekamp,  1987)  four  categories  of  skills  emerged.  Named:  Physical,  Cognitive,  Social  and  Emotional  Skills, 
these  “kids  skills”  are  a set  of  skills  that  teachers  would  most  likely  be  using  to  plan  their  activities.  Here  is  the 
complete  listing  of  our  Kid  Skills  and  the  concept  they  introduce; 


Kid 

Introduces... 

Kid  Skill  Introduces... 

Skill 

Physical 

large  motor 

Cognitive  imagination 

small  motor 

formulating  hypothesis 

levels  of  activity 

ordering 

hand-eye  coordination 

critical  thinking 

Social 

communication  with  others 

levels  of  complexity 
addition  /subtraction 

447 


Emotional 


social  awareness 
social  interaction  levels 
social  leader  or  follower 
social  cooperation 
large  groups 
small  groups 
multicultural 

repertoire  for  solving  social 
problems 

expressing  needs/feelings  in 

appropriately 

self-control 

perseverance 

patience 

orientation  of  self  in  world 
concepts  of  self 


classification 

sedation 

number 

spatial  relationships 
mass  and  quantity 
causality 
time  concepts 
spoken  language 
written  language 
divergent  thinking 
size  of  objects 
balance  and  support  of 
structures 
space  concepts 
(volume/area)  relationships 
of  objects 
observations 
creative  thinking 
creative  expression 


Table  I:  Kid  Skills  and  concepts  introduced. 


For  parents  or  others  looking  for  appropriate  activities  a search  of  *'kid  talents”  or  categories  that  activities 
naturally  fall  into  is  possible.  Kid  talents  are  topics  such  as  “Make  believe”  and  “Arts-n-crafts”.  These  Kid  Talents 
are  matched  to  images  that  depict  the  kid  talent.  Here  is  our  listing  of  Kid  Talents  with  our  description  of  what  kind 
of  activity  falls  into  that  Kid  Talent. 


Talents  Definition 


Arts-n-crafts 
Make  Believe 
Got  Rhythm? 
Word  Play 
On  the  move 
Easy  as  1,  2,  3 
Celebrate  Us! 

Me  Too! 
Hands-On 
I Wonder 
Clubhouse 
Young  Explorers 


Foster  creativity  through  2&3D  art 
Encourage  imaginative  play 
Introduce  body  movements  & music 
Whole  word,  letter  recognition,  phonics 
Gross  motor  skills 
Build  math  skills 

Exploration  of  own  and  other  cultures,  holidays  and  customs 

Activities  with  several  generations 

Manipulative  and  small  motor 

Science  activities 

Circle  time,  social  activities 

Exploration  outside  the  four  walls  of  a classroom 


Table  2:  Kid  Talents  and  their  objective. 


Site  Organization 

The  site  is  broken  into  several  components  that  support  the  site  visitor.  These  components  are  the  database 
of  activities,  a chat  room,  a forum,  a page  to  related  outside  links  and  a gallery  of  children’s  art.  Each  of  these 
supports  the  visitor  in  different  ways.  The  activities  are  the  most  important  part  of  the  site.  This  section  will 
hopefully  grab  the  visitor’s  attention  and  bring  them  back.  The  chat  room  and  forum  and  still  in  their  developmental 
stages.  They  are  intended  to  begin  on  line  discussion  among  teachers,  parents  and  caregivers.  The  chat  is  a place 
where  all  can  gather  to  trade  information  and  ideas  in  real  time.  The  forum  differs  in  that  is  more  of  a posting  or 
bulletin  board  for  discussion.  Presently  there  are  Just  topic  listed,  but  it  is  hoped  that  eventually  guest  speakers  and 
experts  in  the  field  of  childcare  and  early  childhood  education  can  post  their  points  of  view.  There  will  be  available 
along  with  the  opportunity  for  others  to  respond.  The  off  site  links  is  a collection  of  early  childhood  education  links. 
Many  topics  are  available  from  funding  opportunities  for  preschool  and  preschool  teachers  to  child  advocacy  links. 


Overhaul  of  this  section  is  being  done  to  include  a search  similar  to  the  activity  search.  The  last  section  of  preschool 
art  is  a fun  and  playful  way  preschools;  their  teachers  and  even  parents  can  show  off  and  learn  from  preschool  art. 

Appropriate,  effective  use  of  available  technology 

Appropriate  and  effective  use  of  the  available  technology  has  always  been  one  of  our  concerns  for  PreK. 
Too  many  times  a web  site  has  the  potential  to  be  a great  site  but  lacks  function  or  features.  This  is  not  a lack  of 
‘spinning  and  flaming  logos’  but  in  the  lack  of  using  the  technology  that  is  available.  An  argument  may  be  made 
that  you  need  commercial  software  to  keep  a site  up,  looking  good  and  running  well.  We  have  found  the  opposite 
true  and  have  built  and  maintained  PreK  Central  with  very  little  commercial  software. 

PreK  was  created  on  a “shoestring  budget”;  Labor  was  low-cost  with  two  grad  students  developing  in  their 
free  time.  For  most  of  the  site,  free  “open  source”  software  has  been  used  to  bring  it  online,  up  and  running.  Open 
source  software  has  grown  from  software  developers  and  users  who  find  commercial  software  packages  too  limiting 
in  features,  too  difficult  to  modify,  leaving  them  frustrating  and  difficult  to  use.  It  is  well  maintained,  updated  and 
documented,  by  those  who  use  it.  PreK's  server  runs  a free  operating  system  (OS)  called  RedHat  Linux 
(Tittp://www.redhat.comA.  The  web  server  is  Apache  (http://www.apache.org/),  which  is  also  free.  The  database 
that  holds  the  activities  and  links  is  MySQL  (http://www.mysql.com/).  Even  the  scripting  language,  PHP 
(http://www.php.net),  used  to  pull  from  the  database  is  free.  The  chat  and  forum  software  that  we  use  are  also  open 
source  projects. 

Define  look  & feel 

The  “look  and  feel”  of  this  web  site  was  very  important.  Early  in  the  design  process,  a childlike  theme  was 
chosen  for  the  site.  This  playful  design  was  meant  to  encourage  even  visitors  new  to  the  web  pages  to  “come  and 
play”.  Fitting  to  the  theme,  a “childlike  handwriting”  font  and  adopting  a “crayon  colors”  color  scheme  throughout 
the  site.  Images  for  the  site  were  based  on  simple  sketches,  drawings  and  designs  created  with  the  help  of  a two- 
year-old  boy.  This  became  part  of  the  design  process  and  led  to  color  and  playfulness  as  a major  theme.  This 
worked  well  with  the  subject  matter,  the  navigation  and  the  search  method. 

From  the  database  and  search  design,  the  site  had  to  be  easy  to  navigate,  easy  to  use  and  easy  to  find 
information.  With  the  incorporation  of  a database,  the  site  could  easily  be  designed  with  only  three  or  four  pages. 
Search  and  results  pages  could  be  reused,  since  every  search  was  not  a static  page.  This  adds  to  the  consistency  of 
the  “look  and  feel”  since  visitors  become  accustomed  to  seeing  one  page. 

Adding  Depth 

The  next  steps  to  the  growth  of  this  web  site  are  to  add  depth  to  the  site.  Plans  for  this  are  underway  to 
continue  to  add  more  activities,  add  an  easier  way  for  viewers  to  submit  their  ideas  and  to  develop  the  chat  room  and 
forum  section  of  the  site. 

Also  in  the  works  is  the  development  of  a ‘community  of  users’.  The  plan  is  to  start  locally  with  local 
Virginia  residents  and  to  spread  out  to  include  others  in  a more  virtual  community.  Hopefully,  by  adding  depth  to 
the  Forum  to  include  guest  speakers  will  grow  to  virtual  discussion  and  seminars.  There  are  also  plans  to  start  and 
widen  the  collection  for  viewers  to  access  and  educate  themselves. 

Contacts 


You  can  visit  PreK  Central  at  http://prek.dhs.org/  and  see  our  selection  of  activities.  The  email  address  is 
admin@prek.dhs.org  and  a listserv  at  http://prek.dhs.org/mailman/listinfo/prek-announce.  This  paper  can  also  be 
downloaded  from  http://prek.dhs.org/AECT/. 
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Improving  Visual  Recognition  Skills  with  a Digital  Image  Bank:  the  medium  makes  a 

difference. 

J.  Randall  Vance, 

Scott  D.  McDowell 
Ferris  State  University 


Abstract: 

Digital  retinal  photos  were  obtained  for  all  first  year  optometry  students  at  our  College  of  Optometry.  The 
catalog  of  digital  images  was  utilized  in  several  ways  in  print  and  on  the  web  to  enhance  the  development 
of  critical  observation  and  recognition  skills  needed  for  examination  of  the  human  retina.  The  use  of  digital 
images  increased  opportunities  for  practice  and  feedback  resulting  in  improved  performance  compared  to 
baseline  and  to  students  who  had  trained  previously  without  using  the  digital  image  reference  bank. 

Problem: 

Learning  to  examine  the  human  retina  involves  the  development  of  two  skill  sets:  1)  utilizing  ophthalmic 
diagnostic  instruments,  and  2)  interpreting  what  is  observed  with  them.  Training  in  retinal  examination  skills 
(ophthalmoscopy)  begins  in  the  first  semester  of  the  professional  curriculum  at  the  Michigan  College  of  Optometry. 
A typical  entry-level  instructional  sequence  involves  lectures  on  retinal  anatomy  and  features,  and  on 
instrumentation  and  techniques  for  observation.  This  is  followed  by  supervised  practice  in  the  laboratory.  Students 
examine  each  other’s  eyes,  observing,  identifying  and  documenting  variations  in  the  retinal  appearance.  Students 
make  judgments  with  respect  to  critical  features  of  the  retina  and  have  these  verified  by  the  instructor.  Previously, 
the  rate  of  progress  has  been  hampered  by  several  factors: 

1.  Observation  time  was  duplicated  because  instructors  had  to  examine  the  retina  first  before  they 
could  comment  on  the  students’  observations. 

2.  Instructors  had  to  spend  time  looking  for  specific  illustrative  examples  within  the  class. 

3.  Practice  opportunities  outside  of  scheduled  class  time  provided  no  feedback  or  verification  by  the 
instructor. 

4.  Opportunities  to  formally  assess  students'  accuracy  in  observation  were  limited. 


Methods: 

The  availability  of  digital  ophthalmic  photography  and  web  course  management  software  presented  a 
unique  opportunity  to  converge  these  technologies  to  enhance  training  and  performance  of  retinal  examination  skills. 

The  first  phase  of  the  project  involved  an  analysis  of  feasibility,  methods,  materials  and  cost.  An  estimate 
of  image  acquisition  and  processing  time  per  subject  was  determined.  Images  would  be  used  for  both  screen  viewing 
and  print  viewing;  therefore,  various  papers  and  printer  settings  were  compared  for  print  quality.  Estimates  of 
production  time  and  costs  for  the  print  versions  of  the  image  catalog  were  determined.  Then  the  images  were 
acquired  using  the  Digivid®  retinal  video  capture  system  by  Helioasis.  The  software  allows  the  images  to  be 
exported  in  either  TIFF  or  JPEG  graphic  file  formats  for  utilization  outside  of  the  Digivid  database.  Each  of  the  68 
images  was  exported  and  saved  in  the  JPEG  format  immediately  after  capture  and  verification  that  the  captured 
image  was  acceptable. 

The  next  phase  involved  measurement  of  baseline  observational  skill  level  prior  to  any  instruction  on 
ophthalmoscopy  and  retinal  features.  First  year  students  were  pre-tested  via  a time  -limited,  WebCT -delivered  and 
auto-graded  visual  matching  quiz  in  October  of  2000.  Ten  test  images  were  selected  from  the  acquired  image  bank 
and  uploaded  to  the  WebCT  course.  Each  of  the  10  images  was  inserted  into  a short  answer  question  format  using 
WebCT’s  quiz  tool.  The  student  task  was  to  compare  each  image  to  a selection  of  40  print  images  assembled  and 
numbered  in  an  accompanying  catalog.  The  student  answered  by  typing  in  the  number  of  the  image  they  chose  as 
representing  an  exact  match.  The  question  stem  advis  ed  the  student  that  if  a matching  image  was  not  found,  a 
response  of  “0”  should  be  entered.  This  assured  that  all  40  images  had  to  be  considered  before  an  elimination  could 
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be  made.  A ratio  of  1 0 test  images  to  40  potential  matches  and  a time  limit  of  1 5 minutes  were  judged  to  constitute  a 
reasonably  rigorous  test  of  pre -instruction  observational  skills.  Questions  were  presented  one  at  a time,  and  could 
not  be  revisited  once  answered  or  skipped. 

During  the  next  three  weeks,  entry-level  instruction  and  practice  in  the  techniques  took  place.  Instruction 
was  enhanced  by  utilization  of  the  acquired  image  bank  in  several  ways.  Instructors  had  “key”  copies  of  the  class’ 
retinal  image  catalog  printed  with  the  photos  identified  by  name.  This  enabled  quicker  awareness  of  pertinent 
features  of  each  eye  in  the  class,  and  more  effective  guidance  of  the  student’s  interpretation  of  his  or  her 
observations.  This  could  be  done  without  the  instructor  duplicating  the  physical  examination  process.  Furthermore, 
each  student  was  provided  a copy  of  the  class’  image  catalog  printed  without  identifying  the  photos  by  name. 

Similar  to  the  pre-test  task,  now  the  students  were  to  compare  direct  observations  of  their  classmates’  retinas  to  the 
print  images  and  identify  each  image  by  name.  In  this  process,  they  had  to  pay  close  attention  to  minute  details  that 
differentiated  similar  images  from  one  another.  The  student  received  immediate  affirming  or  corrective  feedback 
from  the  instructor.  With  much  repeated  practice,  retinal  details  are  identified  with  greater  speed  and  accuracy.  To 
provide  additional  practice  opportunities  outside  of  scheduled  laboratory  sessions,  the  entire  class  image  bank  was 
uploaded  to  the  WebCT  course  and  placed  within  WebCT’s  image  database  tool.  These  on-line  images  were 
identified  by  name,  so  that  the  students  could  continue  their  matching  exercise  by  viewing  and  comparing  the  screen 
images  to  their  as  yet  unidentified  print  images. 

To  assess  progress  from  baseline,  a time -limited  post-test  was  administered  with  WebCT  in  December, 
2000.  Though  the  format  was  the  same  for  this  test,  previously  unseen  retinal  images  were  used  for  all  questions  and 
for  the  printed  catalog  of  potential  matches. 

In  order  to  compare  performance  to  a control  group,  the  same  pre-  and  post-tests  were  also  administered  to 
second  year  optometry  students,  who  had  trained  the  prior  year  without  the  availability  of  such  an  image  bank.  To 
assess  retention  of  skill  and  the  impact  of  this  technique,  yet  another  matching  test,  based  on  a newly  acquired  image 
set,  was  administered  to  the  experimental  group  in  the  Fall  of  2001 , when  they  were  at  the  same  point  in  training  and 
experience  as  their  predecessors. 

Results: 


Exp.  Pre-Test 

Exp.  Post-test 

Control  Pre-test 

Control  Post-test 

Exp.  Re -test 

Date 

October,  2000 

December,  2000 

October,  2000 

December,  2000 

September, 

2001 

N 

33 

34 

17 

15 

26 

Mean  Score 

84.5% 

90.9% 

85.9% 

84.7% 

91.2% 

s.d. 

13.7 

11.4 

11.8 

20.3 

13.1 

Mean  Time 

10:05 

10:21 

8:30 

9:46 

10:08 

s.d. 

2:28 

2:12 

3:00 

2:29 

2:41 

The  Wilcoxon  signed  ranks  test  was  used  to  determine  whether  the  distribution  of  the  scores  differed  for 
several  comparisons.  Post-test  scores  of  the  experimental  group  showed  a significant  improvement  when  compared 
to  their  pre-test  scores  (p=.006).  Post-test  scores  for  second  year  students  showed  no  significant  differences  from 
their  pre-test  scores.  The  experimental  group’s  third  test  in  September,  2001  showed  a significant  difference  when 
compared  to  their  pre-test  scores  of  a year  prior  (p=.020),  and  no  significant  difference  when  compared  to  their  post- 
test scores  of  December,  2000.  Comparisons  included  only  those  students  who  had  completed  all  tests  for  their 
group.  The  smaller  number  of  scores  in  the  control  group  did  not  enable  valid  inter-group  comparisons  to  be 
calculated.  The  WebCT  quiz  tool  enables  analysis  of  time  elapsed  per  question,  and  time  elapsed  overall.  Though 
there  were  measured  intra-  and  inter-group  time  differences  among  the  several  test  administrations,  none  of  the  time 
differences  were  found  to  be  statistically  significant. 

Discussion: 

The  results  suggest  that  using  the  retinal  image  catalog  in  print  and  on  the  web  heightened  the  experimental 
group’s  attention  to  detail.  While  it  could  be  argued  that  familiarity  with  the  test  format  on  the  second  administration 
may  have  improved  scores  (“learning  effect”),  this  seems  discounted  by  the  fact  that  the  control  group  showed  no 
significant  change  in  their  pre  - and  post-test  scores.  The  retention  of  improved  discrimination  skill  was  confirmed  on 
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re-test  of  the  experimental  group  in  September,  2001.  Having  all  the  images  in  a tangible,  readily  accessible  format 
provided  more  teaching  and  learning  opportunities.  They  saw  more  eyes.  They  had  more  practice.  The  medium 
improved  the  efficiency  of  their  learning.  We  are  continuing  the  process  with  our  current  first  year  class. 
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Motivation  Power:  Exciting  Kids  About  Research 


Ruth  V.  Small 
Syracuse  University 

Marilyn  P.  Amone 
Creative  Media  Solutions,  Inc, 


Introduction 

Information  literacy,  the  ability  to  find,  select,  organize,  use,  manage,  evaluate,  and  present  information,  is 
a critical  literacy  for  the  21*^  century.  In  a society  dominated  (and  often  overwhelmed)  by  information  in  a multitude 
of  formats  (e.g.  video,  audio,  text,  graphics),  a major  challenge  in  education  is  to  teach  our  nation's  youth  to  "be  able 
to  recognize  when  information  is  needed  and  have  the  ability  to  locate,  evaluate  and  use  effectively  the  needed 
information”  (American  Library  Association,  1989,  p.l). 

To  be  information  literate  is  to  be  “an  effective  user  of  ideas  and  information,”  according  io  Information 
Power:  Building  Partnerships  for  Learning,  the  professional  guidelines  for  the  school  library  field  (AASL/AECT,  p. 
6).  Education  goals  on  both  the  national  and  state  levels  specify  the  importance  of  students'  ability  to  use 
information  to  solve  problems,  make  decisions  and  develop  skills  for  lifelong  learning. 

Information  Literacy  Models 

In  the  past  15  years,  a number  of  information  literacy  models  have  been  developed  to  describe  the  skills 
needed  to  successfully  conduct  research  and  solve  information  problems.  For  example,  Eisenberg  and  Berkowitz’s 
Big  Six®  Approach  to  Information  Problem-Solving  (1990)  specifies  a general  approach  to  the  information  problem 
solving  process  consisting  of  six  major  steps.  The  Big  Six  Approach  begins  with  defining  the  parameters  of  the 
information  task  or  problem  and  ends  with  evaluating  both  the  results  of  the  process  and  the  process  itself  Stripling 
and  Pitts’  Research  Process  Model  (1988)  fo  Hows  a more  linear  approach  to  research,  a ten-step  process 
interspersed  with  eight  “reflection  points”  that  allow  students  to  evaluate  and  revise  or  repeat  completed  steps  where 
needed. 

Kuhlthau’s  Model  of  the  Search  Process  (e.g.  1993)  uses  the  results  of  several  years  of  research  to  describe 
the  six  stages  that  students  undergo  during  the  research  process.  She  found  that,  rather  than  a neat  and  sequential  set 
of  skills,  it  is  a learning  process  that  can  be  somewhat  messy  and  more  iterative  in  nature.  She  also  found  that  there 
are  both  cognitive  and  affective  aspects  to  the  information  seeking  process.  For  example,  students  frequently  explore 
and  collect  information  before  they  have  done  an  adequate  job  of  formulating  and  narrowing  their  topic. 

All  of  these  models  are  useful  and  share  a common  framework  that  we  have  synthesized  into  eight  major 
categories  of  information  skills,  each  with  a set  of  sub-skills,  sequenced  across  three  broad  research  stages  (see  Fig. 
1).  The  format  of  this  list  is  not  intended  to  imply  a lockstep  linear  process  but,  in  actuality,  represents  an  iterative 
process  in  which  any  or  all  skills  may  be  revisited  in  order  to  modify  or  expand  on  previous  ones. 

Beginning  Stage 
Definition 

• Identifies  requirements  of  research  task. 

• Determines  amount/type  of  information  needed  to  complete  research  task. 

• Considers  potential  topics. 

Selection 

• Narrows  topic  to  be  explored. 

• Specifies  subtopics  or  related  keywords. 

Planning 

• Formulates  a search  strategy. 

• Identifies  potential  information  sources. 

• Creates  a general  framework  for  organizing  information  found. 

• Identifies  potential  formats  for  presenting  results. 
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During  Stage 
Exploration 


• Uses  indexes  and  search  engines. 

• Locates  and  accesses  information  resources. 

• Explores  range  of  information  resources. 

• Rethinks  research  topic. 

• Finalizes  formulation  of  research  topic. 

Collection 

• Selects  most  appropriate  information  sources. 

• Skims/scans  information  sources. 

• Locates  relevant  information  within  selected  sources. 

• Identifies  and  extracts  relevant  information  from  selected  sources. 

• Uses  highlighting  and/or  note-taking  skills. 

• Evaluates  quality  of  information  and  information  source. 

• Recognizes  when  sufficient  information  has  been  obtained. 

• Stores  information  for  potential  future  use. 

Organization 

• Analyzes  quality  of  information. 

• Filters  out  irrelevant  information. 

• Summarizes/synthesizes/classifies  final  information. 

• Sequences  final  information. 

• Organizes  final  information  for  presentation 

Ending  Stage 
Presentation 

• Selects  most  appropriate  format  for  communicating  results. 

• Assembles  organized  information  for  presentation. 

• Reviews  presentation  for  grammatical,  spelling,  and  other  errors. 

• Cites  sources  appropriately. 

• Presents  results. 

Evaluation 

• Evaluates  end  product. 

• Assesses  the  efficiency  of  the  research  process. 

• Determines  ways  to  improve  future  research  process  and  results. 

• Determines  future  usefulness/applicability  of  research  process. 

The  Power  of  Motivation 

Through  her  landmark  longitudinal  research  in  which  she  observed  students  as  they  completed  a research 
task,  Kuhlthau  (e.g.,  1991,  1993)  identified  not  only  the  cognitive  strategies  students  used  but  also  the  affective 
feelings  that  students  experienced  at  various  stages  in  the  research  process.  For  example,  she  found  that  during  the 
Beginning  and  During  Stages,  when  students  try  to  identify  a research  topic  and  again  as  they  begin  to  explore  and 
collect  information  on  their  topic  from  information  sources,  they  often  experience  feelings  of  anxiety,  uncertainty 
and  information  overload  and  are  more  likely  to  become  discouraged  and  demonstrate  low  confidence.  She  reports 
that  it  is  not  until  about  midway  through  the  process  that  students  begin  to  resolve  this  uncertainty,  formulate  a more 
precise  research  topic  and  search  strategy,  and  feel  more  confident  and  self-determining. 

In  a study  exploring  the  motivational  strategies  used  by  school  library  media  specialists  when  teaching 
information  literacy  skills  to  students.  Small  (1999)  used  the  ARCS  Model  to  analyze  and  describe  her  findings.  She 
found  that  while  library  media  specialists  incorporated  many  motivational  strategies  into  their  instruction,  the 
majority  of  those  strategies  were  Attention  strategies,  with  very  few  Relevance,  Confidence,  and  Satisfaction 
strategies.  Kuhlthau’s  work,  however,  indicates  a need  to  emphasize  Relevance  and  Confidence  strategies  at  certain 
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critical  points  as  students  proceed  through  the  research  process.  While  current  information  literacy  models  do  a good 
job  of  defining  the  appropriate  concepts  and  skills  to  be  taught  (i.e.,  describing  to  teach),  they  all  lack  a 
systematic  approach  for  applying  motivational  principles  to  the  design  of  information  literacy  skills  instruction  (i.e., 
prescribing  how  to  teach  in  a way  that  stimulates  intellectual  curiosity,  encourages  continued  information  seeking, 
and  promotes  a desire  for  lifelong  learning  and  exploration).  To  help  fill  this  gap,  we  have  developed  a way  to 
enhance  existing  information  literacy  models  with  an  overlay  of  motivational  techniques  and  strategies. 

The  Motivation  Overlay  for  Information  Skills  Instruction  (Small  & Amone,  2000)  integrates  several 
theories  and  models  in  information  science  and  motivation,  especially  the  work  of  Kuhlthau  (e.g.,  1993)  and  Keller 
(e.g.  1987)  to  present  a framework  for  designing  motivating  information  literacy  skills  instruction  (see  Fig.  2).  It  is 
called  an  overlay  because  it  is  meant  to  be  superimposed  on  any  existing  information  skills  model  in  order  to  guide 
the  creation  or  selection  of  motivational  techniques  for  information  skills  lessons. 

The  Motivation  Overlay  for  Information  Literacy  Instruction  promotes  an  information  motivation 
perspective  that  excites  students  about  information  exploration  and  knowledge  discovery  and  encourages  self- 
determination  and  self-efficacy  in  the  development  of  information  literacy  competence  for  lifelong  learning.  “The 
Motivation  Overlay  prescribes  an  SOS  (situation-outcomes-strategies)  framework  for  creating  challenging,  student- 
centered,  information-rich  learning  environments  that  (1)  take  into  account  the  motivational  situation,  including  the 
incoming  motivational  profile  (attitudes  and  motives)  of  students,  (2)  target  desired  motivational  outcomes,  and  (3) 
suggest  broad  motivational  techniques  and  specific  strategies  to  engage  learners  in  and  excite  them  about  the 
process  of  constructing  meaningful  knowledge  and  developing  skills  in  order  to  solve  authentic  information 
problems”  (Small  & Amone,  2000,  p.  23). 

The  Motivation  Overlay  for  Information  Skills  Instmction  specifies  motivational  goals  to  guide  the  design 
of  information  literacy  skills  instmction  and  selection  of  motivational  techniques  and  strategies  for  each  of  the 
research  stages.  Motivational  goals  differ  from  instmctional  goals  in  that  the  former  describe  general  feelings, 
attitudes,  and  motives  to  be  achieved  during  instmction  while  the  latter  are  broad  statements  to  describe  the  intended 
learning  results  of  that  instruction.  Specification  of  motivational  goals  is  a necessary  prerequisite  to  the  selection 
(and  later  evaluation)  of  effective  motivational  techniques. 

Based  on  what  we  know  from  Kuhlthau’s  research,  at  the  beginning  of  the  research  process  when  students 
are  required  to  define  their  research  task,  select  and  narrow  their  research  topic,  and  plan  their  research  strategy, 
motivational  goals  should  lead  to  an  instmctional  design  that  (1)  generates  student  interest  in  the  research  process; 
(2)  helps  students  recognize  the  importance  of  learning  information  literacy  skills;  and  (3)  builds  students’ 
confidence  in  their  ability  to  successfully  conduct  research  tasks.  During  the  research  process,  as  students  explore, 
gather,  and  organize  information,  motivational  goals  should  focus  on  (1)  maintaining  students’  interest  in  the 
research  process;  (2)  promoting  continued  valuing  of  learning  information  literacy  skills;  and  (3)  reinforcing 
students’  confidence  in  their  research  ability.  As  students  proceed  through  the  concluding  stage  of  the  process  in 
which  their  research  results  must  be  presented  and  evaluated,  motivational  goals  should  emphasize  (1)  encouraging 
students’  ongoing  confidence  in  their  research  ability;  (2)  promoting  students’  satisfaction  in  their  research 
accomplishments;  and  (3)  motivating  students’  continuing  information  exploration.  The  underlying  motivation 
theories  for  each  of  these  stages  are  also  specified.  (For  a complete  description  of  these  theories  and  their  relation  to 
the  Motivation  Overlay  for  Information  Skills  Instmction,  see  Small  & Amone,  2000). 


Research  Stages 
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DURING 

ENDING 
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Curiosity 
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Social  learning 

Social  learning 

Social  learning 

Fig,  2,  The  Motivation  Overlay  for  Information  Skills  Instruction 

Each  of  the  motivational  goals  may  then  be  used  to  guide  the  design  of  a “Motivation  Toolkit,”  a set  of 
motivational  techniques  and  strategies  for  accomplishing  a particular  goaf  One  technique  and  an  example  strategy 
for  each  of  the  research  stages  is  presented  in  Figure  3.  (For  complete  toolkits  for  all  three  research  stages,  see  Small 
& Amone,  2000,  pp.  66,  109,  and  146), 


Motivation  Toolkit 
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problem  to  explore. 

DURING  STAGE 
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Reinforce  confidence  by  maintaining  an  Provide  practice  with 

in  research  ability  atmosphere  of  acceptance.  feedback  in  using  online  vp 

I T 

i ENDING  STAGE 

' Motivational  Goal  Technique  Example  Strategy  \ \ 

Encourage  ongoing  by  providing  enrichment  Introduce  students  to 

confidence  in  research  opportunities.  more  advanced  research 

ability.  tools  and  materials. 

L ^ 

Fig.  3.  Example  of  a Motivation  Toolkit  (adapted  from  Moyer  (fe  Small,  2001). 


Each  Motivation  Toolkit  can  be  customized  to  meet  the  motivational  needs  of  a particular  lesson.  In 
addition,  each  Toolkit  can  be  expanded  and  updated  overtime.  The  creation  and  use  of  Motivation  Toolkits  will 
enhance  the  teaching  and  learning  of  information  literacy  skills  for  all  students. 
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Abstract 

The  technology  adaptation  is  a very  complex  process.  This  process  includes  many  different  components  or 
variables  such  as  quality  of  technology  teacher  training,  quality  of  hardware  and  software,  strategies  and 
alternative  ways  of  infusing  software  into  lessons  and  effectiveness  of  class  organization  (District  of 
Columbia,  1992).  In  this  paper,  I will  discuss  computers  and  computer-related  issues  in  the  classroom.  This 
paper  specifically  focuses  on  the  following  issues: 

• How  technology /computers  is  integrated  in  the  classroom 

• How  it  changes  teachers ' classroom  practices 

• What  determines  students ' attitudes  toward  technology 

Computer  Integration 

Computers  have  been  in  schools  for  a long  time.  While  some  teachers  have  integrated  them  into  the 
curriculum  successfully,  some  others  have  not  used  it  at  all.  The  following  three  issues  can  be  considered  important 
in  integrating  computers  in  the  classroom:  what  roles  do  computers  play  in  classroom?  To  what  extent  do  teachers 
employ  them  to  teach?  And  what  are  the  specific  procures  and  models  followed  to  integrate  computers  in  the 
classroom? 

Roles  of  Computers  in  the  Classroom 

Computers  have  different  roles  and  functions  in  the  classroom.  Seven  major  uses  of  computers  in  education 
have  been  discussed  in  the  literature:  drill -and-practice,  tutorial,  problem  solving,  simulation,  inquiry,  testing  and 
programming.  In  the  drill-and-practice,  students  are  introduced  new  concepts  and  skills,  and  then  computer  gives 
practice  in  using  them.  A Spanish  teacher,  for  example,  may  spend  one  lesson  explaining  the  use  of  the  imperfect 
tense,  and  for  the  next  lesson  may  design  a computerized  practice  in  handling  this  tense. 

In  the  tutorial,  computer  introduces  and  explains  concepts  and  skills  in  which  it  gives  practices.  Similar  to 
the  programmed  instructional  text  and  teaching  machines,  the  tutorial  presents  informational  frames  and  also  asks 
questions  about  them.  In  the  problem  solving,  students  are  expected  to  solve  problems  posed  by  the  computer.  The 
computer  structures  the  problems  in  a way  that  students  identify  the  solution  step  by  step  and  at  the  end  of  each 
successful  step  the  computer  provides  scheduled  rewards.  Students’  behaviors  are  shaped  towards  thinking  about 
and  tackling  with  problem  solving. 

In  the  simulation,  students  confront  an  environment  that  operates  under  certain  rules.  Their  role  is  to  act 
within  this  environment  and  then  observe  the  results.  For  example,  in  a geography  course,  students  can  create  their 
own  trip  across  the  Atlantic  and  make  adjustments  along  the  way  accounting  changes  in  wind  and  currents.  Students 
also  may  use  computers  to  retrieve  information  from  diskettes,  CDs,  remote  data  sources  to  reach  information  from 
books  and  periodicals  or  the  product  of  electronic  publications  that  appear  in  no  other  form.  The  most  common 
application  of  the  computers’  inquiry  function  would  be  searching  the  World  Wide  Web  pages  to  gather  data  to 
solve  a problem. 

Computers  are  also  ideal  for  presenting  and  scoring  tests.  They  can  automatically  adjust  the  difficulty 
levels  of  test  items  based  on  students’  responses  so  that  students’  performances  are  measured  more  precisely  in  less 
time.  On  the  other  hand,  computers  are  able  to  improve  students’  learning  performance  by  providing  instant 
feedback  based  on  students’  answers.  Computers  are  programmable  tools.  There  are  sets  of  instructions  written  one 
of  several  codes  called  programming  languages.  Although  students  need  to  know  how  to  program  to  benefit  from 
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computer  technology,  gaining  such  a skill  gives  them  greater  control  of  the  medium  and  opens  opportunities  for  later 
employment. 

Teachers’  Level  of  Computer  Use 

It  should  not  be  expected  from  every  teacher  to  use  the  technology  at  the  same  level  because  of  the 
differences  in  their  enthusiasm,  knowledge  and  competencies.  Hardy  (1998)  describes  five  different  types  of 
teachers  using  technology  at  different  levels.  The  first  type,  enthusiastic  beginners,  prefers  very  basic  computer 
applications  to  support  their  direct  instruction.  The  second  type,  supported  integrators,  employs  computers  for  the 
following  purposes:  enabling  students  to  create  their  own  products  and  helping  them  demonstrate  their  skills  and 
ideas  during  the  class.  Supported  integrators  use  varieties  of  computer  programs  and  allow  students  explore  those 
programs  by  themselves.  High  school  naturals  are  concentrating  on  quantitative  and  analytic  functions  of  computers 
and  they  mostly  use  computers’  programming  function.  Unsupported  achievers  employ  computers  for  remediation 
and  they  do  not  make  much  use  of  sophisticated  applications.  Finally,  struggling  aspires  make  very  limited  use  of 
computers  for  their  direct  instruction. 

Moersch  (1995)  describes  seven  levels  of  computer  uses  in  the  classroom.  At  the  first  level  use  teachers  do 
not  use  any  electronic  technology.  The  technology  they  use  is  text -based,  such  as  chalkboard  and  overhead 
projector.  In  the  second  level,  teachers  do  not  use  computers  directly  in  the  classroom.  Students  use  computers 
outside  of  class  to  perform  some  tasks,  such  as  writing  papers  with  word  processors  and  creating  data  sheets  with 
spreadsheet  software.  In  this  case,  there  is  very  little  relevance  to  the  teachers’  instructional  activities.  The  third 
level  application  is  using  computers  as  a supplementary  tool.  Tutorial,  game  or  simulation  programs  are  used  to 
extent  class  activities  or  provide  enrichment.  In  the  infusion  level,  which  is  the  fourth  level,  varieties  of  software 
programs  are  used,  such  as  databases,  graphing  packages  and  multimedia  applications  in  classroom.  However,  those 
applications  are  isolated  from  instructional  events.  For  example,  communication  tools  are  used  among  people  to 
only  share  data  rather  than  actual  teaching  and  learning.  The  forth  level  of  use  are  more  teaching  and  learning 
oriented.  Conputers  are  used  for  presenting  information  in  a meaningful  way  to  students  so  that  students  are  put  in  a 
real-like  environment  for  authentic  learning.  At  the  expansion  stage,  technology  goes  beyond  classroom.  Students 
use  computers  outside  of  class  to  perform  lesson-related  tasks.  The  final  stage  is  refinement.  In  this  stage,  the 
problems  are  authentic.  The  computer  is  the  major  medium  to  search  and  process  data  for  the  problems  and  to  bring 
the  authentic  solutions. 

Planning  the  Integration 

The  use  of  conputers  in  classroom  should  be  carefully  planned  for  successful  implementations 
(Grangennet  et  al.,  1997).  The  literature  describes  several  systematic  and  non-systematic  instructional  planning 
models  for  teachers  to  create  technology-integrated  lessons. 

Guide  (1989)  indicates  that  in  a technology  planning  process  teachers  should  first  identify  the  content  of  a 
lesson,  goals  and  objectives  of  the  lesson,  and  learning  activities  with  computers.  Once  those  are  identified,  the 
lesson  should  be  planned  at  four  levels:  Introduction,  information  presentation,  guided  practice  and  closure.  In  the 
introduction,  students  should  be  motivated  through  discussing  the  topic  and  relevant  experiences,  and  posing 
attention-getting  questions.  During  the  introduction,  the  computer  can  be  employed  to  present  graphical  data  to 
initiate  the  discussion,  record  the  points  and  interesting  views  elicited  during  the  discussion.  Also,  computer 
printouts  can  be  provided  with  students  to  make  them  ready  for  the  following  activities.  In  the  information 
presentation,  students  are  confronted  the  knowledge,  skills  and  competencies  to  be  learned.  Usually,  computers  are 
used  to  present  those  skills  to  students  directly  or  help  students  manipulate  the  new  information.  For  instance, 
students  may  use  a tutorial  software  program  because  these  programs  explain  information  to  be  learned  step-by-step. 
Guided  practice  is  structured  to  provide  independent  practice  to  reinforce  students’  newly  learned  skills  and 
knowledge.  For  example,  drill  and  practice  software  can  be  used  after  learners  understand  how  to  solve  certain 
mathematical  problems.  In  closure,  teachers  should  summarize  the  lesson  and  mention  how  students  may  use  their 
new  skills  to  solve  real  life  problems.  Presentation  or  simulation  software  would  be  appropriate  for  this  stage. 

Another  framework  for  technology  integration  is  NTeQ  (integrating  technology  for  inquiry)  that  provides  a 
guideline  to  create  a computer-integrated  educational  environment  through  solving  meaningful  problems.  According 
to  NTeQ  framework,  to  plan  a lesson,  the  following  steps  should  be  taken:  specifying  objectives,  matching  the 
objectives  to  computer  functions,  specifying  a problem,  planning  the  data  manipulation,  planning  the  presenting  of 
results,  planning  student  activities  and  planning  evaluation.  At  the  first  phase,  specifying  objectives,  teachers  are 
supposed  to  cover  all  the  skills  students  need  to  gain  during  the  lesson  or  unit.  Then,  teachers  evaluate  each 
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objective  and  match  them  with  one  or  more  functions  of  specific  software  programs.  The  functions  are  the  tasks  that 
software  can  do,  such  as  calculation,  drawing,  matching  and  searching.  After  that,  teachers  create  realistic  problems. 
The  problems  are  necessary  because  they  will  create  an  environment  where  students  will  generate  critical  thinking 
skills  and  gain  necessary  knowledge  to  reach  the  objectives.  Also,  teachers  should  consider  how  students  would 
collect  data  related  to  the  problems.  They  may  identify  a potential  use  of  computers  in  data  collection.  Planning  the 
data  manipulation  is  related  to  how  students  will  use  the  computers  or  what  functions  of  computers  they  will  employ 
to  solve  the  problems.  After  students  solve  the  problems  with  computers  teachers  should  determine  in  what  format 
students  would  present  the  results.  Different  presentation  formats  may  help  students  see  the  results  from  different 
critical  perspectives  that  would  help  make  appropriate  conclusion.  The  next  step  is  planning  student  activities. 
Teachers  should  figure  out  what  kind  of  activities  would  be  better  for  students  to  solve  the  problems.  The  activities 
can  either  involve  computer  use  or  other  classical  applications,  such  as  discussion  and  lecturing.  The  final  stage  is 
evaluation.  Teachers  should  consider  wide  varieties  of  evaluation  techniques,  such  as  paper-and-pencil  tests,  a rubric 
and  journal  (Lowther  & Morrison,  1998) 

Sia  (1992)  says  that,  “when  a teacher  utilizes  software  to  enhance  instruction  in  a specific  subject,  he/she  is 
putting  software  infusion  into  practice.’’  The  purpose  of  the  software  infusion  is  to  achieve  predetermined  lesson 
objectives  by  incorporating  appropriate  computer  programs.  The  followings  should  be  considered  in  software 
infusion.  Curriculum  and  software  objectives  should  be  aligned.  Computer  programs  should  be  used  wherever  they 
may  make  contribution  to  the  purpose  of  a lesson.  Teachers  should  not  intend  to  teach  a whole  lesson  by  a single 
software  program.  Rather,  they  should  consider  using  wide  range  of  software  applications  with  their  unique 
contributions  to  curriculum  during  a lesson.  Teachers  should  not  see  technology  integration  very  complicated  and 
narrow  their  view  about  the  utilization  of  the  computer.  For  instance,  besides  using  chalkboard,  they  may  use  a 
word-processing  program  to  help  students  learn  complex  writing  skills. 

New  technologies  that  can  be  used  for  educational  purposes  may  require  new  planning  and  integration 
strategies.  Internet  is  diffusing  into  education  very  quickly.  Several  internet-based  lesson  templates  have  been 
created  for  effective  teaching.  The  WebQuest  is  a web-supported  lesson  template  composed  of  five  sections: 
Introduction,  task,  resources,  evaluation  and  conclusion.  The  purpose  of  introduction  is  to  orient  learners  about  what 
they  will  learn  and  gain  their  attention.  The  task  section  describes  what  student  will  be  doing  during  the  lesson.  The 
resources  have  a list  of  pre-selected  web  pages  that  have  resources  to  help  learners  accomplish  the  task.  However, 
when  needed,  students  may  use  resources  other  than  the  provided  web  pages,  such  as  books,  tapes  and  face-to-face 
interaction.  In  the  evaluation  section,  the  teacher  explains  students  how  they  will  be  evaluated  on  the  task  they  have 
to  accomplish.  The  final  section,  conclusion,  summarizes  the  experience  and  generalizes  what  was  learned. 

Technology,  Teacher  and  Classroom 

Technology  has  changed  the  teachers’  traditional  role  and  expectations  in  the  classroom.  Hardley  and 
Sheingold  (1993)  indicate  that  with  technology,  classrooms  have  been  changed  from  a teacher-centered  educational 
environment  to  a student -centered  environment.  Teachers  see  themselves  as  learning  facilitators  or  tutor  providing 
students  with  help  when  they  encounter  problems  in  the  learning  process  rather  than  as  an  expert  who  performs 
direct  teaching.  Students  work  more  actively  on  their  own  within  small  groups  in  a collaborative  way  (Schofield, 
1995). 

Berg  et  al.  (1999)  observed  that  exemplary  computer-using  teachers  employ  technology  in  their  classrooms 
in  a manner  that  are  overwhelmingly  constructivist.  On  the  other  hand,  students  use  technology  as  a tool  to  explore 
new  information  and  create  new  products.  Because  student  groups  work  independently  in  classroom  teachers 
generate  more  time  to  deal  with  students’  individual  problems  and  provide  more  learning  materials  for  them  so  that 
they  may  be  able  to  learn  and  think  more.  Moersch  (1995)  states  that  “As  teachers  advance  from  one  step  to  another 
step  [improves  their  technology  use  step-by-step]  instructional  focus  shifts  from  being  teacher  centered  to  learner 
centered.  The  computer  technology  is  employed  as  a tool  that  supports  and  extends  students’  understanding  of 
pertinent  concepts,  processes  and  themes  involved  when  using  databases,  telecommunication,  multimedia, 
spreadsheets  and  graphing  applications.  Traditional  verbal  activities  are  gradually  replaced  by  authentic  hands-on 
inquiry  related  to  a problem,  issue  or  theme.” 

Student  Attitude  and  Anxiety  towards  Computers 

There  are  two  important  issues  that  may  affect  the  success  of  technology  infusion  in  classroom:  students’ 
attitude  toward  and  anxiety  about  computers.  Computer  attitude  can  be  defined  as  an  evaluative  disposition  towards 
computers.  On  the  other  hand,  computer  anxiety  is  a fear  or  aversive  behavior  that  occurs  when  students  use 


460 

.94  9 


computers.  Computer  attitude  and  anxiety  are  among  the  major  factors  relating  to  students’  success  in  learning  (Lui 
& Johnson,  1998).  Moreover,  anxiety  affects  “the  ability  of  individuals  to  use  computers”  (Anderson  et  al.,1984) 
and  cause  physical  symptoms  and  discomfort  (Ayersman,  1996).  Due  to  their  importance,  the  constructs  of 
computer  attitude  and  anxiety  occupy  a significant  place  in  the  literature  (Lui  & Johnson,  1998). 

Two  factors  are  considered  important  in  computer  attitude  and  anxiety:  gender  and  experience. 
Investigating  the  relationship  between  gender  and  computer  attitude,  research  studies  revealed  inconsistent  results. 
Nelson  and  Cooper  (1997)  indicate  that  there  is  no  significant  attitudinal  difference  between  boys  and  girls  towards 
computers.  However,  girls  use  computer  less  than  boys  do  and  see  themselves  less  skillful.  Male  students  show 
better  attitude  toward  computers  than  female  students  do.  When  programming  is  concerned  there  is  a marked 
attitudinal  difference  toward  computers  between  boys  and  girls  in  the  favor  of  boys.  Sacks  et  al.  (1994)  indicated 
that  differences  in  findings  may  be  due  to  instability  in  girls’  attitude  about  computer  across  time.  Also,  another 
important  finding  is  that  boys  provide  unstable  attribution  to  their  failure  but  girls  provide  unstable  attribution  to 


Also,  studies  investigating  the  relationship  between  gender  and  anxiety  revealed  inconsistent  results. 
Mclnemet  et  al.  (1994)  indicates  that  in  using  computer  equipment  and  dealing  with  computer  instructions  males  are 
less  anxious  then  females  and  boys  are  more  relaxed  than.  Chuna’s  et  al.  (1999)  extended  study  revealed  that  female 
undergraduates  are  more  anxious  than  male  university  undergraduates  when  they  use  the  computer.  In  the  contrary, 
the  study  by  Anderson  et  al  (1 984)  do  not  support  the  previous  findings,  which  is  that  women  in  general  exhibit 
higher  levels  of  computer  anxiety  than  man.  Maurer  (1994)  took  a different  approach  to  the  gender  issue.  Regardless 
of  the  gender,  he  described  students  with  different  identity  as  feminine,  masculine,  androgynous  and 
undifferentiated.  The  results  showed  that  feminine  identity  students  had  more  computer  anxiety  than  did  masculine 
identity  students  regardless  of  gender. 

Positive  relationship  has  been  found  between  computer  experience  and  attitude- the  more  computer 
experience  students  have  the  higher  attitude  they  show  toward  the  computer  (Levine  & Donita,  1998).  It  was  also 
found  that  computer  experience  might  diminish  the  attitudinal  difference  between  males  and  females.  After  having 
had  the  experience,  female  students  exhibit  better  attitude  toward  computers.  However,  Maurer  (1994)  advocates 
that  there  is  a controversy  in  the  literature  about  experience.  Not  always  experience  brings  high  attitude.  He  says  that 
computer  attitude  is  related  to  the  types  of  computer  experiences  or  computer  education  students  have  .On  the  other 
hand,  Ayersman  (1996)  indicates  that  any  positive  computer  experience  may  reduce  the  computer  anxiety,  including 
workshops,  classes,  hands-on  experiences  and  training.  Even  having  experience  with  a privately  owned  computer  at 
home  improves  students’  attitude. 

In  addition  to  experience  and  gender,  other  variables  are  affective  on  computer  attitude  and  anxiety.  In 
science  and  math  related  areas,  students  experience  attitude  and  anxiety-related  problems  (Sacks  et  al.,  1994). 
Teacher  attitude  also  affects  student  attitude  in  the  classroom.  (Dupagne  & Krendl,  1992).  If  students  are  in  a self- 
directed  learning  environment  which  is  free  of  evaluation  and  assessment  they  have  better  attitude  and  show  less 
anxiety  toward  computer  than  the  otherwise  (Mclnemet,  1 994).  Finally,  access  to  computer  determines  students’ 
attitude  toward  computers  (Lui  & Johnson,  1998).  “Studies  have  showed  that  females  are  less  likely  to  use 
computers  if  they  have  to  compete  for  use  time”  (Sacks  et  al.,  1994). 
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Abstract 


We  are  in  the  computer  age  and  the  federal  and  local  governments  have  invested  significant  amount  of 
money  to  purchase  computers  for  schools.  The  average  computer  to  student  ratio  has  significantly 
increased.  In  the  meantime,  educational  technologists  have  started  questioning  the  effectiveness  of 
computers  in  teaching  and  learning.  The  purpose  of  this  paper  is  to  investigate  advantages  and 
disadvantages  of  using  technology  in  education  and  to  identify  the  environmental  (extrinsic)  and  personal 
(intrinsic)  factors  that  cause  schoolteachers  and  university  professors  unwillingness  and  lack  of  enthusiasm 
to  implement  technology  in  the  classroom. 


Technology  has  a long  history  in  education.  Even  though  it  has  been  accepted  that  the  first  educational 
technology  endeavors  started  at  the  end  of  the  1800s  with  the  school  museums,  the  technology  widely  started  being 
used  after  film,  radio  and  television  entered  in  the  vision  around  the  1920s.  Public  schools  modified  their  programs 
to  some  extent,  such  as  rearranging  class  schedules,  to  infuse  those  technological  innovations  into  the  school  system 
(Cuban,  1985).  However,  schools  came  across  several  problems  including  cost  of  films,  insufficient  hardware, 
awkward  scheduling  of  broadcasts  and  inadequate  teacher  training.  Moreover,  the  technological  innovations  at  that 
time  did  not  bring  a new  educational  perspective.  For  instance,  TVs  were  used  as  an  accessory  and  substituted  by 
teachers’  lectures  rather  than  as  an  essential  instructional  tool.  Eventually,  they  disappear  from  schools  very  rapidly. 

Now  we  are  in  the  computer  age  and  the  federal  and  local  governments  have  invested  significant  amount  of 
money  to  purchase  computers  for  schools.  The  average  computer  student  ratio  significantly  increased.  In  the 
meantime,  educational  technologists  have  started  questioning  the  effectiveness  of  computers  in  teaching  and 
learning.  The  effectiveness  of  computers  is  still  an  issue  and  a consensus  has  not  been  provided  among  the 
researchers.  However,  the  computers,  as  opposed  to  the  other  technologies  such  as  TV  and  radio,  have  significantly 
impacted  education  from  several  perspectives.  It  has  changed  the  teachers’  role,  the  school  curriculum  and  the 
educational  research  perspective. 

The  current  studies  illustrate  that  the  teachers  who  are  successfully  employing  the  computers  change  their 
classical  roles  in  the  classroom,  which  are  organizing,  presenting  and  evaluating  information.  They  prefer  to  become 
a mentor  directing  and  motivating  students  to  create  questions,  explore  and  manipulate  information,  and  create 
solutions  for  the  questions  by  themselves.  They  use  more  student-centered  teaching  approach  and  promote  self- 
directed  learning.  Aligned  with  this  approach,  the  curriculum  has  been  modified  to  accommodate  more  problem- 
solving activities  (Diem,  1996). 

The  potentials  of  computers  have  shifted  researchers’  attention  to  cognitive  psychology  (Shrock,  1995). 
Psychologists’  traditional  approach  to  education,  which  is  measuring  changes  in  students’  behavior  following 
educational  stimuli,  is  joined  with  a concern  about  the  process  of  cognition.  They  started  considering  human 
learning  as  the  product  of  a cognitive  process  working  based  on  cognitive  rules  and  strategies  rather  than  solely 
stimulus-reinforcement  contingency.  According  to  cognitive  principles,  education  is  control  of  cognitive  processes 
in  the  learner.  Learning  is  more  and  more  being  understood  as  recognition  of  learner  knowledge  structures  by 
mapping  teacher  or  expert  subject  matter  knowledge  onto  the  learner’s  knowledge  structure  (Popkewitz  & Shutkin, 
1995). 

Advantages  of  Using  Technology 

Technology  has  produced  several  positive  outcomes  in  education;  enhancement  of  motivation,  attitude  and 
enjoyment,  new  peer  interaction  patterns  and  learning  performance  (Schofield,  1995).  The  idea  that  computer  use 
often  enhances  students’  motivation  and  attitude  has  been  experimented  frequently  in  the  recent  years.  It  is 
supported  by  many  studies  that  computers  are  able  to  enhance  student  motivation  about,  interest  in  and  attention  to 
classroom  activities. 

In  a study  consisting  of  30  fifth  grade  students  in  an  inner  city  school,  the  effect  of  computer  in 
mathematics,  language-arts  and  social  studies  were  measured.  The  results  showed  that  the  students  had  more 
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positive  attitudes  towards  the  educational  experience  and  their  attitudes  improved  during  the  course  of  the  study  in 
the  area  of  confidence  (Kitabchi,  1987).  Based  on  a meta  analysis  of  199  papers  of  which  32  were  conducted  in 
elementary  schools,  43  in  high  schools,  101  in  universities  and  24  in  adult  education  centers,  it  was  founded  that 
students  liked  the  classes  more  and  developed  more  positive  attitudes  toward  computers  when  they  received 
computer  help  (Kulik  et  al,  1987). 

Having  high  attitude,  students  have  more  enjoyment,  work  harder  and  show  more  involvement  in 
classroom  while  working  on  computers.  Why  are  students  motivated  and  enthusiastic  with  computers?  The  literature 
brings  some  explanations  for  this  question.  The  first  is  that  students  respond  well  to  computers  because  they  are 
relatively  new  in  their  school  experience.  The  second  is  that  students  seem  they  like  working  on  computers  because 
computers  introduce  variety  into  the  school  routine.  Another  approach  is  that  knowing  how  to  use  a computer  would 
be  useful  to  students  in  the  later  life  (Schofield,  1995). 

Educators  have  been  concerned  with  interactions  among  students  in  a computer-supported  instructional 
environment.  Many  think  that  students  are  isolated  from  the  classroom’s  social  environment  when  using  computers. 
Yet,  studies  showed  that  when  students  deal  with  computer  related  tasks  interaction  among  students  increases.  As  a 
matter  of  fact  the  interaction  becomes  a learning  management  tool  and  students*  experiences  in-group  work  may 
have  a direct  effect  on  learning  and  achievement  (Webb,  1989) 

Peer  interaction  has  been  affected  by  several  factors  such  as  student  characteristics,  the  structure  of  tasks, 
and  the  reward  structure  (Webb,  1989).  Besides  those  factors,  locations  of  computers,  the  ratio  of  students  to 
computers  and  how  teachers  chose  to  handle  the  educational  environment  when  there  are  more  students  than 
computers  determine  the  pattern  of  interaction  among  students.  Teachers  may  control  collaboration  among  students 
through  software.  One  of  the  purposes  of  the  education  is  to  teach  students  necessary  skills  to  prepare  them  for  the 
job  market.  Even  though  there  are  some  objections  numerous  researchers  indicate  that  the  computer  is  a good  means 
towards  that  purpose.  Many  studies  proved  that  students  better  learn  in  a computer-supported  environment  than 
traditional  classroom  environment. 

Kulik  (1994)  conducted  a meta  analysis  on  more  than  500  individual  research  studies  of  computer-based 
instruction.  He  found  that  on  average,  students  who  used  computer  based  instruction  scored  at  the  64^*^  percentile  on 
the  tests  of  achievement  compared  to  students  in  the  control  conditions  without  computers  who  scored  at  the  50‘*^ 
percentile.  Also  students  learned  more  in  less  time  when  they  received  computer-based  instruction.  In  a similar 
investigation,  219  research  studies  on  the  effect  of  technologies  on  learning  and  achievement  across  from  1990  to 
1997  revealed  that  students  in  technology  rich  environments  experienced  positive  effects  on  achievement  in  all 
major  subject  areas  and  showed  increased  achievement  in  preschools  through  higher  education  for  both  regular  and 
special  needs  children  (Sivin-Kachala,  1998). 

Disadvantages  of  Using  Technology 

The  technology  implementations  are  not  free  of  risk.  Once  the  technology  starts  diffusing  into  schools  it 
comes  with  unique  problems.  Some  of  the  important  problems  are  equity  and  access,  time  to  plan  and  implement  the 
technology  and  teachers*  resistance  to  change. 

In  the  technology -based  educational  change,  which  could  be  school-wide,  d istrict-wide  and  statewide, 
equity  and  access  refers  to  whether  each  individual  student  utilizes  computers  or  the  technology  at  the  same  level 
and  under  the  same  conditions  (Knupfer,  1995).  The  following  variables  would  explain  the  obstacles  that  may  cause 
inequity  in  the  computer  access:  Geographic  region,  socioeconomic  status,  gender,  race,  various  kinds  of  handicaps, 
and  special  learner  groups  within  school.  The  research  indicates  that  minorities,  women,  the  handicapped  and  the 
poor  have  less  access  to  computers  (Anderson,  Welch,  & Harris,  1984). 

Some  other  factors  also  may  determine  the  equal  access,  such  as  familiarity  with  hardware  and  software, 
the  classroom  structure,  time,  students*  skill  levels  and  locations  of  computers.  Becker  (1985)  found  that  above- 
average  students  dominantly  use  computers.  Also,  his  study  revealed  that  placement  of  computers  within  libraries 
promoted  more  equal  usage  of  computers  between  above-  and  below-average  students. 

Successful  technology  adaptation  requires  a careful  planning  which  demands  plenty  of  time.  However, 
teachers  already  undergo  time  shortage  with  their  current  tasks  (Knupfer,  1995;  Hardy,  1998).  Time  necessary  for 
technology  adaptation  is  not  just  limited  to  the  planning.  Teachers  also  have  to  commit  some  time  to  learn  how  to 
plan  the  technology  integration  into  curriculum  and  develop  appropriate  materials.  After  all,  they  will  need 
classroom  time  to  implement  the  technology.  In  the  current  education  system,  besides  other  necessary  classroom 
events  not  enough  time  is  left  to  carry  out  instructionally  sound  and  proper  computer  activities  (Dupagne  & Krendl, 
1992).  The  literature  confirms  that  teachers  who  are  motivated  to  use  the  computer  technology  in  their  teaching  are 
more  likely  to  do  so  if  time  is  provided  to  develop  materials  (Hardy,  1998). 
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Teachers  show  resistance  to  educational  change  in  which  they  should  use  educational  computing.  Among 
several  others,  two  concerns  are  critical  for  teachers  exhibiting  the  resistance:  Concerns  about  their  machine  skills 
and  concerns  about  taking  a risk  (Andris,  1996).  Teachers  are  supposed  to  be  competent  about  computer  machine- 
related  skills  for  classroom  and  lab  activities  at  least  at  the  elementary  level.  Though,  usually  teachers  learn  those 
skills  through,  if  possible,  school  or  district  supported  training  and  peer  tutoring  after  for  a while  they  do  not  value 
their  computer  machine  skills.  Although  these  teachers  agree  that  their  machine  skills  improve  over  time  as  they 
operate  computers,  they  distinguish  those  skills  fromother  teaching  skills  and  do  not  recognize  them  as  relevant  to 
their  teaching  and  they  do  not  think,  “operating  computers  make  them  a better  teacher”. 

It  takes  time  for  teachers  to  become  familiar  with  computer  hardware  and  software.  Because  technology 
vendors  continuously  upgrade  their  products  and  schools  always  acquire  new  equipment  and  computer  programs, 
this  is  a recurrent  problem  in  schools.  Teachers  indicate  that  until  becoming  accustomed  to  computers  and  programs 
their  schools  have  teaching  with  them  becomes  less  efficient  and  less  productive  than  teaching  with  classical 
methods.  They  think  using  unfamiliar  computer  materials  and  methods  may  risk  their  consistent  level  of  classroom 
performance  as  well  as  effectiveness  of  their  lessons. 

The  Effective  Integration  of  Technology 

The  technology  integration  in  classroom  is  perceived  a complex  and  challenging  procedure  by  new 
adaptors.  Yet,  later,  getting  more  expert  on  educational  technology  competencies  they  see  the  integration  easy  and 
useful  (Scrogan,  1989).  Computers  are  promising  educational  tools  facilitating  teachers’  tasks  and  improving 
students’  performance.  In  addition  to  those,  technology  plays  a central  role  in  educational  change  (Sudzina,  1993). 
However,  still  educators  exhibit  reluctance  to  integrate  computers  into  classroom  (Dunn  & Ridgway,  1991). 

Researchers  have  been  investigating  the  reasons  why  educators  at  all  levels,  schoolteachers  and  university 
professors;  show  unwillingness  and  lack  of  enthusiasm  about  the  technology.  Several  causes  have  been  discussed. 
However,  it  is  very  hard  to  put  those  reasons  into  an  accurate  categorization  because  they  are  not  clearly  separated 
from  each  other.  Besides  their  effects  on  inadequate  computer  integration,  they  interact  with  each  other,  as  well. 
However  a categorization  of  the  factors  would  be  as  follow:  Environmental  (extrinsic)  factors  and  personal 
(intrinsic)  factors  (Dusic,  1998;  Ertment,  1999).  Intrinsic  factors  are  the  ones  caused  by  the  setting  or  situation  in 
which  the  technology  is  implemented.  Extrinsic  factors  are  coming  from  teachers’  personalities  and  understanding 
of  technology  integration. 

Environmental  factors 

Providing  adequate  hardware  and  software  is  an  important  factor  in  promoting  technology  integration 
(Zammit,  1992).  If  computers  are  not  available  during  convenient  times  and/or  software  is  unavailable  in  sufficient 
quantities  or  at  an  adequate  level  of  quality  one  should  not  expect  high  levels  of  usage  regardless  of  the  level  of 
interest  (Stieglitz  & Costa,  1988).  Schoolteachers  and  university  professors  indicate  that  the  quality  and  quantity  of 
hardware  and  software  is  insufficient  (Hoffman,  1998).  Schools  and  departments  need  more  computers  and 
computer  peripherals,  such  as  scanners  and  data  projectors.  The  ones  having  computers  and  peripherals  should 
continuously  upgrade  them  due  to  the  rapid  change  in  technology.  Software  is  not  satisfying  the  educators’  need. 
Teachers  generally  evaluate  software  as  being  pedagogically  weak  or  inappropriate  and  think  it  is  not  worth  the 
effort  to  use  it  (Zammit,  1992).  What  is  generally  needed  is  well  designed,  adaptable,  user-friendly  and 
uncomplicated  computer  programs.  (Hardy,  1998;  Downes,  1993;  Ritchie,  1996;  Cafolla  & Knee,  1995;  Sheingold 
& Hadley,  1990;  Dunn  & Ridgway,  1991) 

Support  also  plays  an  important  role  in  technology  diffusion  in  education.  The  support  may  be  in  three 
different  forms:  Technical  support,  pedagogical  support  and  management  support.  Technical  support  is  important 
because  teachers  and  faculties  always  need  help  with  the  equipment  in  classrooms.  Most  of  the  time,  they  are  not 
able  to  overcome  technical  problems  occurring  during  instructions  and  need  to  call  a support  person  (Dusic,  1998; 
Hardy,  1998). 

Pedagogical  support  is  related  to  technology  planning,  development,  implementation  and  teacher 
consulting.  Pedagogical  support  should  be  provided  by  technology  coordinators  (Zammit,  1992).  Technology 
coordinators  are  supposed  to  inform  teachers  of  how  to  use  certain  equipment.  Equipment  use  is  not  necessarily  only 
limited  to  physical  use  but  related  to  how  that  piece  of  technology  is  integrated  into  instruction,  how  to  plan  for  its 
use,  and  how  to  improve  students  learning  performance  and  motivation.  Also,  technology  coordinators  should 
enlighten  teachers  with  concurrent  educational  technology  innovations  and  learning  theories/models  (Ritchie,  1996). 
Hoffman  (1998)  claims  that  the  pedagogical  support  provided  by  the  coordinators  leads  to  greater  use  of  software 
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that  promotes  higher  order  thinking  skills,  and  greater  use  of  computers  as  tools  in  academic  activities  rather  than  as 
mere  drill -an d-practice. 

School  boards,  districts  and  school  management  are  not  providing  adequate  administrative  support  for 
technology  infusion  (Cafolla  & Knee,  1995).  Administrators  from  different  management  levels  are  key  people 
making  strategic  and  executive  decisions  within  schools  or  school  systems  and  universities.  With  those  decisions, 
administrators  may  provide  teachers  with  directions  about  educational  technology  use,  involve  teachers  in  the 
technology  adaptation  process,  provide  necessary  hardware  and  software,  provide  incentives  that  can  encourage  and 
motivate  teachers  to  start  and  continue  integrating  technology  into  their  lessons  (Hoffman,  1998;  Knupper,  1989; 
Dupagne  & Krendl,  1992).  One  solution  to  overcome  this  problem  and  widespread  the  technology  in  schools  is  to 
train  administrators  on  educational  technology  and  make  them  comfortable  computer  users  so  that  their  attitude 
towards  technology  is  improved  and  they  provide  more  help  teachers  to  integrate  technology  in  their  lessons 
(Ritchie,  1996).  The  following  factors  are  also  considered  related  to  teachers’  educational  computer  use:  Risk  of 
using  technology,  sharing  of  technology  resources  between  teachers  (Dusic,  1998),  discouraging  climate  to  use 
computer  within  schools,  lack  of  use  of  computers  for  personal  purposes  and  not  having  a computer  at  home 
(Downes,  1993). 

Teacher  training 

To  this  point,  several  environmental  factors,  such  as  lack  of  hardware  and  software,  pedagogical  and 
technical  support,  management  support  and  so  forth,  have  been  discussed.  Teacher  training  also  occupies  an 
important  place  as  an  environmental  obstacle  (Hardy,  1998;  Dusic,  1998).  Significant  number  of  teachers  had  very 
little  in-service  training  about  educational  technology  (Zummit,  1992).  This  might  be  the  reason  that  one  of  the 
major  concerns  teachers  and  faculties  carry  is  “hows”  of  using  technology  in  the  classroom  (Dupagne  & Krendl, 
1992).  Instructional  computer  applications  require  new  competencies  and  knowledge.  Not  having  those 
competencies  and  knowledge,  teachers  should  not  be  expected  to  adopt  technology  in  the  classroom 
(Marcinkiewicz,  1995).  Successfully  technology -using  teachers  indicate  that  they  learned  their  technology  skills 
thorough  formal  and  non-formal  training;  such  as  workshops,  courses  at  local  colleges,  in-service  training  offered  by 
their  districts,  in  -service  training  at  their  school  site  and  non-in-service  courses  offered  by  districts  (Hoffman,  1998). 

Even  though  teachers  have  positive  attitudes  towards  technology  and  want  to  improve  their  teaching 
performance  through  technology  implementations  they  are  not  able  to  accomplish  it.  They  are  not  having  knowledge 
to  use  the  machine,  and  not  having  any  kind  of  familiarity  or  expertise  with  computer  based  or  computer  managed 
instruction  (Onika,  1992).  The  reason  is  that  experienced  teachers  have  not  had  appropriate  training  on  how  to 
effectively  use  the  computer  in  the  classroom  and  on  technology,  skills,  ideas  and  ways  to  integrate  instructional 
technology  into  the  curriculum  (Dunn  & Ridgway,  1991).  In  addition  to  this,  new  teachers  have  very  limited 
knowledge  about  educational  computer  use.  Teacher  students  do  not  have  adequate  exposure  to  instructional 
technology  because  many  educational  institutions  and  faculties  within  those  institutions  have  not  adopted 
technology.  Whereas,  the  more  teachers  have  exposure  to  and  experience  with  computers  the  better  they  integrate 
computers  into  their  teaching.  (Hardy,  1998) 

Comprehensive  staff  technology  teacher  development  models  and  programs  are  needed.  Those  programs 
should  provide  clear  directions  for  teachers  on  integrating  technology  in  classroom  and  help  them  construct  the 
purpose  and  meaning  of  educational  technology  (Hardy,  1998).  The  training  should  be  designed  in  a way  so  that  it 
contributes  to  teachers’  continuous  development.  Therefore,  new  adopters  or  new  teachers  should  be  encouraged  to 
try  out  their  developing  IT  skills  early  in  their  carriers,  and  not  wait  until  their  theoretical  knowledge  is  highly 
developed  (Dunn  & Ridgway,  1991). 

Personal  Factors 

Besides  environmental  factors  as  explained  above,  some  psychological  factors  or  variables,  such  as 
confidence,  fear,  will  and  motivation,  may  determine  teachers’  use  of  technology  in  classroom.  Hardy  (1998) 
indicates  that  around  40-50  percent  of  teachers  avoid  using  computers  because  they  lacked  confidence,  felt 
uncomfortable,  and  were  frightened,  threatened  and  intimidated  by  computers.  Sometimes  teachers’  or  faculties’ 
belief  about  technology  and  education  may  determine  their  behavior  towards  technology  use.  They  think  computers 
are  complicated  machines  to  use  and  master.  Also,  some  think  it  is  a temporary  movement  within  current  schooling 
system  rather  than  a useful  trend.  Teachers’  traditional  belief  and  experience  with  schooling  inhibits  them  from 
taking  instructional  risks  and  implementing  technological  innovations  in  the  classroom  (Sudzina,  1993). 
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The  literature  concentrates  on  three  major  personal  variables  or  factors;  Anxiety  about  technology, 
teachers*  or  faculties*  personalities  and  attitudes  towards  the  technology  integration.  The  major  indicator  of 
computer  anxiety  is  avoiding  from  or  interacting  with  computers  (Dusic,  1998).  Hardy  (1998)  indicates  in  a study 
investigating  computer  aversion  it  was  found  that  teachers  are  very  hesitant  about  computer  related  tasks,  which 
includes  using  computer  machine  and  related  peripherals  in  teaching,  helping  fellow  teachers  when  they  have 
trouble  with  computers  and  applying  to  a job  requiring  an  initial  computer  training.  Some  reasons  are  brought  for 
computer  anxiety,  such  as  inadequate  planning  and  applications  of  technology-based  educational  change  and 
ineffective  communication  between  instructors  and  administrators  (George,  1996).  Jordan  (1993)  adds  the  following 
three  reasons:  “Teachers,  trained  to  master  the  traditional  tools  and  materials  of  their  profession,  fear  their  lack  of 
expertise  with  computers  will  be  embarrassing  and  undermine  their  classroom  authority,  some  teachers  may  be 
uncomfortable  with  the  ways  that  classroom  roles  and  relationships  between  teacher  and  student  change  when 
computers  are  introduced  into  the  classroom,  teacher  productivity  and  student  success  can  be  monitored  with 
computers  easily,  but  many  teachers  worry  about  accountability  since  the  problem  solving  skills  they  try  to  teach 
may  not  be  measurable  through  assessment  instruments  they  have  been  using**.  Improving  self-efficacy  would  be  a 
useful  method  to  decrease  teachers*  anxiety.  Self-efficacy  is  achieved  through  helping  teachers  use  computers 
effectively,  having  them  observe  other  successful  users,  mentoring  teachers  on  the  educational  technology  and 
creating  anxiety-free  environments  or  situations  (Dusic,  1998). 

Psychologists  classify  people  according  to  their  personalities.  Some  empirical  research  data  shows  that 
there  are  connections  between  educators*  types  of  personalities  and  use  of  educational  technology.  Smith  et  al. 
(1995)  attributes  the  features  of  being  creative,  analytical,  logical  and  imaginative  to  institutive/thinking  types  of 
educators  and  says  they  are  more  open  to  educational  technology  than  sensory  types  of  people  who  are  practical, 
realistic  and  sociable.  On  the  other  hand,  comparing  the  other  personality  traits,  sensory/feeling  types  of  people 
show  very  reluctant  behavior  towards  adapting  technology  in  the  classroom.  In  a similar  study,  the  personality  types 
were  classified  as  follow:  extraversion/introversion,  sensing/intuition,  thinking/feeling,  judging/perceptive  and  it 
was  found  that  those  personality  variables  may  determine  the  amount  of  technology  training  taken,  perceived 
adequacy  of  the  training,  perceived  support  from  management  and  perceived  factors  or  barriers  to  adapt  computers 
in  curriculum  (Knupper,  1989). 

Teacher  attitude 

Teacher  attitude  is  the  most  commonly  used  term  in  the  literature  to  describe  practitioners*  appealing  to 
educational  computing.  For  instance  some  teachers  perceived  that  computers  did  not  provide  a distinct  advantages 
over  traditional  methods  of  teaching  (McCormak,  1995)  some  others  value  them  as  a useful  tool  to  support 
meaningful  learning.  Attitude  is  defined  as  evaluative  disposition  based  on  cognition,  effective  reactions  and 
behavior  intentions  and  determines  future  behavior  as  using  the  computer  as  a professional  tool  and  integrating 
technology  in  the  classroom  (Dusic,  1998).  As  can  be  seen  from  the  definition  it  covers  a very  broad  meaning.  Due 
to  that  it  is  used  interchangeably  with  motivation  and  anxiety.  As  a matter  of  fact,  attitude  scales  are  created  based 
on  other  psychological  states.  For  instance,  Loyd  (1985)  created  a computer  attitude  scales  derived  from  computer 
anxiety,  computer  confidence,  computer  liking  and  computer  usefulness.  Computer  anxiety  is  related  to  fear  of 
computer,  computer  confidence  is  about  self-reliance  to  learn  and  use  computers,  computer  liking  is  enjoyment  from 
working  with  computers  and  computer  usefulness  is  related  to  perceived  effectiveness  of  computer. 

Significant  attitudinal  difference  towards  educational  computing  is  found  between  teachers  who  are 
technology  users  and  those  who  are  non-users  (Galowich,  1999).  The  more  teachers  are  willing  to  use  computers  in 
the  classroom,  the  more  their  attitudes  are  favorable  toward  computers  (Dupagne  & Krendl,  1992).  In  addition  to 
that,  the  teacher  attitude  is  significantly  related  to  computer  literacy  knowledge.  Also,  it  is  expected  that  there  is  a 
connection  between  using  computer  outside  of  work  and  the  attitude  (Galowich,  1999).  Attitude  is  not  a clear-cut 
measure  to  indicate  teachers*  disposition  towards  technology,  such  as  high/good  attitude  and  low/bad  attitude. 
Teachers  having  different  experiences,  varieties  of  supports  and  different  incentives  and  barriers  may  exhibit 
different  attitudes. 
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Abstract 

Within  the  last  20  years,  the  quantity  of  computers  in  American  schools  has  significantly  increased. 
However,  teachers  have  not  been  adequately  using  computers  in  the  classroom.  With  a similar  trend,  the  Turkish 
Ministry  of  Education  has  invested  significant  amount  of  money  to  purchase  computers  for  public  schools  in 
Turkey.  In  this  study,  the  effect  of  an  educational  computing  training  program  on  Turkish  teachers’  attitudes  toward 
computers  and  the  effect  of  computer-integrated  classes  on  Turkish  students’  motivation  will  be  investigated. 

The  subjects  of  this  study  are  21  middle  school  teachers  and  their  school  students  in  Turkey.  There  are  two 
independent  variables.  The  first  independent  variable  is  teacher  training.  The  second  one  is  computer-supported 
instruction  implemented  by  the  trained  teachers. 

Introduction 

This  study  will  investigate  the  use  of  computers  in  Turkish  schools  through  teachers’  attitudes  toward 
computers  and  students'  motivation  to  utilize  computer-integrated  lessons.  However,  since  empirical  studies  about 
computer  usage  in  Turkish  schools  could  not  be  located,  some  data  for  American  schools  will  be  presented  first  to 
indicate  what  might  also  be  true  for  Turkish  schools. 

Within  the  last  20  years,  the  number  of  computers  in  American  schools  has  significantly  increased.  While 
there  were  46,000  school  computers  in  1980  (Adams,  1985),  the  Office  of  Technology  and  Assessment  (1995) 
reported  that  this  number  had  increased  to  400,000  by  the  end  of  the  1 980s.  As  of  1 994,  schools  nationwide  owned 
an  estimated  5.5  million  computers  (Mehlinger,  1996).  Even  though  the  number  of  school  computers  increased 
sharply,  the  percentage  of  computer  use  by  teachers  for  instructional  has  remained  low.  The  Office  of  Technology 
Assessment  (1995)  reported  that,  "a  substantial  number  of  teachers  indicated  little  or  no  use  of  computers  for 
instruction."  Based  on  a national  survey,  Becker  (1991)  reported  that  only  1 1%  of  secondary  mathematics,  science, 
and  English  teachers  utilized  computers  in  the  classroom  "throughout  the  year"  or  "intensively,  but  only  for  certain 
units." 

Another  issue  in  educational  computing  is  that  computers  have  not  been  employed  in  an  appropriate 
manner  in  the  classroom.  There  are  large  qualitative  gaps  among  teachers  in  using  computers  for  instruction. 
Sheingold  and  Hadley  (1990)  conducted  a research  study  on  who  were  exemplary  computer  using  teachers  and 
noted  what  made  them  the  exemplary.  It  was  found  that  exemplary  teachers  utilized  the  computer  as  a multipurpose 
tool.  On  average,  95%  of  the  exemplary  teachers  used  the  computer  as  a word  processing  tool,  89%  as  an 
instructional  tool,  87%  as  an  analytic  and  information  tool,  84%  as  a programming  tool,  and  81%  as  a 
game/simulation  tool.  On  the  other  hand,  a 1989  national  survey  on  how  computers  are  usually  used  in  American 
schools  revealed  that  only  1%  of  computer-using  math  teachers  applied  spreadsheets  more  than  five  times  during  a 
year,  while  just  11%  of  computer-using  English  teachers  said  they  used  spell  checkers  regularly  (Becker,  1991, 
1994).  This  was  a sharp  distinction  when  compared  to  56%  and  61%,  respectively,  between  exemplary  computer 
using  teachers  and  typical  computer  using  teachers  (Sheingold  and  Hadley,  1 990). 

Possible  reasons  for  these  discrepancies  may  be  that  teachers  do  not  have  the  knowledge  and  skills  about 
how  to  adapt  computers  to  the  curriculum  or  they  have  not  received  adequate  teacher  training  on  educational 
computing.  The  American  Association  of  Colleges  for  Teacher  Education  (1987)  conducted  a survey  among  new 
teachers.  This  survey  revealed  that  only  20%  of  the  teachers  entering  the  profession  perceived  themselves  as 
prepared  to  teach  with  computers.  In  another  study,  Wiley  (1992)  selected  23 1 teachers  to  survey  who  would 
participate  in  a technology  staff  development  program.  He  found  that  though  these  teachers  had  positive  attitudes 
toward  computers,  they  had  insufficient  knowledge  about  computers  and  strategies  to  effectively  integrate 
computers  into  the  curriculum.  Okinaka  (1992)  concluded  that  lack  of  knowledge  or  understanding  on  how 
computers  can  be  effectively  used  in  the  classroom  is  the  most  significant  factor  slowing  the  computer  adoption 
process  by  teachers. 
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The  puq30se  of  this  study  is  to  understand  how  Turkish  teachers  respond  to  an  instructional  computing 
training  program  design  based  on  data  from  research  conducted  in  the  United  States,  and  how  Turkish  students 
respond  to  computer  integrated  lessons  created  by  trained  teachers.  More  specifically,  this  study  is  to  investigate 
whether  computer  trained  Turkish  teachers  improve  their  attitudes  toward  utilizing  computers  in  regular  course 
instruction.  In  addition,  this  study  will  investigate  whether  Turkish  students'  motivation  toward  lessons,  i.e. 
mathematics,  science,  and  Turkish,  could  be  improved  through  computer-integrated  classes  designed  and 
implemented  by  trained  teachers. 

Literature  Review 

Kay  (1993)  constructed  and  applied  a 7 -point  Likert -type  computer  attitude  measure  based  on  four 
constructs.  These  constructs  were  affective  attitude,  cognitive  attitude,  behavior,  and  perceived  control  over 
computers.  She  found  that  there  were  significant  relationships  between  affective  attitude  toward  computers  and 
computer  behaviors,  as  well  as  cognitive  attitude  toward  computers  and  computer  behaviors.  In  another  study. 
Fencer  et  al.  (1992)  surveyed  230  psychology  undergraduate  students  to  predict  computer-related  behavior  based  on 
computer-related  attitude.  Regression  analysis  shown  that  1 7 computer-related  behaviors  could  be  predicted  from 
the  subjects'  underlying  attitudes. 

For  validating  a scale  to  assess  changes  in  teachers’  attitudes  toward  computers,  Kluever  et  al.  (1994) 
surveyed  265  teachers  participating  in  a training  program.  Significant  difference  was  found  between  the  students' 
attitude  toward  computers  pretest  scores  before  the  training  and  posttest  scores  after  the  training.  Yildirim  (2000) 
investigated  "the  changes  in  attitudes  of  pre -service  and  in-service  teachers  due  to  participation  in  an  educational 
computing  class  and  the  factors  that  contributed  to  the  teachers'  use  of  computers."  It  was  found  that  after  the  class 
teachers'  computer  anxiety  was  significantly  decreased  and  their  computer  liking  and  computer  confidence  were 
significantly  increased. 

Investigating  the  relationship  among  computer  achievement,  attitude  toward  computers,  and  environmental 
variables  - such  as  computer  access,  computer  help,  and  computer  requirements  - Liu  & Johnson  (1998)  conducted 
research  involving  138  female  and  70  male  teacher  education  students  enrolled  in  a required  basic  computer 
technology  course.  The  regression  analysis  showed  that  there  was  a significant  motivational  impact  on  students' 
learning  performance. 

A study  was  conducted  by  Perez  and  White  (1985)  to  "explore  motivational  and  educational  differences 
between  microcomputer  activities  and  classical  classroom  activities."  In  the  study,  it  was  concluded  that  "a 
computer  learning  environment  introduces  and  increases  usage  of  varied  motivational  and  educational  factors  which 
have  the  potential  to  improve  learning  as  well  as  academic  interests"  (p.42). 

Seymour  and  Sullivan  (1984)  surveyed  139  fifth  and  sixth  grade  students  in  six  classrooms  to  investigate 
the  relationship  among  continuing  motivation  of  instructional  medium,  task  difficulty,  and  sex  of  subjects.  The  two 
media  that  were  used  as  instructional  interventions  were  computer  and  paper/pencil  with  two  difficulty  levels:  hard 
and  easy.  Continuing  motivation  was  defined  as  "students’  choice  of  instructional  medium  for  a second  learning  task 
after  they  had  completed  an  initial  task  either  on  a microcomputer  or  paper/pencil  form."  Factorial  ANOVA  analysis 
detected  that  there  was  a significant  motivational  difference  between  the  two  groups  in  terms  of  preferring  one  of  the 
two  learning  mediums  for  a possible  second  learning  task. 

Method 

Participants 

The  participants  of  this  study  are  middle  school  teachers  and  their  students  of  a private  school  in  Turkey 
that  offers  kindergarten  through  eighth  grade.  The  number  of  middle  school  teachers  participating  in  this  study  is  2 1 
(ten  females  and  eleven  males).  Their  distribution  according  to  the  subjects  they  teach  and  gender  is  as  follow:  four 
mathematics  (two  females  and  two  males),  two  science  (two  males),  four  Turkish  (two  females  and  two  males), 
three  social  science  (one  female  and  two  males),  and  eight  English  (five  females  and  three  males).  Their  teaching 
experience  ranges  from  two  to  ten  years  with  an  average  of  4.5  years. 

Independent  Variables 


471 


Since  there  are  two  interventions  in  this  study,  there  are  two  independent  variables.  The  first  independent 
variable  is  teacher  training.  The  second  is  computer-supported  instruction  implemented  by  the  trained  teachers  in 
the  actual  classroom  environment. 

The  first  independent  variable,  teacher  training,  had  two  levels:  (a)  no  teacher  training  and  (b)  teacher 
training.  The  training  was  designed  to  teach  in-service  Turkish  teachers  educational  computing  competencies  rated 
as  important  or  very  important  by  their  American  counterparts.  During  the  training,  Turkish  teachers  were  provided 
with  exposure  to,  knowledge  about,  and  experience  with  educational  computing. 

Initially,  teacher  computer  competencies  were  identified.  The  first  set  of  competencies  was  identified  by 
Scheffler  (1995),  who  found  127  teacher  computer  competencies  through  literature  review  and  interview  with 
computer  representatives.  Using  the  Delphi  technique  and  survey  method,  Scheffler  (1995)  reduced  127 
competencies  to  67. 

In  another  study  used  to  discover  teacher  computer  competencies,  Berg  et  al.  (1998)  utilized  two 
evaluation  phases  to  identify  exemplary  technology  uses  by  teachers  and  students  in  elementary  classrooms  in 
southwestern  Ohio.  39  distinct  exemplary  classroom  applications  of  computers  were  found,  1 1 of  which  were 
specifically  for  teachers. 

The  specific  computer  competencies  taught  in  the  training  were  derived  from  those  two  lists.  From  the  first 
list,  the  items  that  were  rated  4 or  5 Omportant  and  very  important)  and  related  directly  to  teaching  and  learning 
activities  performed  by  classroom  teachers  were  selected.  The  ones  that  were  not  directly  related  to  computer  skills 
teachers  may  require,  not  directly  related  to  teaching  and  learning  in  the  classroom,  and  the  ones  that  were  similar  in 
nature  were  eliminated.  Some  eliminated  competencies  were  "demonstrate  to  students  and  other  classroom  teachers 
the  computer  as  a beneficial  tool  that  increases  efficiency  and  productivity,"  "demonstrate  skills  in  using  a computer 
keyboard,"  "demonstrate  appropriate  use  of  computer  technology  for  basic  skill  instruction,"  "use  modem  for 
communication  between  computers,"  "define  elements  of  a local  educational  agency  technology  plan,"  and  "plan 
methods  to  integrate  computer  awareness  and  literacy  into  the  existing  curriculum."  This  elimination  resulted  in  26 
usable  competencies. 

From  the  second  list  created  by  Berg  et  al.  (1998),  the  competencies  specific  for  teachers  and  rated  5 or  6 
(moderately  important  and  highly  important)  were  selected.  This  selection  resulted  in  5 competencies.  The  total 
competencies  grew  to  31 

The  second  independent  variable,  teachers'  implementation  of  a computer-supported  class  in  an  actual 
classroom,  environment,  has  two  levels:  (a)  implementing  a computer-supported  class  in  the  classroom  and  (b) 
implementing  a non-computer-supported  (classical)  class  in  the  classroom.  After  the  training,  teachers  will  be 
required  to  design  and  implement  a sample  course  in  an  actual  classroom  setting  using  the  skills,  knowledge,  and 
competencies  acquired  through  the  training.  This  second  intervention  will  be  used  to  measure  to  what  extent 
teachers  who  had  the  educational  computer  training  will  increase  their  students'  motivation  towards  lessons. 

Dependent  Measures 

There  will  be  two  dependent  measures: 

1 . Teachers'  attitude  towards  computers  as  measured  by  the  Computer  Attitude  Scale. 

2.  Students'  motivation  towards  courses  by  the  Course  Interest  Survey. 

Attitude  is  defined  as  "an  evaluative  disposition  based  upon  cognition,  effective  reactions,  behavior 
intentions,  and  past  behaviors  which  can  influence  future  cognitions,  effective  responses,  intentions,  and  behaviors" 
(Dusic,  1998).  In  this  study,  the  Computer  Attitude  Scale  (CAS)  (Loyd  & Loyd,  1985)  will  be  employed  to  measure 
teachers'  attitudinal  change  towards  computers  after  the  intervention. 

The  CAS  has  40  Likert -type  items  presenting  statements  of  attitudes  towards  computers  and  the  use  of 
computers.  The  items  are  divided  into  four  categories  each  of  which  represents  one  subscale  of  the  CAS:  (a)  anxiety 
or  fear  of  computers  that  represents  the  Computer  Anxiety  (CA)  subscale,  (b)  confidence  in  ability  to  use  or  learn 
about  computers  that  represents  the  Computer  Confidence  (CC)  subscale,  (c)  liking  computers  or  enjoying  working 
with  computers  which  is  the  Computer  Liking  (CL)  subscale,  and  (d)  perceived  usefulness  of  computers  in  present 
or  future  work  representing  the  Computer  Usefulness  (CU)  subscale.  Each  subscale  has  ten  items  and  respondents 
rate  items  by  indicating  to  what  extent  they  agree  or  disagree  with  the  expressions  in  each  item  (from  strongly 
disagree  to  strongly  agree  with  four  choices) 

The  estimated  total  alpha  reliability  coefficient  of  the  CAS  is  0.95  with  the  following  coefficients  for  the 
subscales:  0.90  for  Computer  Anxiety,  0.89  for  Computer  Confidence,  0.89  for  Computer  Liking,  and  0.82  for 
Computer  Usefulness.  The  CAS  is  a reliable  and  valid  instrument  to  assess  teacher  attitudes  toward  computers 
(Loyd  & Loyd,  1985). 
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Driscoll  (1993)  defined  learning  motivation  for  a student  as  "deciding  to  engage  in  a learning  task  and 
persisting  in  that  task."  In  this  study,  Keller's  (1995)  Course  Interest  Survey  (CIS)  was  used  to  measure  students’ 
motivation  toward  computer-supported  and  non-computer-supported  lessons.  The  CIS  measured  students' 
motivation  to  learn  in  a particular  course.  It  had  34  items  divided  into  four  categories:  (A)  attention,  (R)  relevance, 
(C)  confidence,  and  (S)  satisfaction.  The  items  in  the  attention  category  measure  whether  the  interest  of  learners 
was  captured  and  their  curiosity  to  learn  stimulated  by  the  lesson.  The  relevance  items  inquire  as  to  whether  the 
personal  needs/goals  of  the  learner  were  met  to  affect  a positive  attitude.  Items  related  to  confidence  evaluate  the 
perception  of  learners  about  whether  they  will  be  able  to  succeed  and  control  their  success.  Finally,  the  satisfaction 
items  measure  whether  students’  accomplishments  in  the  classroom  were  reinforced. 

Cronbach’s  alpha  coefficient  for  the  total  survey  was  found  to  be  0.95.  The  subscales'  coefficient  values 
were:  0.84  for  attention,  0.84  for  relevance,  0.81  for  confidence,  and  0.88  for  satisfaction.  Additionally,  it  was 
found  that  there  were  significant  correlations  between  the  CIS  results  and  course  grades  (Keller,  1995).  They  show 
that  CIS  was  a reliable  and  valid  tool  to  measure  students'  motivation  in  a specific  classroom  situation. 

Both  instruments,  the  CAS  and  CIS,  were  originally  written  in  English  and  the  English  versions  were 
validated.  Because  the  teachers’  and  students’  native  language  in  this  study  is  Turkish,  the  researcher  will  translate 
the  English  versions  of  the  surveys  into  Turkish.  The  translated  surveys  will  be  checked  and  corrected  by  two 
language  experts.  The  language  experts  are  studying  Turkish  linguistics  and  literature  in  the  United  States,  and  are 
fluent  in  both  American  English  and  Turkish.  No  data  is  available  about  the  reliability  of  the  Turkish  versions  of  the 
surveys.  They  will  be  calculated  after  the  surveys  are  administered. 

Procedure 

Using  a stratified  random  sampling  method,  I assigned  21  middle  school  teachers  to  either  a control  group 
or  an  experimental  group.  First,  all  21  teachers  were  stratified  according  to  their  subject  matters:  mathematics 
(four),  science  (two),  social  science  (four),  Turkish  (three),  and  English  (eight).  Second,  half  of  the  teachers  within 
each  stratified  group  were  randomly  assigned  to  either  a training  group  or  a control  group.  However,  some  teachers 
who  were  randomly  assigned  to  the  training  group  indicated  that  they  would  not  be  available  for  the  training.  They 
were  then  replaced  with  teachers  in  the  control  group.  This  assignment  resulted  in  1 1 teachers  in  the  training  group 
and  10  teachers  in  the  control  group.  All  21  teachers  will  complete  the  Teacher  Consent  Form  and  the  CAS  before 
training  is  administered. 

The  training  will  take  place  at  the  school  site.  It  will  be  given  by  the  researcher  in  Turkish  and  take 
approximately  30  hours.  The  training  will  be  given  in  a classroom  lecture  format  to  provide  teachers  with  verbal 
information  on  how  computers  can  be  integrated  into  curriculum.  Besides  the  lectures,  computer  lab  sessions  will 
be  arranged  to  provide  teachers  with  hands-on  computer  skills  and  competencies.  From  the  instructional  flow 
diagram,  it  is  predicted  that  the  lab  section  will  take  approximately  15  hours  and  the  class  section  20  hours. 

After  completing  the  training,  teachers  will  design  a sample  computer-supported  class  based  on  the 
knowledge  and  skills  they  gained  through  training  and  implement  that  sample  class  in  an  actual  classroom  setting. 
Before  implementation,  each  teacher  will  have  students  complete  the  Student  Consent  Form  and  the  CIS.  After  the 
implementation  of  the  computer-integrated  class,  students  will  again  complete  the  CIS.  Meanwhile,  when  teachers 
in  the  control  group  teach  the  same  topics  in  the  same  lessons,  such  as  mathematics,  science,  and  Turkish  without 
computer  support,  they  will  have  their  students  complete  the  Consent  form  as  well  as  the  CIS.  Finally,  all  21 
teachers  will  fill  out  the  CAS. 

Because  the  same  students  and  the  same  teachers  will  answer  the  pretest  as  well  as  the  posttest,  pretest 
sensitization  is  an  issue.  Pretest  sensitization  refers  to  "improved  score  on  a posttest  resulting  from  subjects  having 
taken  a pretest.  In  other  words,  taking  a pretest  may  improve  performance  on  a posttest,  regardless  of  whether  there 
is  any  treatment  or  instruction  in  between"  (Gay,  1996).  To  avoid  the  pretest  sensitization,  the  pretest  will  be 
administered  at  the  beginning  of  the  study.  This  will  make  approximately  50-day  gap  between  pretest  and  posttest. 
This  time  gap  may  minimize  the  pretest  sensitization  (Gay,  1996). 


Hypotheses 

Derived  from  the  four  research  questions  in  the  Introduction  section,  two  hypotheses  are  provided.  Each 
hypothesis  was  based  on  the  information  in  the  literature  review  section. 

Hypothesis  1 : Effect  on  Teacher  Attitude  Towards  Computers 
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That  the  trained  teachers  will  change  their  attitude  towards  computers,  which  will  be  detected  by  the  CAS 
employed  after  the  training.  More  specifically,  that  the  teachers  who  receive  the  educational  computer  training, 
which  aims  to  teach  the  computer  competencies  rated  as  important  or  very  important  by  American  computer  using 
teachers,  will  record  higher  scores  on  the  attitude  questionnaire  than  those  teachers  in  the  control  group  who  receive 
no  training. 

Hypothesis  2:  General  Effect  on  Students’  Motivation 

Students  who  receive  the  computer-supported  classes  will  exhibit  higher  motivation  detectable  by  the  CIS 
form.  In  addition,  students  who  have  computer-supported  classes,  designed  and  implemented  by  the  teachers  who 
received  the  educational  computer  training,  will  exhibit  higher  motivation  towards  the  lessons  than  those  students 
who  receive  the  same  versions  of  the  classes  with  no  computer  support. 


Research  Design  and  Data  Analysis 

For  each  hypothesis,  a different  research  design  and  data  analysis  technique  will  be  employed  to  minimize 
the  statistical  errors. 

Hypothesis  1 : For  the  first  hypothesis,  the  research  design  technique  will  be  a pretest/posttest  control  group 
design.  Because  the  study  deals  with  differences  in  teachers’  computer  attitudinal  scores,  a control  group  will  be 
used  to  better  understand  whether  the  motivational  difference  between  the  control  and  experimental  groups  is  due  to 
the  treatment,  and  not  due  to  other  factors.  In  this  type  of  design,  there  are  two  groups  of  subjects,  both  of  which  are 
given  a pretest  of  the  dependent  variable.  Then,  one  group  receives  treatment  and  the  other  group  does  not.  Finally, 
both  groups  take  the  posttest.  The  effectiveness  of  the  treatment  is  determined  by  comparing  the  posttest  scores 
(Gay,  1996). 

In  this  study,  there  are  only  21  teacher  subjects.  This  limited  number  of  subjects  may  bring  some  questions 
about  validity  and  reliability  of  research  due  to  the  violation  of  some  assumptions  in  parametric  tests.  For  the  data 
analysis  of  the  first  design,  the  Wilcoxon-Mann-Whitney  test  will  be  employed  to  measure  the  significance  of  the 
training.  The  Wilcoxon-Mann-Whitney  test  is  a powerful  non-parametric  test  that  is  useful  alternative  to  a t-test 
procedure  when  a researcher  wants  to  avoid  the  negative  effects  of  assumptions  in  parametric  statistics,  such  as  the 
normal  distribution  assumption  and  equal  variance  assumption  (Siegel  & Castellan,  1988). 

Hypothesis  2:  Similar  to  the  previous  method,  the  pretes  t/posttest  control  group  experimental  design 
procedure  will  be  employed  for  the  second  hypothesis.  Because  analysis  of  pretest/posttest  gained  scores  has 
several  disadvantages,  pretest  differences  can  be  better  controlled  by  covariance  (Gay,  1996).  Thus,  ANCOVA 
(analysis  of  covariance)  will  be  employed  for  data  analysis  in  this  study,  and  pretest  scores  will  be  used  as  a 
covariate. 

Because  no  study  was  found  where  the  CAS  and  CIS  were  administered  to  the  control  group  and  the 
experimental  group  at  the  same  time,  no  previous  data  is  available  about  the  mean  scores  and  variances  of  both 
groups,  and  statistical  power  cannot  be  estimated  without  this  information  (Feldt,  1993).  After  collecting  data  and 
calculating  descriptive  statistical  values,  the  statistical  power  will  be  calculated. 
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Abstract 

One  of  the  biggest  challenges  facing  distance  education  programs  is  attrition.  One  potential  way  of 
reducing  attrition  is  to  foster  a sense  of  community  among  students.  Students  who  are  emotionally  and 
intellectually  invested  in  each  other  and  in  their  program  are  more  likely  to  prosper  in  a multi-year 
distance  program.  This  paper  briefly  explores  those  effect  ascribed  to  community  that  are  assumed  to  be 
crucial  for  distance  education  programs  to  succeed,  and  then  focuses  on  a theoretical  framework  knows  as 
the  Psychological  Sense  of  Community  (PSOC).  From  this  basis  the  communication  between  graduate 
students  in  a distanced  education  cohort  is  explored  to  see  how  well  the  PSOC  can  be  applied  to  this 
environment. 

Introduction 

The  advent  of  the  World  Wide  Web  has  brought  with  it  a massive  proliferation  of  distance  education 
options,  many  of  them  based  around  extensive  online  communication.  Unlike  the  correspondence  courses  that 
preceded  them,  this  new  brand  of  distance  education  features  opportunities  for  student-student  and  student-teacher 
interaction.  Many  are  also  built  on  a different  economic  model;  a full  degree  program  may  take  several  years, 
requiring  a long-term  commitment  between  the  student  and  the  offering  institution. 

Modem  pedagogical  models  encourage  the  use  of  group  work,  learning  communities  and  cohorts  to  help 
prepare  students  for  a work  world  in  which  teamwork  plays  an  important  role.  Implemented  as  a distance  education 
option,  this  model  requires  a robust  technology  infrastructure  and  intensive  faculty  involvement — requirements  that 
have  a significant  economic  impact.  A program  that  experiences  high  attrition  (for  example,  starts  with  20  students 
and  ends  up  with  only  3 at  the  end  of  a 3-year  degree  program)  will  quickly  become  a monetary  sinkhole  as  course 
design,  development,  and  faculty  interaction  time  is  subsidized  by  fewer  and  fewer  students. 

With  both  pedagogy  and  attrition  in  mind,  many  scholars  have  begun  to  focus  on  creating  community, 
particularly  “learning  communities.”  Unfortunately,  the  terms  are  often  vaguely  defined.  How  does  one  know  to 
what  degree  community  is  present?  What  positive  attributes  can  we  ascribe  to  such  a community  if  it  does  exist? 

This  paper  briefly  explores  those  effects  ascribed  to  community  believed  to  be  crucial  for  distance 
education  programs  to  succeed,  and  then  focuses  on  a specific  theoretical  framework,  the  Psychological  Sense  of 
Community  (PSOC),  that  seeks  to  define  community.  From  this  basis,  the  communication  between  graduate  students 
in  a distance  education  cohort  is  explored  to  see  how  well  the  PSOC  can  be  applied  to  this  environment. 

Review  of  the  Literature 

Distance  education  and  attrition 

Among  the  challenges  faced  by  those  implementing  a distance  education  program  — poorly  understood 
technology,  high  demands  on  faculty  time,  and  growing  competition  — perhaps  the  most  significant  challenge 
comes  in  the  form  of  attrition.  Reports  of  attrition  rates  at  the  course  level  in  distance  education  vary  wildly  from 
study  to  study  but  are  generally  higher  than  on-campus  courses  (Dille  & Mezack,  1991 ; Kember,  1995). 

Isolation,  “the  state  where  one's  achieved  level  of  social  contact  is  lower  than  one's  desired  level  of  contact” 
(Altman,  1975),  is  one  of  the  most  prevalent  reasons  given  for  dropping  out  of  distance  education  courses  (Morgan 
& Tam,  1999).  Cut  off  from  in-class  as  well  as  serendipitous  encounters  with  their  classmates  and  instructors, 
students  lose  the  sense  that  they  belong  to  something  (Morgan  & Tam,  1999;  Want  & Grimes,  2000). 

Attrition  can  potentially  be  addressed  through  the  formation  of  a learning  community,  specifically  by 
engendering  in  students  feelings  of  belonging.  Of  course,  feeling  isolated  is  not  the  only  factor  influencing  student 
drop-out  (Brown,  1996;  Nippert,  2000;  Want  & Grimes,  2000),  but  belonging  to  a community  seems  to  serve  both 
the  student  and  the  institution  by  reducing  such  feelings  and  hence  attrition  (Haythomthwaite,  Kazmer,  Robins,  & 
Shoemaker,  2000;  McCarthy,  Pretty,  & Catano,  1990;  Morgan  & Tam,  1999;  Palloff  & Pratt,  1999).  Encouraging  a 
sense  of  community,  then,  offers  us  a way  to  address  the  issue  of  retention. 
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Community  and  attrition 


Lack  of  community  is  linked  with  two  student  attributes  associated  with  attrition:  student  burnout 
(McCarthy  et  al.,  1990)  and  feelings  of  isolation  (Haythomthwaite  et  al.,  2000).  McCarthy  et  al.  (1990)  note  that 
undergraduates  who  experience  a strong  psychological  sense  of  community  in  their  living  environment  reported 
lower  burnout  on  the  Meier  Burnout  Assessment  and  the  Maslach  Burnout  Inventory,  compared  to  students  who  did 
not.  They  suggest  that  programs  and  interventions  to  prevent  or  decrease  burnout  should  focus  not  simply  on 
individual  students  (such  as  improving  their  coping  skills),  but  the  college  community  itself.  Haythomthwaite  et  al. 
(2000)  found  that  students  who  do  not  make  connections  with  their  classmates  at  a distance  “report  feeling  more 
isolated  and  stressed  than  those  who  are  more  active;  exchanges  with  other  students  become  vital  for  validating  their 
experiences  and  for  overcoming  isolation”  (p.  1). 

Learning  communities  and  virtual  communities 

The  notion  of  a community  of  learners  has  been  gaining  currency  in  various  educational  fields  for  a number 
of  years.  Similar  in  many  respects  to  Lave  and  Wenger’s  (1991)  community  of  practice,  a community  of  learners, 
or  learning  community,  can  take  a number  of  forms  and  can  fulfill  a variety  of  needs.  This  term  has  become  a 
buzzword  applied  to  almost  any  educational  or  workplace  group,  especially  with  respect  to  online  or  virtual 
communities.  While  there  is  a wide  variety  of  research  focusing  on  such  learning  communities,  many  (Baker  & 
Moss,  1996;  Bielaczyc  & Collins,  1999;  Boehmer  & Waugh,  1997;  Brower  & Dettinger,  1998;  Cross,  1998;  Hill, 
1985;  Lawrence,  2000;  Pike,  1997;  Shapiro  & Levine,  1999;  Wilson  & Ryder,  1996)  seem  to  accept  the  term  at  face 
value  without  attempting  to  rigorously  define  the  characteristics  that  differentiate  a true  learning  community  from 
any  other  class.  Others  describe  characteristics  of  community,  but  do  not  offer  a way  to  evaluate  presence  or  level  of 
community  (Bauman,  1997;  Bruckman,  1996;  Cox,  1997;  Donath,  1999;  Everhart,  1999;  Kim,  2000;  Kollock  & 
Smith,  1999;  Preece,  2000;  Schwier,  2001;  Selznik,  1996;  Wellman  & Guila,  1999a)  . 

Linked  to  this  question  of  how  to  evaluate  the  existence  and  effectiveness  of  community  are  the  ways  in 
which  researchers  view  community.  The  preponderant  method  is  to  establish  a list  of  characteristics  indicating  the 
presence  of  community.  Using  these  characteristics  as  a lens,  the  researcher  then  looks  for  indicators  of  those 
characteristics  in  the  available  data,  and  determines  whether  (or  to  what  extent)  the  group  is  a community  (see,  e.g., 
Kim,  2000;  Schwier,  2001;  Selznik,  1996;  Wenger,  1998). 

Another  way  to  define  community  is  through  its  structure.  Social  network  theory  uses  relationships  among 
people  (as  defined  by  “weak”  and  “strong”  ties  and  relations)  to  determine  a person’s  social  network  (Wellman, 

1979,  1999;  Wellman,  Carrington,  & Hall,  1988;  Wellman  & Guila,  1999a,  1999b).  To  define  virtual  community  by 
interaction,  a researcher  determines  what  kinds  of  exchanges  are  occurring  among  classmates,  how  regular  and 
frequent  they  are,  what  the  tone  and  level  of  intimacy  are,  and  what  the  potential  topics  are  (e.g.,  “work-related”  or 
“friendship” -based).  These  data  are  used  to  create  maps  of  how  the  students  interact,  allowing  the  researchers  to 
derive  models  of  how  information  and  other  resources  flow  through  the  group. 

The  methods  and  models  referenced  above  focus  on  determining  whether  or  not  a group  of  people  exhibits 
externally  defined  indicators  of  community.  Puddifoot  (1996)  calls  into  question  such  quantification:  “It  is  not 
apparent  whether  community  identity  can  be  established  in  any  empirically  objective  way,  or  indeed  whether  this 
should  even  be  the  goal”  (Puddifoot,  1996,  p.  328).  There  is  a construct,  well  known  in  the  community  psychology 
literature,  of  “psychological  sense  of  community”  (PSOC)  (Glynn,  1981;  McMillan,  1976;  McMillan  & Chavis; 
1986,  McMillan,  1996).  Simply  put,  this  is  the  individual’s  perception  of  whether  or  not  she  belongs  to  a 
community,  and  it  is  this  construct  that  forms  the  basis  of  our  investigation. 

McMillan  & Chavis’  Psychological  sense  of  community 

In  their  seminal  1986  article,  McMillan  and  Chavis  sought  to  describe  a sense  of  community  and  offered 
four  criteria  necessary  for  any  acceptable  definition.  Any  definition,  they  said,  must  be  explicit  and  clear.  It  must  be 
concrete,  with  its  parts  identifiable.  Finally,  say  McMillan  and  Chavis,  it  must  represent  the  “warmth  and  intimacy 
implicit  in  the  term;”  (p.  9)  and  provide  a dynamic  description  of  the  development  and  maintenance  of  the 
experience.  Their  model  has  formed  the  basis  of  much  of  the  work  done  in  the  field  of  community  psychology,  but 
has  not  had  a noticeable  impact  in  the  world  of  education. 

The  PSOC,  as  its  name  implies,  is  based  in  the  idea  that  many  of  the  benefits  ascribed  to  community  come 
from  an  internal  sense  of  community,  irrespective  of  any  externally -observable  characteristics  about  the  group  in 
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question.  While  designed  with  place-based  neighborhoods  in  mind,  McMillan  and  Chavis  assert  that  their  definition 
of  sense  of  community  will  apply  equally  to  both  place-based  and  non-place-based  communities. 

The  model  entails  four  elements  (Membership,  Influence,  Needs,  and  Emotional  Connection),  each  of 
which  has  a series  of  sub-characteristics.  These  elements,  shortened  to  the  acronym  MINE  throughout  the  rest  of  this 
paper,  are  described  in  detail  below. 

Membership 

Membership  deals  not  only  with  who  is  in  or  not  in  a community,  but  with  the  sense  of  safety  that 
accompanies  such  delineation.  The  ability  to  identify  another  member  of  a community  allows  one  to  better 
determine  how  to  spend  resources  and  with  whom  to  feel  comfortable.  Integral  to  the  idea  of  membership  is  the 
concept  of  boundaries.  It  is  perhaps  just  as  important  to  know  who  is  not  in  the  community  as  it  is  to  know  who  the 
members  are. 

Boundaries  can  be  created  and  enforced  in  many  ways,  including  a group’s  use  of  language,  styles  of  dress, 
and  rituals.  Gang  members,  for  example,  are  able  to  tell  at  a glance  if  they  are  facing  a friend  or  foe  by  looking  at  the 
person’s  colors.  In  this  case,  even  more  than  most,  the  creation  and  maintenance  of  boundaries,  as  demonstrated  by 
dress  or  rites,  is  a protection  against  external  threat.  Similarly,  a common  symbol  system  aids  in  creating  and 
maintaining  group  boundaries.  These  symbols  combined  create  a social  convention  that  again  delineates  the  “us” 
versus  “them.”  Symbols  may  operate  at  the  group  level  (black  leaders  using  Black  Power  and  clenched  fist),  the 
neighborhood  level  (name,  landmark,  logo,  architectural  style),  or  national  level  (holidays,  flag,  language,  currency). 

Safety,  especially  emotional  safety,  is  embodied  in  the  idea  of  security  in  one’s  community.  Established 
boundaries  provide  structure  and  security,  protecting  group  intimacy.  In  many  cases,  such  support  is  emotional  in 
nature,  but  in  the  case  of  gangs  or  warring  factions,  the  security  is  physical;  for  collectives  and  cooperatives,  the 
security  can  be  financial. 

The  expectation  that  one  fits  the  group  and  is  accepted  by  the  group  is  a sense  of  belonging  and 
identification.  The  member  feels  he  has  a place  there,  and  is  willing  to  make  sacrifices  for  the  group.  The  member 
identifies  with  the  group,  which  is  reflected  in  reciprocal  statements  such  as  “This  is  my  group”  or  “I  am  part  of  this 
community.” 

Personal  investment  also  contributes  to  an  individual’s  feeling  of  group  membership  and  feelings  of 
belonging.  McMillan  (1976)  asserted  that  working  for  membership  will  provide  the  feeling  that  one  has  earned  a 
place  in  the  group,  and  that  consequently,  this  personal  investment  will  make  the  membership  more  valuable. 

Influence 

Influence  is  the  second  overarching  element  of  the  psychological  sense  of  community.  Influence  is  bi- 
directional: in  order  to  be  attracted  to  the  group,  an  individual  must  have  the  potential  of  influencing  the  group.  The 
reverse  case  — the  ability  of  the  group  to  influence  its  members — is  crucial  to  maintaining  cohesiveness.  These 
seemingly  opposite  forces  do  appear  to  work  simultaneously,  indeed,  in  concert.  Note  that  influence  often  operates 
independently  of  positions  of  authority. 

An  important  aspect  of  influence  is  the  idea  of  consensual  validation,  which  assumes  that  “people  possess 
an  inherent  need  to  know  that  the  things  they  see,  feel,  and  understand  are  experienced  in  the  same  way  by  others” 
and  people  will  go  to  great  psychological  and  emotional  lengths  to  reassure  themselves  that  they  are  not  crazy 
(McMillan  & Chavis,  1986,  p.  11).  One  cause,  then,  of  group  conformity,  is  the  pressure  on  the  individual  to 
experience  harmony  with  the  group’s  world  view.  Again,  this  pressure  can  move  from  the  individual  into  the  group 
as  well  as  being  imposed  by  the  group  on  the  individual,  so  that  the  group  is  “operating  to  consensually  validate  its 
members  as  well  as  to  create  group  norms”  (McMillan  & Chavis,  1986,  p.  11). 

Integration  and  fulfillment  of  needs 

The  third  element  of  the  psychological  sense  of  community  is  the  integration  and  fulfillment  of  needs,  most 
commonly  encompassed  by  issues  of  reinforcement.  Obviously,  the  individual’s  association  with  the  group  must  be 
rewarding  for  the  members.  In  many  cases,  a reinforcing  element  is  just  the  status  of  being  a member  of  that  group. 
The  benefit  of  being  a member  of  the  “in  crowd”  is  simply  association  with  that  group. 
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Communities  are  also  strengthened  by  group  accomplishments.  Simply  stated,  successes  associated  with 
group  activities  bring  members  closer  together,  McMillan  and  Chavis  assert  that  competence  is  personally  attractive 
and  that  people  will  gravitate  towards  groups  and  other  people  that  offer  the  most  reward. 

A third  way  in  which  need  fulfillment  is  given  direction  is  through  the  concept  of  shared  values.  People 
with  shared  values  come  together  and  find  they  have  similar  goals,  priorities,  and  needs,  and  is  more  easily  able  to 
focus  resources  on  issues  that  speak  to  those  values . This  encourages  the  belief  that,  as  a group,  they  are  better  able 
to  fulfill  their  needs  in  a continual,  mutually-beneficial  way.  In  this  case,  it  is  shared  values  that  act  as  an 
“integrative  force  for  cohesive  communities”  (p.  13).  Note,  however,  that  a group  with  a strong  sense  of  community 
in  which  members  do  not  necessarily  have  identical  goals  and  priorities  will  still  work  together  to  fulfill  all 
members’  needs. 

Shared  emotional  connection 

The  final  component  of  the  psychological  sense  of  community  is  a shared  emotional  connection,  which  is 
based,  in  part,  on  a shared  history.  McMillan  and  Chavis  point  out  that  it  is  not  necessary  that  all  group  members 
have  participated  in  the  history  in  order  to  share  it,  but  they  must  identify  with  it. 

To  share  a connection  with  others,  of  course,  presupposes  interaction  with  them.  The  “contact  hypothesis” 
asserts  that  the  more  people  interact,  the  more  they  are  likely  to  become  close.  The  quality  of  interaction  is  also 
important,  in  that  positive  experiences  create  greater  bonds;  as  was  noted  earlier,  group  success  creates  cohesion. 

Sharing  emotional  events  is  crucial  in  creating  a sense  of  connection.  The  “shared  valent  event  hypothesis” 
states  that  the  more  important  the  shared  event  is  to  the  people  involved,  the  greater  the  community  cohesion. 

Groups  who  survive  a crisis  together  feel  an  increased  bond  (e.g.,  war  veterans).  Closure  to  events  is  an  important 
part  of  community  unity;  if  the  group’s  tasks  are  unresolved  and  interaction  is  ambiguous,  the  cohesbn  will  suffer. 

Investment  in  the  community  “determines  the  importance  to  the  member  of  the  community’s  history  and 
current  status”  (McMillan  & Chavis,  1986,  p.  14).  People  who  expend  time  and  energy  on  projects  will  feel  more 
emotionally  involved  in  their  outcome. 

Finally,  various  types  of  intimacy  affect  the  shared  emotional  connection.  Intimacy  is  a type  of  investment: 
the  emotional  risk  one  takes  with  other  members  of  the  group  can  affect  (and  be  affected  by)  one’s  sense  of 
community. 

Applying  the  PSOC  framework 

Working  from  this  theoretical  base,  Chavis,  Hogge,  McMillan  & Wandersman  (1986)  took  these  four  major 
categories  and  used  them  to  derive  an  instrument  to  measure  PSOC  in  an  individual,  the  Sense  of  Community  Index 
(SCI)..  The  SCI,  and  hence  the  PSOC,  have  been  used  to  describe  both  place-based  (geographical,  e.g.  a 
neighborhood)  and  non -place-based  (relational,  e.g.,  an  ethnic  group)  communities.  McCarthy  (1990)  and  Pretty 
(1990)  validated  the  SCI  for  the  undergraduate  university  community.  Pretty  has  used  the  instrument  in  both  the 
corporate  world  (1991)  and  with  adolescents  (1994). 

In  1996,  Sonn  (1996)  developed  an  “open  response  format  interview  schedule”  to  asses  the  four  elements 
of  the  sense  of  community  framework,  then  conducted  semi -structured  interviews  (lasting  25-50  minutes)  with  23 
participants  (p  420).  He  then  used  the  elements  of  the  psychological  sense  of  community  to  frame  themes  emerging 
from  the  interview  data,  providing  evidence  that  the  PSOC  model  has  construct  validity  in  this  environment. 

Despite  its  preponderant  use  in  the  field  of  community  psychology,  only  a single  reference  to  this  model  in 
education  was  found:  dissertation  work  by  Chao  (1999).  In  the  next  section,  use  of  the  theory  underlying  the  PSOC 
to  examine  communications  among  members  of  an  online  distance  education  program  is  discussed. 

Methods 

Context 


The  research  proposed  focuses  on  a particular  program,  a master’s  degree  in  educational  technology  offered 
by  a large,  Midwestern  university.  The  Education  Technology  Online  Master’s  (ETOM)  is  a three-year,  cohort- 
based  program  designed  to  give  working  professionals  an  opportunity  to  earn  an  M.S.  in  educational  technology. 

The  overriding  concerns  center  on  (a)  the  ability  to  maintain  high  standards  of  academic  quality  at  a distance  in  a 
highly  accessible  format,  and  (b)  economic  feasibility  and  sustainability  of  the  program. 

The  ETOM  seeks  to  build  on  the  success  and  structure  of  the  on-campus  master’s  program,  while  taking 
into  account  the  needs  of  full-time  employees.  The  coursework  is  essentially  the  same  as  that  on  campus;  project- 
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based  with  a great  deal  of  group  work,  some  individual  development  projects,  a substantial  amount  of  writing,  and  a 
mastery -based  assessment  process.  These  characteristics  are  considered  essential  by  the  faculty  of  the  department, 
and  frame  the  challenges  of  a distance  program  along  both  pedagogical  and  economic  dimensions. 

The  ETOM  is  set  up  as  a cohort;  each  group  of  students  will  travel  through  the  three-year  program  together. 
While  not  all  students  will  take  all  the  same  courses,  the  required  courses  will  be  taken  with  the  same  group  of 
people  in  an  effort  to  instill  a sense  of  community  and  trust. 

Participants 

Participants  for  this  study  are  distance  students  drawn  from  the  online  master’s  program  previously 
described.  Researchers  received  permission  to  view  postings  and  chat  transcripts  from  15  of  the  16  students  in  the 
cohort. 

All  participants  hold  full-time  jobs,  their  positions  split  roughly  equally  among  K- 1 2,  higher  education,  and 
the  corporate  world.  Geographically,  they  are  spread  across  three  time  zones.  They  range  in  age  from  25  to  55.  Nine 
of  the  seventeen  are  women.  As  individuals  who  chose  to  pursue  an  online  degree  in  educational  technology,  these 
students  all  had  above-average  technical  skills. 

Data  sources  and  collection 

All  data  collected  were  the  products  of  a one-semester  online  course  offered  during  the  Fall  2000  semester. 
Data  were  collected  primarily  from  two  sources.  First  is  a series  of  online  chats  that  took  place  throughout  the 
semester,  held  most  Wednesday  nights  at  8:00  and  lasting  between  1 .5  and  2 hours.  These  weekly  chats  were  the 
only  scheduled  “real-time”  interaction  the  students  had  with  the  instructor,  and  were  designed  to  give  the  students 
access  to  the  professor  to  discuss  issues  surrounding  the  content,  assignments,  and  due  dates. 

Attendance  at  the  chats  (after  an  initial  time-zone  mix-up)  was  excellent  at  the  beginning  of  the  semester 
and  grew  spottier  as  time  went  on.  Roughly  two-thirds  of  the  way  through  the  semester  the  instructor  discontinued 
these  whole -group  chats  because  so  few  people  were  attending.  They  were  reinstated  at  the  request  of  a vocal 
minority,  and  attendance  for  the  rest  of  the  semester  hovered  around  7-8,  or  half  the  students.  In  total,  there  were  1 1 
chat  sessions  comprising  over  1000  separate  entries. 

The  second  source  of  data  was  asynchronous  postings  to  the  whole-group  spaces  of  the  course’s  web-based 
conferencing  system.  These  were  open  throughout  the  semester,  but  the  vast  majority  of  the  almost  200  postings  in 
the  conferences  took  place  in  two  distinct  threads  dealing  with  end-of-project  “lessons  learned”.  All  these  data  were 
collected  from  the  electronic  systems  and  reformatted  for  easier  reading  and  for  import  into  nVivo™. 

Data  analysis 

Data  analysis  began  with  an  examination  of  transcripts  from  both  the  asynchronous  conference  and  the 
real-time  chats.  Each  entry  was  examined  (in  context)  for  communication  beyond  simple  task-oriented  interaction; 
these  were  assigned  a low-level  code  to  denote  the  type  of  utterance.The  unit  of  analysis,  then,  was  not  specifically 
limited  to  a whole  phrase  or  even  sentence;  units  ranged  in  size  from  a short  phrase  to  several  sentences. 

Data  were  coded  until  the  list  of  codes  stabilized  (at  about  29  of  these  low-level  categories).  The  entire  data 
set  was  imported  into  nVivo'*'*^  and  coded  by  the  researchers;  codes  were  often  merged  or  added  to  the  list  as 
necessary.  Finally,  already-coded  data  were  re -checked  to  make  sure  the  definitions  of  codes  hadn’t  drifted  during 
this  process. 

Since  the  concern  of  this  study  is  the  students’  sense  of  community,  all  comments  made  by  the  instructor 
were  ignored  (although  responses  directed  at  the  instructor  were,  of  course,  included).  Also  ignored  were  those 
comments  that  involved  purely  giving  and  receiving  course-related  information  (e.g.,  asking  the  instructor  to  define 
a term),  answering  a simple,  information-oriented  question  (such  as  “What  were  we  supposed  to  read  this  week?”), 
and  in  general,  those  questions  and  the  resulting  exchanges  that  were  so  task-oriented  as  to  give  no  insight  into  the 
community  feelings  codified  in  the  MINE  framework.  The  researchers’  interpretation  of  McMillan  and  Chavis  led  to 
the  conclusion  that  such  purely  functional  transactions  are  of  little  value  when  looking  for  evidence  ofPSOC. 

In  the  end,  the  original  29  codes  had  been  modified  and  reduced  to  a list  of  22  different  types  of  comments 
that  seemed  to  relate,  in  some  way,  to  issues  represented  in  the  sense  of  community  model  driving  the  analysis. 
Armed  with  this  list  of  codes,  the  researchers  returned  once  more  to  the  PSOC  theory.  Guided  by  McMillan  and 
Chavis  (1986),  and  with  ideas  further  refined  from  McMillan  (1996)  and  Chao  (1999),  the  data  were  categorized 
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into  the  four  major  components  of  PSOC  (membership,  influence,  fulfillment  of  needs,  or  shared  emotional 
connection).  The  full  results  of  this  categorization  can  be  seen  in  tables  1 through  4. 

Findings  and  interpretation 

Of  the  original  200  posts  in  the  asynchronous  conference  and  almost  1000  comments  made  in  the  chats 
(together  equating  to  roughly  120  printed  pages),  1 87  utterances  were  coded  into  at  least  one  of  the  22  relevant 
categories.  These  broke  down  into  the  four  MINE  categories  as  indicated  in  each  of  the  following  sections. 

Membership 


Themes  Indicating  Membership 

Number 

Indication  this  is  a safe  space 

2 

Offer  of  help  /information  without  request 

3 

Shared  symbol  system 

1 

Basic  verbal  support 

34 

Humor  of  a personal  nature 

37 

Reference  by  name 

18 

Member  check 

4 

General  question  implying  a request  for  support 

2 

TOTAL 

101 

Table  1:  Themes  indicating  Membership 

Membership  subsumes  ideas  of  boundaries,  language,  and  other  representations  of  a common  symbol 
system,  and  a sense  of  belonging  to  the  group  and  the  emotional  safety  that  comes  from  that  group  identification. 
These  ideas  manifested  themselves  in  four  broad  categories. 

In  the  case  of  this  cohort,  boundaries  were  artificially  created  and  maintained  by  the  enrollment  structures. 
Only  people  enrolled  in  the  program  (and  the  professors  and  staff)  had  any  contact  with  the  group  as  a whole,  so 
there  was  really  no  need  to  either  delineate  or  enforce  boundaries. 

Working  in  an  online  format  makes  certain  demands  on  students  with  respect  to  communication,  but  it  also 
provides  opportunities  for  the  group  to  adopt  certain  shared  symbols.  The  use  of  emoticons  (“smilies,”  especially) 
was  very  common,  and  a few  students  had  their  own  symbols  that  they  attempted  to  share  with  the  group  (further 
discussed  in  the  section  on  influence).  With  the  exception  of  the  construct  “*see*”  to  indicate  online  presence  (“I’ll 
*see*  you  tomorrow”),  non-standard  symbols  were  not  adopted  by  the  entire  group. 

A few  themes  emerged  that  suggested  that  the  students  felt  a certain  degree  of  emotional  and  intellectual 
safety  in  the  group.  One  student  implied  that  she  felt  safe  in  the  community  and  with  respect  to  the  instructor:  “I 
can’t  believe  I’m  admitting  this  to  the  teacher,  but  I feel  much  better  now  that  I’ve  learned  what  I can  ignore”.  One 
reason  perhaps  that  people  feel  emotional  safety  in  their  group  is  that  support  is  available.  We  infer  that  asking  a 
question  implying  support:  (“Is  everyone  surviving?”)  or  giving  basic  verbal  support  (“I’ll  second  that ;-)”) 
demonstrates  the  feeling  is  that  someone  is  available  to  provide  the  asked-for  support.  Thirty-four  instances  of  basic 
verbal  support  were  found.  McMillan  and  Chavis  claim  that  use  of  humor  may  also  be  an  indication  of  emotional 
safety  that  shows  the  speaker  is  confident  in  her  ability  to  connect  with  her  peers  on  an  emotional  level.  There  were 
37  uses  of  humor  in  the  data  set. 

The  feeling  that  one  belongs  and  is  accepted  by  the  group  can  manifest  itself  in  a few  ways.  Riger  (1981) 
cites  social  bonding  indicators  that,  for  co-present  communities,  include  the  ability  to  identify  neighbors  and  their 
children.  In  the  studied  environment  this  can  be  demonstrated  by  individuals’  referring  to  each  other  by  name 
(“Grace — I agree”).  The  use  of  names  aids  in  feelings  of  belonging  in  that  it  creates  intimacy.  Students  referred  to 
each  other  by  name  18  times. 

Overall,  more  than  100  of  the  187  coded  utterances  were  related  to  the  Membership  dimension.  This  makes 
it  by  far  the  most  often  expressed  aspect  of  community. 

Influence 


Themes  Indicating  Influence 

Number 

Voice  minority  opinion 

9 
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Attempt  to  build/enforce  symbol  system 

1 

Ask  for/give  neutral/popular  opinions 

17 

Ask  for  clarification 

2 

TOTAL 

29 

Table  2:  Themes  indicating  Influence 

As  was  mentioned  previously,  influence  is  bi-directional.  Although  individuals  must  have  the  potential  to 
influence  the  group,  the  group’s  cohesiveness  is  contingent  on  its  ability  to  influence  members. 

Although  the  role  of  an  assigned  instructor  is  different  from  that  of  an  emergent  an  instructor  can  be 
considered  an  authority  figure  with  whom  structures  (and  hence  influence  over  the  class)  can  be  negotiated.  The  bi- 
directionality of  such  influence  was  first  illustrated  at  the  face-to-face  orientation.  The  instructor  had  mandated  a 
Friday  night  due  date  for  weekly  assignments,  but  then  noted  that  she  wouldn’t  look  at  them  until  Monday.  The 
students  pointed  out  that  most  of  the  time  they  had  free  to  work  on  the  course  was  on  the  weekend,  and  so  negotiated 
a Sunday  night  due  date.  This  set  the  stage  for  later  debates  about  due  dates  and  assignments,  immediately  showing 
students  that  they  had  some  power  over  the  course  and  hence  should  be  less  hesitant  to  invest  in  the  group. 

While  the  existence  and  use  of  a shared  symbol  system  is  an  indication  of  membership  (via  explicated 
boundaries),  an  attempt  to  create  or  enforce  such  a system  is  a matter  of  influence.  Some  students  used  acronyms 
and  constructs  common  in  online  chat  sessions,  not  all  of  which  were  taken  up  by  the  larger  group.  In  one  case,  the 
student  explicated  her  use  of  actions  between  asterisks  to  describe  asides  (e.g.,  *looks  around  for  help*). This  is  way 
of  expressing  oneself  that  is  standard  and  common  in  many  chat  rooms,  but  it  was  new  to  the  students  with  more 
limited  online  communication  experience  and  was  never  adopted  by  the  rest  of  the  group. 

One  of  the  most  powerful  indicators  of  influence  is  expressing  a minority  opinion.  In  this  case,  the  speaker 
must  take  the  risk  of  “going  against  the  grain”  and  taking  a stand  opposite  that  of  the  group,  or  at  least,  the  opinions 
expressed  thus  far.  The  level  of  risk  perceived  by  the  speaker  is  inherent  in  his  preface  to  the  divergent  opinion:  “If  I 
could  respectfully  disagree  and  restate  I believe  them  to  be  at  least  equal  in  a highly  complex  subject  area.”  We 
found  fewer  than  10  examples  of  students  expressing  a minority  opinion. 

If  expressing  a minority  or  divergent  opinion  is  a strong  indicator  of  sense  of  community;  a related 
indication  of  influence  is  trying  to  elicit  or  giving  neutral  or  popular  opinions:  “Does  everyone  else  agree  with  this 
view?” 

A final  indicator  of  influence  that  emerged  from  the  data  was  when  someone  simply  asked  for  clarification. 
This  shows  a willingness  to  be  influenced  by  another  individual:  at  the  least,  it  indicates  some  level  of  investment  in 
what  the  other  person  is  expressing:  “I’m  intrigued  by  this  question.  Explain  what  you  are  thinking  a little  more.” 

Twenty -nine  of  the  total  utterances  were  related  to  influence.  This  represents  roughly  one-sixth  of  the  total, 
and  most  of  these  were  related  to  neutral  or  popular  opinions. 

Needs 


Themes  Indicating  Integration  and  Fulfillment  of  Needs 

Number 

Ask  for  understanding  or  apologize 

4 

Request  basic  or  immediate  info 

22 

Exhibit  experience/expertise 

9 

Express  thanks 

8 

Be  self-effacing/express  doubts  hoping  for  support 

6 

Express  frustration 

1 

Request  elaboration 

1 

TOTAL 

51 

Table  2:  Themes  indicating  Integration  and  Fulfillment  of  Needs 

According  to  McMillan  and  Chavis,  an  individual  seeks  out  and  maintains  membership  in  a group  because 
the  group  somehow  fulfills  his  needs.  Many  of  the  utterances  in  this  category  are  basically  indicators  of 
reinforcement  and  support,  either  expressing  a need,  or  offering  to  fulfill  another’s  need.  For  example,  when 
someone  asks  for  understanding  or  apologizes  (“Sorry,  I lost  my  connection”)  she  is  assuring  the  group  of  her 
continuing  membership  and  indirectly  requesting  validation  of  the  membership/relationship  in  spite  of  the  error  or 
violation. 

On  a more  concrete  level,  some  needs  are  not  emotional  in  nature  but  are  related  to  resources.  As  a member 
“in  good  standing”  of  a particular  group,  an  individual  has  access  to  the  expertise  and  intellectual  resources  of  the 
group.  It  requires  a certain  level  of  emotional  security  within  the  group  to  ask  for  help  (“I  can’t  get  the  Participants 
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list  to  show  anyone”),  or  to  show  unfamiliarity  and  ask  for  information  that  appears  to  be  common  knowledge:  “I 
hate  to  admit  my  ignorance,  but  what  is  IRC?”. 

The  other  side  of  the  needs  fulfillment  coin  is  the  offer  of  help,  either  solicited  or  unsolicited.  Volunteering 
useful  information  to  the  group — especially  in  the  absence  of  a request  for  such  information — fulfills  a number  of 
roles,  including  exhibiting  experience  or  expertise  that  other  members  will  find  attractive:  “Karl  and  I were  looking 
at  AOL,  and  think  it  will  speed  things  up  a little.”  It’s  also  an  indication  that  the  speaker  feels  it  is  worth  her  time 
and  effort  to  help  out  another  member  of  the  group.  This  could  be  an  indication  that  the  speaker  believes  mutual 
success  is  linked  to  helping  behaviors. 

Finally,  there  is  the  need  for  external  validation  of  suffering:  the  desire  to  complain  to  someone  who  will 
understand.  This  powerful  attribute  of  community  is  used  e^^licitly  in  support  groups  and  similar  counseling 
situations,  but  is  no  less  important  to  a group  that  occasionally  needs  a safe  space  and  sympathetic  ears  for  venting: 
“We  were  dying  in  our  group  chat!*’ 

Fifty-one  of  the  total  coded  utterances  were  related  to  fulfillment  of  needs.  However,  almost  half  of  these 
were  simple  requests  for  basic  information. 


Shared  emotional  connection 


Themes  Indicating  Shared  Emotional  Event/Connection 

Number 

References  to  orientation 

2 

Ask  about  shared  history 

2 

Express  happiness  being  part  of  group 

2 

TOTAL 

6 

Table  4:  Themes  Indicating  Shared  Emotional  Event/Connection 

With  respect  to  shared  emotional  connection,  we  found  three  distinct  themes:  asking  about  shared  history, 
expressing  happiness  at  being  part  of  the  group,  and  making  references  to  the  pre-program  orientation,  the  cohort’s 
first  shared  event,  and  their  only  face-to-face  event  to  date:  “I  think  that  [orientation]  was  real  important  for 
retaining  people,  keeping  people  on  the  program,  so  that  they  felt  some  sort  of  commitment  now  to  this  group.” 

Bonds,  as  they  relate  to  shared  experiences,  seem  to  correspond  to  the  level  of  emotion  involved  rather  than 
whether  or  not  the  experience  was  a negative  one.  Shared  positive  experiences  can  create  strong  bonds,  but  an 
increased  bond  is  also  reported  by  groups  who  survive  a crisis.  In  this  case,  the  students  experienced  a 4-day  face-to- 
face  orientation  that  was  intentionally  structured  to  have  a high-intensity  “boot  camp”  feel.  Students  immediately 
got  down  to  work,  worked,  ate,  and  lodged  together,  and  had  four  very  full  days  of  togetherness  and  things  they  had 
to  accomplish.  The  orientation  fits  into  the  shared  valent  event  hypothesis,  which  states  that  the  more  important  the 
shared  event  is  to  the  people  involved,  the  greater  the  community  cohesion.  The  final  indicator  of  a shared  emotional 
connection  is  a student  expressing  happiness  at  being  part  of  the  group;  there  were  two  utterances  on  this  theme.  As 
can  be  seen  in  table  4,  only  six  of  the  original  187  utterances  spoke  to  the  shared  emotional  connection. 

Discussion 

As  can  be  seen  from  the  four  tables,  not  only  was  there  little  evidence  of  PSOC  in  general  (only  1 87  coded 
utterances  in  over  120  pages  of  text),  but  these  did  not  represent  all  four  areas  of  community  well.  The  data  show  a 
paucity  of  evidence  of  a shared  emotional  connection  (only  six  examples),  and  more  than  half  of  these  coded 
passages  spoke  to  issues  of  membership. 

Perhaps  more  important,  the  majority  of  these  utterances  are  in  forms  that  could  be  easily  construed  as  not 
indicating  community  so  much  as  polite  conversation.  Roughly  half  of  the  coded  data  refers  to  three  items:  humor, 
giving  basic  verbal  support,  and  simple  questions  asking  for  simple  information.  While  these  items  certainly 
represent  their  respective  characteristics  of  community,  they  seem  to  be  in  a form  that  could  be  incorporated  into  the 
pattern  of  conversation,  mitigating  their  power  as  indicators.  It  is  also  the  default  for  people  who  don’t  know  each 
other  well,  who  don’t  feel  safe  with  each  other,  and  who  treat  each  other  as  strangers. 

Given  these  findings — lack  of  coverage  and  the  prevalence  of  coded  items  that  may  be  primarily  indicative 
of  simple  etiquette — we  are  hesitant  to  speak  about  any  substantial  sense  of  community  in  this  group.  The 
discussions  studied  had  a task  focus,  lacking  many  of  the  social  cues  one  would  expect  to  see  in  a group  with  a high 
PSOC.  Why  was  there  so  little  evidence  of  community?  While  there  are  various  limitations  to  the  study  (discussed 
below)  that  could  account  for  the  failure  to  find  an  existing  community,  the  first  and  most  obvious  explanation  is 
that  substantial  evidence  of  community  simply  isn’t  there  to  be  found.  Another  issue  may  be  the  timeline  — one 


972 


483 


semester.  While  some  articles  speak  about  developing  community  within  the  span  of  a single  course  (Eastmond, 
1995;  Hill  & Raven,  2000)  it  simply  may  not  be  long  enough  using  this  type  of  analysis. 

As  indicated,  there  was  also  a noticeable  focus  purely  on  the  tasks  under  discussion.  This  is  partially  an 
outcome  of  the  communications  available  for  analysis.  Given  a limited  time  in  which  students  could  discuss  course 
assignments  and  feedback  with  the  instructor,  it  is  not  surprising  that  little  of  it  would  be  spent  providing  the  social 
cues  and  debates  that  would  indicate  PSOC. 

Limitations 

This  study  uses  ‘‘found  data”  — data  collected  after  the  course  and  analyzed  long  after  its  creation.  While 
there  are  complete  transcripts  of  the  chat  sessions  and  the  contents  of  the  asynchronous  discussion,  the  data  were 
analyzed  without  the  insight  into  changing  tensions  and  attitudes  that  analysis  during  the  course  itself  may  have 
provided.  A future  study  will  take  place  concurrently  with  the  course  itself,  allowing  for  the  exploration  of  issues  as 
they  arise. 

This  study  also  suffers  from  limited  data  access.  While  we  had  full  access  to  the  chats  and  asynchronous 
conferences,  no  interviews  were  completed  and  therefore  it  was  not  possible  to  follow  up  on  interesting  points.  Data 
collected  by  colleagues  speak  to  the  importance  of  the  orientation,  which  the  current  data  set  only  begins  to  suggest. 

Post-course  communication  with  some  students  indicates  the  interactions  that  would  have  been  found  most 
interesting  took  place  between  individuals  via  personal  email.  Not  only  are  researchers  unlikely  to  be  given  access  to 
personal  email,  but  there  is  a strong  sentiment  that  it  is  inappropriate  to  even  ask  for  it.  Students  need  at  least  one 
avenue  of  communication  that  they  feel  is  secure. 

Future  work 

Data  analysis  is  hampered  by  the  lack  of  a rubric  for  determining  what  makes  a particular  utterance  a 
“strong”  one.  While  the  researchers  don’t  rely  heavily  on  counting  utterances  to  gain  insight  into  the  class,  we  do 
feel  we  need  a way  to  indicate  that  saying  someone’s  name  is  not  as  powerful  an  expression  of  sense  of  community 
as  expressing  a minority  opinion.  The  researchers  hope  to  develop  such  a set  of  criteria  and  apply  it  to  data  from 
these  same  students  in  the  upcoming  semesters  of  their  degree  program. 

Studying  these  new  courses  using  data  collection  methods  such  as  interviews  and  surveys  will  help  to  the 
researchers  accomplish  two  goals.  First,  it  may  be  possible  to  refine  and  validate  the  PSOC  model  for  use  in 
qualitative  research  of  an  online  cohort,  perhaps  using  the  SCI  itself  to  validate  our  analyses.  Second,  albeit  may 
eventually  be  possible  to  correlate  specific  course  structures  and  salient  events  to  changes  in  the  PSOC.  This 
information  can  be  used  to  inform  the  second  round  of  design  for  the  online  program  and,  hopefully,  have  an  impact 
on  attrition  and  student  satisfaction. 

Conclusions 

The  psychological  sense  of  community,  as  defined  in  McMillan  and  Chavis’  (1986)  model,  appears  to  be  a 
meaningful,  well-established  and  powerful  tool  for  the  rigorous  investigation  of  community.  The  model  offers  a lens 
through  which  to  explore  how  a group  of  students  perceives  itself.  Application  of  the  sense  of  community  to 
qualitative  methods  is  in  its  infancy,  but  this  model  lends  itself  well  to  this  type  of  investigation.  Future  work  will 
help  determine  if  the  model  is  able  to  offer  insight  into  real-world  applications  of  course  design  and  structures  that 
may,  in  turn,  affect  attrition  rates  and  student  satisfaction. 
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Abstract 

Before  a student  can  become  information  literate,  he  or  she  must  he  taught  information  literacy  skills. 
Higher  education  cannot  produce  information  literate  students  if  it  does  not  first  have  information  literate 
teachers.  The  purpose  of  this  needs  assessment  was  to  investigate  the  current  and  optimal  levels  of 
information  literacy  among  the  faculty  members  at  South  Georgia  College.  A few  key  questions  this 
assessment  answers  are:  What  is  the  current  state  of  information  literacy  among  SGC  faculty?  What  is  the 
optimal  state  of  information  literacy  among  SGC  faculty?  For  those  faculty  members  who  are  information 
literate,  what  technology  are  they  using  to  demonstrate  information  literacy  in  their  instruction?  For  those 
faculty  members  who  do  not  use  technology  for  gathering  and  presenting  data  in  their  course  instruction, 
what  are  the  reasons  for  why  they  do  not?  How  can  those  who  are  not  information  literate  be  brought  up  to 
proficiency?  How  can  those  who  are  information  literate  help  those  who  are  not? 

South  Georgia  College  is  a two-year  public  college  under  the  University  System  of  Georgia.  Average 
student  enrollment  for  SGC  is  between  1,200  - 1,250  students.  There  are  41  full-time  faculty  members  who 
teach  at  the  college.  Only  full-time,  teaching  faculty  members  from  the  mapr  academic  divisions  on 
campus  were  considered  as  subjects  for  research.  Each  faculty  member  was  asked  to  complete  a closed- 
ended  survey  of  ten  questions  to  determine  the  overall  information  literacy  of  SGC*s  faculty  based  upon 
qualitative  analysis.  Then,  six  faculty  members,  two  from  each  of  the  three  major  divisions  on  campus, 
were  randomly  selected  and  asked  to  participate  in  an  in-depth  interview  to  provide  the  qualitative  analysis 
of  the  needs  assessment. 

The  two  instruments  used  to  conduct  the  research  were  in-depth  interviews  and  a closed-ended  survey.  The 
closed-ended  survey  asked  objective,  general  questions  concerning  the  perceived  current  and  optimal 
levels  of  information  literacy  among  SGC  faculty,  the  needs  of  those  who  are  information  literate  and  the 
needs  of  those  who  are  not,  and  the  causes  for  why  a gap  exists  between  the  current  and  optimal  levels  of 
information  literacy  among  the  faculty.  The  in-depth  interviews  were  conducted  after  the  surveys  were 
tallied.  The  interviews  focused  on  gathering  subjective  data  from  participants.  The  interview  questions 
dealt  mainly  with  measuring  the  gap  between  the  current  and  optimal  levels  of  information  literacy  among 
SGC  faculty,  identifying  areas  of  concern  for  those  faculty  who  are  using  technology  as  well  as  for  those 
who  are  not,  and  soliciting  support  from  those  faculty  members  who  are  information  literate  to  help  bring 
those  who  are  not  up  to  task. 

From  the  data  collected,  on  average,  the  faculty  at  SGC  were  found  to  be  performing  at  an  acceptable  level 
of  information  literacy.  Both  the  survey  and  interview  data  supported  this.  However,  several  intriguing 
results  surfaced,  as  did  a few  problems  in  the  design  of  the  instruments.  The  survey  produced  the  finding 
that  faculty  members  still  prefer  traditional  printed  resources  for  gathering  information,  probably  because 
most  still  use  lesson  plans  that  were  developed  before  electronic  resources  were  popular  or  available. 
However,  the  first  place  faculty  would  look  today  for  finding  supplemental  information  is  not  traditional 
printed  resources  but  electronic  resources.  The  survey  findings  also  revealed  that  the  majority  of 
participants  do  not  know  the  laws  and  ethical  standards  associated  with  copyright  on  the  Internet. 
Furthermore,  most  faculty  members  surveyed  believed  that  even  if  specific  technology  were  made 
available,  a teacher  would  continue  to  rely  on  traditional  means  of  information  gathering.  The  greatest 
design  flaw  in  the  survey  was  that  several  participants  could  not  speculate  on  general  questions  that  asked 
them  to  rate  the  information  literacy  of  all  faculty.  Academic  Freedom  is  highly  upheld  in  post  secondary 
education;  thus,  participants  did  not  want  to  guess  at  what  their  colleagues  did  in  their  classrooms  because 
they  truly  do  not  know.  The  instrument  designer  did  not  consider  this  at  the  time  the  survey  was  created. 

The  most  surprising  issue  revealed  by  the  in-depth  interviews  was  that  most  participants  were  able  to 
distinguish  between  technology  literacy  and  information  literacy,  even  though  the  interviewer  did  not 
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discuss  technology  literacy,  A majority  of  the  participants  agreed  that  a teacher's  use  of  technology  in 
course  instruction  does  not  necessarily  reflect  his  or  her  knowledge  of  information  literacy.  The  in-depth 
interviews  were  not  without  flaw  either.  Only  four  of  the  Information  Literacy  Standards  used  to  rate 
participants ' responses  were  addressed  in  the  interview  questions.  This  was  a huge  oversight  on  behalf  of 
the  instrument  designer. 

In  conclusion,  several  steps  could  be  taken  to  help  faculty  members  become  even  more  information  literate. 
One  possible  solution  was  addressed  in  the  in-depth  interview  results.  Several  subjects  said  that  on-campus 
conferences  and  workshops  involving  technology  used  to  gather  information  would  certainly  help  them.  An 
inexpensive  solution  would  be  to  hold  periodical  teaching  circles  in  which  a group  of  teachers  from  various 
disciplines  convene  once  a month  to  discuss  how  they  gather  information  and  use  it  in  their  course 
instruction.  This  would  be  the  most  feasible  solution  to  the  problem.  A third  solution  would  be  for  the 
administration  to  promote  the  scholarship  of  teaching,  which  would  include  the  exercise  of  information 
literacy  skills.  Through  course  instruction  improvements,  faculty  members  can  begin  to  vie  for  specific 
grants  and  fellowships  that  reward  those  who  excel  in  the  scholarship  of  teaching. 

Introduction 

Several  college  faculty  members  have  incorporated  technology  and  various  forms  of  up-dated  information 
systems  into  their  course  instruction.  However,  for  every  faculty  member  who  has  up-dated  his  or  her  delivery 
methods,  there  is  a faculty  member  who  continues  to  utilize  the  methods  associated  with  stale  tradition.  Interactive 
web  pages,  PowerPoint  slides,  and  the  Internet  are  specific  technologies  that  faculty  members  are  - or  are  not  - using 
in  lieu  of  textbooks,  chalkboards,  and  printed  materials  to  present  new  ideas  on  how  to  retrieve  and  deliver 
information  to  their  students.  The  resulting  gap  is  very  clear,  some  faculty  members  are  quite  information  literate  in 
terms  of  electronic  resources  and  some  are  not;  nevertheless,  all  should  be  information  literate  to  a certain  extent. 
Located  in  Douglas,  Georgia,  South  Georgia  College  became  part  of  the  University  System  of  Geoigia  in 
1932.  One  of  its  primary  missions  is  to  promote  the  scholarship  of  teaching  via  the  latest  technology  and 
instructional  designs  available.  Within  the  past  three  years,  two  computer-based  classrooms  have  been  added  to  the 
campus  to  help  promote  teaching  with  the  use  of  technology.  Today,  SGC  has  a total  of  three  model  classrooms  used 
by  a variety  of  faculty  members  to  teach  their  subjects,  such  as  psychology,  history,  and  physics.  Average  student 
enrollment  for  is  between  1,200  - 1,250  students.  There  are  41  full-time  faculty  members  who  teach  at  the  college. 
The  Information  Literacy  Competency  Standards  for  Higher  Education,  approved  by  the  Association  of 
College  and  Research  Libraries  in  January  2000,  provides  a list  of  standards,  performance  indicators,  and  outcomes 
that  define  what  the  information  literate  student  must  be  capable  of  achieving.  The  information  literate  student  must 
be  able  to  analyze  what  information  is  needed,  as  well  as  what  information  is  and  is  not  appropriate  for  the  purpose 
involved.  He  or  she  then  must  know  how  to  retrieve  the  information  either  individually  or  with  a group  for 
accomplishing  a specific  purpose  (ACRL,  2000).  However,  before  the  student  can  become  information  literate,  he  or 
she  must  be  taught  information  literacy  skills.  That  is  where  the  faculty  member’s  information  literacy  comes  into 
question.  Higher  education  cannot  produce  information  literate  students  if  it  does  not  first  have  information  literate 
teachers  (Roth,  1999). 

Not  only  will  this  needs  assessment  be  beneficial  to  the  specific  clients  involved,  it  will  also  contribute  to 
research  within  the  scholarship  of  teaching.  Similar  two-year  institutions  can  use  this  study  as  an  example  for 
conducting  research  at  there  own  institutions.  They  can  implement  the  recommendations  from  the  results  of  this 
needs  assessment  at  their  institutions  without  having  to  conduct  a formal  needs  assessment  of  their  own.  Finally, 
several  new  questions  and  needs  emerged  from  the  results  of  this  research,  such  as  the  need  for  faculty  members  at 
South  Georgia  College  to  become  better  aware  of  the  ethics  and  copyright  laws  associated  with  using  materials  from 
the  Internet  for  instruction.  The  conductors  of  this  needs  assessment  could  certainly  continue  their  research  in  more 
depth  at  South  Georgia  College;  other  assessors  could  continue  this  type  of  research  at  their  schools  as  well. 

A few  key  questions  this  assessment  answers  are:  What  are  the  standards  that  will  determine  who  is 
information  literate  and  who  is  not?  What  is  the  current  state  of  information  literacy  among  SGC  faculty?  What  is 
the  optimal  state  of  information  literacy  among  SGC  faculty?  For  those  faculty  members  who  are  information 
literate,  what  technology  are  they  using  to  demonstrate  information  literacy  in  their  instruction?  For  those  faculty 
members  who  do  not  use  technology  for  gathering  and  presenting  data  in  their  course  instruction,  what  are  the 
reasons  for  why  they  do  not?  How  can  those  who  are  not  information  literate  be  brought  up  to  proficiency?  How  can 
those  who  are  information  literate  help  those  who  are  not? 
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South  Georgia  College  consists  of  41  faculty  members  from  various  academic  backgrounds.  Only  full-time, 
teaching  faculty  members  from  the  major  academic  divisions  on  campus  were  considered  as  subjects  for  research. 

All  academic  faculty  members  were  asked  to  complete  a survey.  Six  faculty  members,  two  from  each  of  the  four 
major  divisions  on  campus,  were  randomly  selected  and  asked  to  participate  in  an  interview.  Those  divisions 
included  the  Division  of  Humanities  and  Learning  Support,  Division  of  Business  and  Social  Sciences  and  Division 
of  Science  and  Mathematics. 

The  two  instruments  used  to  conduct  research  were  in-depth  interviews  and  a closed-ended  survey.  The 
closed-ended  survey  consisted  of  general  questions  designed  to  validate  the  issues,  needs  and  causes  of  the  needs 
assessment.  In  other  words,  the  survey  asked  objective,  general  questions  concerning  the  perceived  current  and 
optimal  levels  of  information  literacy  among  SGC  faculty,  the  needs  of  those  who  are  information  literate  and  the 
needs  of  those  who  are  not,  and  the  causes  for  why  a gap  exists  between  the  current  and  optimal  levels  of 
information  literacy  among  the  faculty.  The  in  -depth  interviews  were  conducted  after  the  surveys  were  tallied.  The 
interviews  focused  on  gathering  subjective  data  from  participants.  The  interview  questions  dealt  mainly  with 
measuring  the  gap  between  the  current  and  optimal  levels  of  information  literacy  among  SGC  faculty,  identifying 
areas  of  concern  for  those  faculty  who  are  using  technology  as  well  as  for  those  who  are  not,  and  soliciting  support 
from  those  faculty  members  who  are  information  literate  to  help  bring  those  who  are  not  up  to  task. 

The  procedures  for  developing  and  conducting  research  consisted  of  several  steps.  First,  permission  from 
the  Vice  President  of  Academic  Affairs  to  conduct  research  on  the  campus  of  South  Georgia  College  was  obtained. 
Each  interview  subject  signed  a written  consent  form  before  participating  in  this  study.  Finally,  permission  from  the 
VSU  Institutional  Review  Board  for  the  Protection  of  Human  Subjects  in  Research  and  Research  Related  Activities 
was  obtained  before  carrying  out  this  needs  assessment. 

Next,  the  standards  used  to  represent  the  optimal  level  of  information  literacy  among  SGC  faculty  were 
developed.  These  standards  were  adopted  from  the  Association  of  College  and  Research  Libraries  (ACRL) 
Information  Literacy  Competency  Standards  for  Higher  Education  (ACRL,  2000).  Although  these  standards  and 
performance  indicators  are  relative  to  the  students  of  higher  education,  they  also  apply  to  teachers  of  higher 
education  since  it  is  the  teachers  who  must  teach  the  students  how  to  be  information  literate.  Thus,  the  Faculty 
Information  Literacy  Standards  for  assessing  the  optimal  level  of  information  literacy  among  SGC  faculty  were 
designed  from  those  proposed  for  students  by  the  ACRL.  Those  standards  include  the  following  - Standard  I:  The 
information  literate  teacher  determines  the  nature  and  extent  of  the  information  needed;  Standard  II:  The  information 
literate  teacher  accesses  needed  information  effectively  and  efficiently;  Standard  III:  The  information  literate  teacher 
evaluates  information  and  its  sources  critically  and  incorporates  selected  information  into  his  or  her  knowledge  base 
and  value  system;  Standard  IV:  The  information  literate  teacher,  individually  or  as  a member  of  the  a group,  uses 
information  effectively  to  accomplish  a specific  purpose;  and  Standard  V:  The  information  literate  teacher 
understands  many  of  the  economic,  legal,  and  social  issues  surrounding  the  use  of  the  information  and  accesses  and 
uses  information  ethically  and  legally.  The  specific  objectives  that  accompany  each  benchmark  are  those  same 
objectives  that  can  be  found  on  the  ACRL’s  web  site  for  the  student  Information  Literacy  Competency  Standards  for 
Higher  Education. 

After  gathering  research,  the  data  from  the  interviews  was  compared  to  these  standards  to  assess  which 
faculty  members  are  information  literate  and  which  are  not.  Each  subject’s  interview  was  rated  to  determine  how 
information  literate  those  faculty  interviewed  were  on  a scale  of  one  to  five,  one  being  “Optimal  Level  of 
Information  Literacy”  and  five  being  “No  Level  of  Information  Literacy.” 

After  the  standards  were  set  for  assessing  the  optimal  level  of  information  literacy,  the  instruments  were 
designed  and  developed.  The  in-depth  interview  questions  were  developed  from  a sample  interview  outlined  in 
Chapter  3,  Appendix  B,  of  The  User-Friendly  Handbook  for  Mixed  Method  Evaluation.  The  closed-ended  survey 
was  developed  from  the  format  of  a sample  instructor  evaluation  used  by  the  Division  of  Business  and  Social 
Sciences  at  South  Georgia  College. 

The  first  instrument  distributed  was  the  closed-ended  survey.  Through  campus  mail,  it  was  delivered  to  all 
fulltime,  academic  faculty  members.  The  survey  focused  on  more  general  and  objective  data  that  pertained  to  the 
information  literacy  perceptions  of  all  faculty  members  from  the  various  academic  disciplines  on  campus.  A memo 
from  the  Vice-President  of  Academic  Affairs  accompanied  the  survey,  asking  all  faculty  to  participate  in  this 
research  endeavor.  A statement  at  the  top  of  the  survey  explaining  the  purpose  of  the  research  ensured  that  the 
subjects  knew  the  validity  and  confidentiality  of  the  data  retrieved.  Each  subject  was  asked  to  complete  the  survey 
and  return  it  once  they  had  answered  all  the  questions.  The  data  from  all  surveys  were  tallied  using  an  Excel 
spreadsheet. 

Six  faculty  members,  roughly  15%  of  the  faculty  population,  were  randomly  selected  to  interview.  One 
interviewee  was  from  each  of  the  following  academic  areas:  mathematics,  sciences.  Humanities,  learning  support. 
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business  and  social  sciences.  Before  administering  the  interview,  each  participant  completed  a consent  form  so  they 
fully  understood  the  confidentiality  of  the  interview.  All  interviews  but  one  was  tape-recorded.  The  interviews  were 
not  documented  verbatim,  but  rather  documented  through  the  “note  expansion”  method  of  data  collection.  The  tape 
recordings  were  only  used  for  clarity  and  backup.  The  data  from  all  interviews  were  complied  using  a spreadsheet  in 
Microsoft  Excel. 

The  information  found  from  the  survey  was  complied  using  the  program  Microsoft  Excel.  First,  all 
responses  from  the  survey  were  entered  into  an  Excel  spreadsheet.  Each  survey  question  was  typed  into  the 
spreadsheet  with  each  subject’s  response  listed  in  columns  running  along  side  the  corresponding  questions.  The 
closed-ended  questions  on  the  survey  easily  allowed  the  options  of  “Strongly  Agree,”  “Agree,”  “Disagree,”  and 
“Strongly  Disagree”  to  be  translated  into  a scale  of  one  to  four,  one  representing  “Strongly  Agree”  and  four 
representing  “Strongly  Disagree.”  After  the  surveys  were  conducted,  a rating  of  five,  which  stood  for,  “Don’t 
Know,”  was  included  since  some  participants  wrote  this  answer  in.  The  ratings  for  each  question  were  averaged, 
establishing  an  overall  rating  for  each  question. 

The  data  collected  from  the  interviews  were  compiled  and  analyzed  in  much  the  same  way  as  the  survey. 
The  interview  questions  were  documented  in  a spreadsheet  and  all  responses  from  those  interviewed  were  typed  into 
columns  running  side-by-side  next  to  the  corresponding  question.  This  allowed  for  the  answers  given  to  a particular 
question  to  be  seen  at  once  for  easy  comparison  to  the  standards.  Each  interview  was  rated  on  a scale  of  one  to  five, 
one  representing  “optimal  level  of  information  literacy”  and  five  representing  “no  level  of  information  literacy.” 
Finally,  the  overall  rating  for  each  interview  was  averaged  to  find  the  mean  of  all  the  interviews.  This  helped 
establish  an  information  literacy  rating  for  the  entire  faculty.  This  information  was  then  compared  with  the  results 
found  from  the  survey  data.  Our  hypothesis  was  that  the  general  consensus  of  the  survey  would  match  the  subjective 
observations  of  the  interviews  to  provide  an  objective  conclusion  concerning  the  optimal  and  current  levels  of 
information  literacy  among  the  faculty  of  SGC. 

Results 


Out  of  all  surveys  administered,  18  were  returned.  The  responses  of  the  surveys  were  tallied  and  the  results 
placed  in  an  Excel  spreadsheet.  Each  subject’s  response  was  tallied  according  to  a rating;  then  each  question 
received  an  average  rating  that  allowed  us  to  conclude  our  results. 

Several  intriguing  results  surfaced,  as  did  a few  problems  in  the  design  of  the  survey  questions.  On 
questions  one  (Information  literacy  is  the  ability  to  recognize  when  information  is  needed,  then  locate,  evaluate  and 
use  that  information  effectively  to  fulfill  the  need.  Given  this  definition  of  information  literacy,  at  least  60%  of  SGC 
faculty  are  information  literate.),  two  (Given  the  definition  of  information  literacy  stated  above  [in  question  one], 
you  consider  yourself  to  be  information  literate.),  seven  (All  academic  information  obtained  from  the  Internet  does 
not  need  to  be  considered  under  copyright;  it  is  available  for  all  to  use  freely.),  and  nine  (Faculty  members  who  do 
not  demonstrate  information  literacy  in  their  course  curriculum  should  attend  workshops  or  activities  to  learn  how  to 
do  so.),  we  received  the  expected  outcome  that  we  had  hoped  for.  We  had  hypothesized  that  an  average  of  the 
participants  would  have  agreed  that  the  majority  of  his  or  her  colleagues,  as  well  as  him  or  herself,  were  information 
literate.  We  had  also  expected  that  an  average  of  the  participants  would  have  disagreed  on  the  issue  of  all  academic 
information  obtained  from  the  Internet  not  being  considered  under  copyright.  Finally,  we  also  anticipated  that  the 
average  answer  would  be,  “Agree,”  when  the  participants  were  asked  if  faculty  members  who  are  not  information 
literate  need  to  attend  workshops  or  activities  to  learn  how  to  become  information  literate. 

The  other  questions  proved  to  have  differing  results  from  what  we  had  expected.  Question  three  asked 
participants  if  multimedia,  electronic  databases,  and  web  sites  were  the  primary  media  used  by  faculty  to 
demonstrate  information  literacy  in  their  course  curriculum.  The  average  answer  was  “Disagree.”  We  concluded  that 
this  finding  suggests  faculty  members  still  prefer  traditional  printed  resources  to  gain  information,  probably  because 
most  still  use  lesson  plans  that  were  developed  before  electronic  resources  were  popular  or  available.  This  finding 
concurred  with  the  results  of  our  interviews  as  well.  However,  our  results  from  question  six  (The  first  place  you 
would  look  to  find  supplemental  information  on  your  academic  subject  would  be  to  obtain  printed  materials,  such  as 
books,  journals,  and  magazines,  from  the  school  library.)  showed  that  “the  tides  are  beginning  to  turn.”  The  average 
answer  to  this  question  was,  “Disagree.”  This  means  that  the  first  place  faculty  would  look  for  finding  supplemental 
information  is  not  traditional  printed  resources  but  rather  non-traditional  resources,  e.g.  electronic  resources.  As  for 
whether  they  would  recommend  a web  site  to  their  students  as  the  first  place  to  find  supplemental  material  for  the 
course,  question  eight,  the  average  response  was  barely  “Agree”  ~ 2.44.  This  proved  that  some  teachers  would  have 
immediately  recommended  a web  site  and  others  would  not  have  done  so. 
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Questions  four  and  five  yielded  a great  deal  of  controversy  since  several  people  wrote  in  answers  of, 

“Don’t  Know,”  or  “Problematic  question,”  rather  than  answering.  We  believe  that  this  confusion  on  question  four  - 
which  asks  if  a faculty  member  uses  on-site  library  and  printed  resources  in  his  or  her  instruction,  he  or  she  is 
information  literate  - could  have  resulted  from  a design  flaw  in  the  survey.  Participants  may  have  thought  from  the 
previous  three  questions  that  information  literacy  only  deals  with  technology.  If  question  five  - combining  both 
printed  and  electronic  media  to  yield  the  most  information  literate  teacher  - had  come  before  question  four,  we 
believe  more  accurate  results  would  have  emerged  for  question  four. 

The  greatest  design  flaw  that  we  found  from  the  survey  was  that  several  participants  answered,  “Don’t 
Know,”  to  questions  one,  three,  four,  and  seven.  These  questions  asked  the  participant  to  answer  based  on 
generalizations  concerning  the  entire  faculty.  We  felt  that  because  Academic  Freedom  is  highly  upheld  in  post 
secondary  education,  those  participants  who  answered,  “Don’t  Know,”  did  not  want  to  guess  at  what  their 
colleagues  did  in  their  classrooms  because  they  truly  do  not  know.  The  questions  should  have  been  reworded  to  ask 
the  participants  what  they  did  in  their  own  course  instruction,  not  to  generalize  for  the  entire  target  population.  This 
held  true  for  all  these  questions  except  question  seven.  Subjects  responded,  “Don’t  Know,”  to  this  question  because 
they  truly  do  not  know  the  laws  and  ethical  standards  associated  with  copyright  on  the  Internet. 

The  most  significant  discovery  from  the  survey  results  was  that  the  average  answer  for  question  ten  - 
whether  or  not  those  teachers  who  are  not  information  literate  would  become  more  information  literate  if  they  had 
better  access  to  technology  - was,  “Disagree.”  This  showed  that  most  of  those  faculty  members  surveyed  believe 
that  even  if  specific  technology  were  made  available,  a teacher  would  either  continue  to  rely  on  traditional  means  of 
information  gathering  or  that  technology  has  no  effect  on  how  information  literate  one  is. 

The  interviews  were  conducted  within  the  span  of  one  week.  The  following  questions  were  asked  of  each 
participant  - Question  one:  What  types  of  resources  do  you  use  to  gather  information?  Question  two:  How  do  you 
use  technology  to  gather  information?  (If  you  do  not  use  technology  to  gather  information,  why  not?)  Question 
three:  Do  you  think  a teacher’s  use  of  technology  in  course  instruction  reflects  knowledge  of  information  literacy? 
Why  or  why  not?  Question  four:  What  methods  of  information  retrieval  are  best  suited  for  your  course  instruction? 
Question  five:  What  methods  of  information  retrieval  would  you  suggest  to  students  for  finding  additional 
information  for  your  course?  Question  six:  What  programs  or  activities  do  you  think  would  help  you  become  more 
information  literate?  Would  technology  be  a part  of  those  programs  or  activities? 

The  results  from  the  In-depth  Interviews  supported  some  of  the  findings  from  the  survey  in  greater  detail. 
The  average  interview  rating  was  2.83,  which  ranked  the  average  participant  as  being  “Average  Level  of 
Information  Literacy.”  We  found  that  to  be  a very  positive  result  from  the  interviews.  However,  two  interviewees 
ranked  as  being,  “Above  Average,”  three  ranked  as,  “Average,”  and  one  ranked  as  being,  “Below  Average.”  After 
having  compared  the  participants’  answers  to  the  Faculty  Information  Literacy  Standards  devised  for  this  needs 
assessment,  we  ranked  them  according  to  the  rating  scale.  The  main  areas  that  kept  some  interviewees  from  ranking 
higher  were  in  Standard  1 where  they  did  not  show  that  they  used  a variety  of  types  and  formats  of  potential 
resources  for  information,  particularly  electronic  resources.  For  example,  the  three  subjects  who  ranked  “Average” 
drew  most  of  their  resources  from  printed  information,  such  as  library  databases,  reference  books,  and  textbooks. 
Very  few  electronic  resources  were  used  by  these  three  subjects  for  gathering  information.  Two  participants  showed 
competencies  in  all  areas  of  the  standards  while  one  person  only  showed  competency  in  one  standard  - Standard  IV. 
Subject  #3  did  not  use  a variety  of  resources  (Standard  I),  did  not  retrieve  information  online  or  in  person  using  a 
variety  of  methods  (Standard  II),  and  did  not  compare  new  knowledge  to  determine  the  value  added,  contradictions, 
or  other  unique  characteristics  of  the  information  (Standard  III).  He  relied  mostly  on  his  textbooks  for  all 
information  gathering.  This  type  of  observation  is  also  seen  in  the  survey  results  on  a more  general  scale. 

Only  four  of  the  Standards  were  used  when  comparing  the  participants’  answers  to  the  standards  since  none 
of  the  interview  questions  related  to  Standard  V.  This  was  a huge  oversight  on  the  part  of  the  instrument  designers. 
All  other  questions  related  to  Standards  I-IV,  but  no  questions  asked  interviewees  about  their  understanding  of 
copyright  or  ethical  use  of  information  gathering.  However,  Subject  #5  did  volunteer  that  she  would  like  to  have 
programs  or  workshops  to  learn  more  about  copyright  laws  as  part  of  her  answer  to  question  six.  This  participant’s 
interview  rating  was  “Above  Average  Level  of  Information  Literacy.” 

Several  positive  findings  did  emerge  from  the  interviews,  particularly  in  the  answers  to  questions  three  and 
six.  Most  subjects  answered  question  three  with  a clear  understanding  between  being  technology  literate  and 
information  literate,  even  though  technology  literacy  was  not  discussed  by  the  interviewer.  Question  three  asked 
whether  or  not  a teacher’s  use  of  technology  in  course  instruction  reflects  his  or  her  knowledge  of  information 
literacy.  A majority  of  participants  stated,  “Not  necessarily,”  and  went  on  to  explain  why.  Subject  #6  gave  the  best 
example,  “For  example,  you  may  know  how  to  use  PowerPoint  but  not  know  how  to  find  information  on  the  subject 
for  the  presentation.”  This  was  one  of  the  greatest  discoveries  from  the  interview  results.  Question  six  yielded  a 
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great  deal  of  possible  workshops  or  activities  that  could  be  implemented  as  part  of  the  solution  to  this  needs 
assessment. 

Summary  and  Recommendations 

The  purpose  of  this  needs  assessment  was  to  investigate  the  current  and  optimal  levels  of  information 
literacy  among  SGC  faculty  members.  In  other  words,  a gap  existed  between  how  faculty  members  currently  gather 
information  and  how  they  should  be  gathering  information.  The  data  collected  and  analyzed  for  this  assessment 
showed  how  faculty  members  are  gathering  information.  The  Faculty  Information  Literacy  Standards,  adopted  from 
the  ACRL’s  Information  Literacy  standards,  shows  the  assessors  how  SGC  faculty  members  should  be  gathering 
information.  We  believe  that  from  the  data  collected,  on  average,  the  faculty  members  at  SGC  are  performing  at  an 
acceptable  level  of  information  literacy.  However,  several  steps  could  be  taken  to  help  the  majority  of  faculty 
members  become  even  more  information  literate. 

One  possible  solution  was  addressed  in  the  In-depth  Interview.  Interviewees  were  asked  what  programs  or 
activities  might  help  them  become  more  information  literate  and  would  technology  be  a part  of  those  programs  or 
activities.  Several  subjects  said  that  conferences  and  workshops  involving  particular  types  of  technology  would 
certainly  help  them.  For  example,  one  person  would  be  interested  in  further  Galileo  training,  two  would  like  to  learn 
more  about  how  to  retrieve  information  from  the  web  and  research  using  the  Internet,  yet  another  would  like  to 
explore  how  online  chats  can  be  used  to  hear  and  write  Spanish. 

One  workshop  that  was  highly  recommended  is  one  that  was  posed  by  Subject  #5  from  the  Interview  - 
programs  or  workshops  to  learn  about  copyright  laws  on  the  Internet.  From  both  the  survey  and  interview  results, 
this  is  one  area  of  the  Information  Literacy  Standards  that  the  SGC  faculty  members  do  not  know  nor  understand. 
Knowing  copyright  and  ethical  laws  concerning  what  to  use  and  how  much  of  it  to  use  from  the  Internet  and  World 
Wide  Web  in  course  instruction  is  certainly  a paramount  issue  vis  -a-vis  information  literacy.  These  workshops  do 
not  have  to  be  expensive  or  lengthy  in  time.  As  Subject  #6  suggested  in  his  interview,  brown  bag  workshops  held 
during  lunch  could  be  provided  on  campus  by  librarians  or  those  faculty  members  considered  highly  information 
literate  to  help  bring  others  up  to  task.  To  encourage  participation,  door  prizes  could  be  given  out  at  the  beginning 
of  the  sessions  or  a brown  bag  lunch  be  provided  by  the  school  for  those  who  do  come. 

Another  inexpensive  solution  to  bridging  the  gap  between  the  current  and  optimal  states  of  information 
literacy  among  faculty  would  be  to  hold  periodical  teaching  circles  in  which  a group  of  teachers  from  various 
disciplines  come  together  once  a month  to  discuss  how  they  gather  information  and  use  it  in  their  course  instruction. 
This  could  also  be  done  during  a lunch  hour  in  which  the  teachers  go  out  to  a favorite  restaurant  or  bring  a brown 
bag  lunch.  Not  only  would  teaching  circles  promote  information  literacy,  but  it  would  also  promote  camaraderie 
among  the  faculty  members.  This  would,  in  turn,  provide  a solid  networking  foundation  for  faculty  members  to  pull 
upon  each  other  as  resources.  It  would  also  help  expose  all  faculty  members  to  a variety  of  up-to-date  information 
search  methods.  This  would  be  the  most  feasible  solution  to  implement, 

A third  solution  that  would  help  teachers  become  more  information  literate  at  SGC  would  be  for  the 
administration  to  make  the  scholarship  of  teaching  an  even  higher  priority  than  what  it  is  now.  This  could  be 
relatively  inexpensive  for  the  school  depending  on  the  help  from  administrators  and  division  chairpersons.  The 
scholarship  of  teaching  would  deal  with  faculty  members  making  a conscious  effort  to  improve  their  course  delivery 
methods  and  up-date  their  information  literacy  to  make  teaching  a valid  area  of  research.  Through  course  instruction 
improvements,  faculty  members  can  begin  to  vie  for  specific  grants  and  fellowships  that  reward  those  who  excel  in 
the  scholarship  of  teaching.  However,  this  will  require  that  the  administration  support  a faculty  member’s  endeavors 
and  provide  him  or  her  with  the  necessary  means  of  improving  course  delivery.  This  could  include  buying  or 
providing  up-dated  technology  or  paying  for  particular  off-campus  workshops  or  grant  writing  conferences. 

In  conclusion,  a more  in-depth  needs  assessment  should  be  conducted  in  the  area  of  information  literacy  at 
SGC,  The  needs  assessment  conducted  here  was  extremely  small  in  scale  and  uses  instruments  flawed  in  design. 
Although  the  majority  of  results  obtained  are  valid,  further  investigation  in  this  area  would  bring  about  more 
accurate  results.  Therefore,  more  information  does  need  to  be  gathered,  not  only  from  the  perspective  of  the  faculty, 
but  also  from  the  perspectives  of  the  administration  and  the  students.  If  objective  and  thorough  research  is  to  be 
found,  more  subjects  need  to  be  interviewed  and  surveyed,  and  a wider  variety  of  instruments  need  to  be  used  for 
finding  data. 
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What  Types  of  Online  Facilitation  Do  Students  Need? 


Abstract 


Shujen  L.  Chang 
Florida  State  University 


The  purpose  of  this  study  is  to  investigate  facilitation  in  an  online  learning  environment.  The  types  of 
facilitation  are  examined  using  distance  course  students  as  subjects.  Students  ’ characteristics  such  as  prior 
experience  in  distance  courses,  demographic  attributes  such  as  age,  gender,  and  grades  are  incorporated 
as  variables  to  explore  their  relationships  with  facilitation.  This  study  first  identifies  major  types  of  online 
facilitation  that  students  requested  in  a web-based  course.  Then  this  study  inspects  whether  students  with 
different  characteristics  requested  different  types  of  online  facilitation.  Finally  this  study  examines 
whether  the  amount  of  message  requesting  online  facilitation  is  associated  with  learning  achievement. 

The  participants  of  this  study  were  29  college  students  in  a web-based  course  at  a state  university.  The 
course  requires  students  to  actively  participate  in  online  activities  and  all  course  assignments  are 
submitted  as  electronic  files  via  the  course  site.  Research  data  are  the  threaded  messages  that  were  posted 
by  students,  the  instructor,  and  the  TA  in  the  designated  areas  in  the  course  site.  Threaded  messages  posted 
through  entire  semester  are  collected. 

Five  types  of  online  facilitation  were  identified  in  this  study.  Findings  and  implications  of  findings 
regarding  the  relationships  among  types  of  online  facilitation,  student  characteristics,  and  learning 
achievement  were  discussed.  It  was  found  that  students  requested  more  facilitation  in  assignments  and 
grade  criteria,  and  in  network  access.  Different  student  characteristics  were  found  to  prefer  different  types 
of  facilitation.  Implication  of  this  study  for  web-based  course  design  and  teaching  was  then  suggested. 

Introduction 

As  more  college  courses  are  placed  online,  effective  instruction  tools  of  online  courses  has  become  one  of 
key  issues  of  investigation.  Among  the  instruction  tools  adopted  in  online  courses,  facilitation  is  a variable  less 
studied.  The  purpose  of  this  study  is  to  explore  facilitation  as  an  effective  instruction  tool  and  examine  the 
relationships  among  types  of  online  facilitation,  student  characteristics,  and  learning  achievement.  Students  taking 
online  courses  tend  to  learn  alone  in  front  of  their  computers  and  are  distant  from  their  instructors  or  classmates  both 
physically  and  psychologically.  Therefore,  they  need  assistance  in  solving  problems  such  as  accessing  to  course 
materials  in  course  site  or  in  clarifying  expectations  of  assignments  online.  The  instructor  or  student  peers  may 
respond  to  a student’s  request  so  the  student  can  learn  to  get  what  he  needs  online.  Therefore,  facilitation  is  even  a 
more  important  factor  in  scaffolding  learning  in  web-based  environments  as  compared  to  the  traditional  classrooms. 
However,  very  few  existing  studies  have  looked  into  the  nature  of  facilitation  in  online  learning  environments.  This 
has  presented  a research  opportunity  for  this  study  to  explore  types  of  facilitation  and  variables  affecting  facilitation. 


The  findings  of  the  study  will  provide  new  insight  into  designing  and  teaching  web-based  courses. 
Specifically,  the  study  identifies  types  of  online  facilitation  and  suggests  that  both  web-based  course  designers  and 
instructors  should  pay  more  attention  to  some  particular  types  of  facilitation  that  students  requested  more  often.  The 
findings  also  suggest  that  different  students  may  require  different  amount  of  facilitation. 

Literature  Review 


Online  Facilitation 


In  online  learning  environments,  on-line  facilitation  has  been  proposed  in  previous  studies  as  an  effective 
pedagogical  strategy  to  increase  interaction  and,  then,  to  enhance  learning  achievement.  Althaus  (1997)  studied  the 
effects  of  computer-mediated  discussion  (CMD)  on  learning  achievement.  He  recommended  providing  facilitation  in 
order  to  encourage  students  to  participate  in  CMD,  especially  those  students  who  lacked  online  skills.  Hmelo, 
Guzdial,  and  Turns  (1998)  investigated  the  effect  of  an  online  forum  for  collaboration  and  reflection  in  the 
Collaborative  and  Multimedia  Interactive  Learning  Environment  (CaMILE).  The  authors  suggested  that  some  forms 
of  online  facilitation  were  needed  either  to  provide  a logical  connection  between  student  activities  and  conceptual 
topics  or  to  provide  the  concrete  referent  that  was  needed  for  reflective  discussion.  Furthermore,  Garland  (1993) 
inspected  students’  perceptions  of  barriers  in  completing  distance  courses.  To  overcome  these  barriers,  she 
recommended  providing  adequate  facilitation  to  students  in  terms  of  more  communication.  Although  online 
facilitation  has  been  suggested  as  an  effective  pedagogical  strategy  by  many  researchers,  it  is  unclear  that  whether 
online  facilitation  is  equally  effective  in  different  student  demography. 

Student  Characteristics 

Student  characteristics  have  been  predicted  to  have  impact  on  learning  outcomes.  Major  student 
characteristics  investigated  in  computer-aided  or  online  learning  environments  are  prior  experience  in  similar 
learning  environments,  personal  attributes,  and  learning  achievement.  As  to  prior  experience,  previous  studies 
suggest  that  prior  experiences  in  using  computer  are  positively  related  to  academic  achievement.  Althaus  (1997) 
found  that  students  who  had  more  e-mail  experience  were  more  willing  to  participate  in  Computer-Mediated 
Discussion  (CMD),  and  students  who  fully  participated  in  CMD  performed  better  than  others  who  participated  less. 
Grantham  and  Vaske  (1985)  found  that  the  amount  of  prior  experience  in  e-mail  was  positively  associated  with 
telecomputing  use.  Hiltz  (1993)  concluded  that  students’  comfortableness  with  computers  was  one  of  the  major 
characteristics  that  would  lead  to  better  learning  in  computer-aided  environments.  Also,  Harris  and  Grandgenett 
(1996)  found  that  telecomputing  experience  was  positively  related  to  greater  online  time,  and  greater  online  time 
could  lead  to  higher  achievement. 

The  consistent  positive  association  between  prior  experience  in  using  computer  and  more  efforts  on  online 
learning  may  imply  that  students  who  have  no  experience  in  distance  learning  may  need  more  facilitation  to  assist 
them.  But,  there  are  few  studies  examining  the  relationship  between  the  need  for  facilitation  and  prior  experience. 

Regarding  student’s  personal  attributes  such  as  gender  and  age,  there  is  little  research  that  studies  how 
personal  attributes  affect  the  need  for  facilitation.  Previous  studies,  investigating  learner  characteristics  and 
computer  anxiety  in  computer-aided  learning  environments,  suggest  that  such  a link  may  exist.  High  computer 
anxiety  students  may  need  more  facilitation  to  reduce  anxiety  level.  However,  previous  studies  on  computer  anxiety 
showed  inconclusive  results  concerning  the  relationship  between  personal  attributes  and  computer  anxiety. 

A meta-analysis  of  81  studies  reported  that  computer  anxiety  was  not  significantly  correlated  with  gender 
and  age  (Rosen  & Maguire,  1990).  But,  another  study  reported  that  feminine-identity  students  showed  higher 
anxiety  than  masculine-identity  students  did  (Rosen  & Sears,  1987).  Other  studies  also  reported  significant 
correlation  between  computer  anxiety  and  age.  In  one  study  it  was  reported  that  older  adults  (55  years  old  and  over) 
appeared  to  have  less  computer  anxiety  than  younger  adults  (30  years  old  and  under)  did  (Dyck  & Smither,  1994). 
However,  in  other  studies  it  was  reported  that  older  students  were  more  computer  anxious  than  younger  students 
(Rosen,  Sears,  & Weil,  1987;  Jones  & Wall,  1989).  Five  studies  of  over  450  college  students  compared  the  computer 
anxiety  of  computer  and  business  major  students  with  that  of  other  major  students  and  found  that  computer  anxiety 
consistently  correlated  with  students’  majors  (Rosen,  Sears,  & Weil,  1987). 

Learning  Achievement 

While  previous  studies  addressed  very  little  about  the  relationship  between  online  facilitation  and  learning 
achievement,  some  studies  examining  the  relationship  between  computer  anxiety  and  learning  achievement.  A study 
explored  components  of  computer  anxiety  and  found  there  was  no  significant  relationship  between  anxiety  and 
students  ’ academic  achievement  in  the  computer  course  as  measured  by  their  course  grades  (Jones  & Wall,  1 989). 
Although  computer  anxiety  may  be  viewed  as  an  indicator  to  the  need  for  online  facilitation,  it  is  hard  to  draw  any 
conclusion  without  more  studies.  Thus,  this  study  is  devoted  to  investigating  the  relationship  between  learning 
achievement  and  online  facilitation. 


Method 


Participants 


The  participants  of  this  study  were  29  college  students,  21  females  and  8 males,  who  enrolled  in  a web- 
based  course  at  a state  university.  14  participants  majored  in  Open  Distance  Learning  and  1 5 participants  in  other 
degree  programs.  Participants’  ages  are  19  and  above.  All  29  participants  were  voluntarily  participated  in  this  study. 

The  Course 


This  study  was  conducted  in  a course  that  was  entirely  delivered  via  a web-based  course  site.  The  course 
title  was  Learning  Theories  and  Cognition  in  Instruction  and  was  offered  by  the  College  of  Education  at  a state 
university  in  the  southeast  of  the  United  States.  Students  were  required  to  submit  all  assignments  electronically  via 
the  course  site.  Also  online  participation  in  activities  was  mandatory.  The  final  course  grade  consisted  of  four 
assignments  (60%)  and  online  participation  (40%). 

Three  designated  areas  in  the  course  s ite  were  created  for  students  to  post  requests  for  facilitation.  These 
areas  were  Online  Office,  Student  Lounge,  and  Information  Sharing  (Figure  2).  Students  were  encouraged  to  post 
messages  in  these  areas  for  asking  extra  assistance,  raising  questions,  e?q)ressing  concerns,  and  sharing  information 
with  the  class.  The  instructor,  the  teaching  assistant  (TA),  and  students  might  respond  to  messages  in  these  areas. 
Also,  the  instructor  and  the  TA  might  also  respond  in  the  Announcements  area  in  the  course  site  (Figure  3) 
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Figure  2.  Online  Facilitation  Areas 


Figure  3.  Announcements 


Data  collection  and  Analysis 

Research  data  were  the  threaded  messages  that  posted  by  students,  the  instructor,  and  the  TA  in  the  Online 
Office,  Student  Lounge,  Information  Sharing,  and  Announcements  areas  in  the  course  site.  These  data  were 
collected  through  an  entire  semester.  In  the  beginning  of  the  semester,  a survey,  which  collected  the  information  of 
students’  demography,  personal  particulars,  and  prior  experience  in  distance  courses,  was  filled  by  participants. 

First,  based  on  the  senders  of  messages,  messages  were  classified  into  two  categories:  requesting  for 
facilitation  and  responding  to  requests.  Then,  based  on  the  content  of  questions  or  responses  in  the  message  texts, 
these  messages  were  further  classified  into  different  facilitation  types  within  each  category.  Total  frequencies  of 
requesting  facilitation  and  responding  to  requests  were  tallied.  Finally,  cross  tables  of  facilitation  type,  student 
characteristics,  and  learning  achievement  were  generated. 


Results 


Types  of  Online  Facilitation 

Five  types  of  online  facilitation  were  identified  from  the  content  analysis  of  the  threaded  messages  as 
posted  by  students  in  the  course  site.  The  total  number  of  request  messages,  which  requested  for  facilitation,  was  33. 
These  five  facilitation  types  with  descriptions  are  listed  in  the  following: 

1.  Assignments  and  grades:  Messages  that  asked  questions  about  due  dates,  the  instructor’s 
expectation,  grading  criteria  of  assignments,  and  course  grades 

2.  Network  access:  Messages  that  asked  questions  about  the  access  to  WWW  network  and  online 
course  materials  in  the  course  site 

3.  Online  discussion:  Messages  that  asked  questions  about  clarification,  reflections,  and  suggestions 
of  online  discussion 

4.  Group  activities:  Messages  that  asked  questions  about  group  activities 
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5.  Other  course  materials  access:  Messages  that  asked  questions  about  the  access  to  the  textbook, 
study  guide,  the  university,  and  local  libraries 

The  results  indicated  that  the  most  requested  type  of  facilitation  was  Assignments  and  grades,  which 
included  13  messages  and  represented  39%  of  total  messages  that  asked  for  facilitation.  The  second  most  requested 
type  of  facilitation  was  Network  access,  which  included  1 1 messages  representing  23%  of  total  messages  that  asked 
for  facilitation.  See  Table  1 for  the  frequency  of  messages  that  requested  online  facilitation  in  each  facilitation  type. 

There  were  85  response  messages  that  responded  to  the  request  messages.  Based  on  the  persons  who 
responded,  the  messages  were  sorted  into  three  categories:  instructor,  TA,  and  peer.  Peer  responses  were  the  highest 
responses  in  all  five  facilitation  types.  Peers  responded  54  messages,  which  consisted  of  64%  of  total  responded 
messages.  The  instructor  and  TA  responded  16  and  15  messages  and  consisted  of  19%  and  18%  of  total  response 
messages,  respectively.  See  Table  1 for  the  frequencies  of  response  messages  from  the  instructor,  TA,  and  peer  in 
each  facilitation  type. 

Student  Characteristics  and  Online  Facilitation 

Experience  in  distance  courses.  There  were  33  request  messages  posted  by  24  experienced  students  and  0 
messages  posted  by  5 non-experienced  students.  The  average  number  of  messages  was  1.38  per  experienced  student 
and  0.00  per  non-experienced  student. 


Table  1.  Types  of  facilitation  that  students  need  in  a web-based  course 


Facilitation  Type 

Frequency** 

Responded  by*** 

Total 

Responses 

Instructor 

TA 

Peer 

Assignments  & grades  * 

13*  (39%) 

10(25%) 

8 (20%) 

22  (55%) 

40 

Network  access* 

11*  (33%) 

0 (0%) 

2 (7%) 

26  (93%) 

28 

Online  discussion 

5(15%) 

5 (50%) 

2 (20%) 

3 (30%) 

10 

Group  activities 

3 (9%) 

1 (17%) 

3 (40%) 

2 (33%) 

6 

Other  course  materials  access 

1 (3%) 

0 (0%) 

0 (0%) 

1 (100%) 

1 

Total 

33 

16(19%) 

15(18%) 

54  (64%) 

85 

*The  most  needed  two  types  of  facilitation  **  Percentage  of  total  frequency*  **Percentage  of  total  responses 


Gender,  Among  the  33  messages  requesting  for  facilitation,  27  messages  were  posted  by  21  females  and  6 
messages  were  posted  by  8 males.  The  average  number  of  messages  was  1.29  per  female  and  0.75  per  male. 

Age,  There  were  13  messages  posted  by  17  younger  students  (age  19-35  years)  and  20  messages  posted 
by  12  older  students  (age  36  and  above).  The  average  number  of  messages  was  0.76  per  younger  student  and  1 .66 
per  older  student. 

Major,  There  were  29  messages  posted  by  14  students  of  Open  Distance  Learning  major  and  4 messages 
posted  by  students  of  other  majors.  The  average  number  of  messages  was  2.07  per  student  of  Open  Distance 
Learning  major  and  0.27  per  student  of  other  majors. 

GPA,  In  order  to  have  similar  number  of  students  in  each  cluster,  GPA  was  classified  into  three  clusters: 
low  GPA  (3.0-3.69),  medium  GPA  (3.70-3.99),  High  GPA  (4.00).  There  were  16  messages  posted  by  1 1 high  GPA 
(4.00)  students,  17  messages  posted  by  9 medium  GPA  (3.70-3.99)  students,  and  0 messages  posted  by  8 low  GPA 
(3.00-3.69)  students.  The  average  number  of  messages  was  1.45  per  high  GPA  student,  1.89  per  medium  GPA 
student,  and  0.00  messages  per  low  GPA  student.  Table  2 lists  the  amount  and  average  of  messages  that  requested 
facilitation  under  each  student’s  characteristic. 


Table  2.  Student  Characteristics  and  Online  Facilitation 


Student  Characteristic 

Total  number 
of  students 

Total  frequency  of 

requesting 

facilitation 

Average 
Frequency  per 
student 

Experience  in 
distance  courses 

No 

5 

0 

0.00 

Yes 

24 

33 

1.38 

Gender 

Female 

21 

27 

1.29 

Male 

8 

6 

0.75 

Age 

Younger  (age  19-35) 

17 

13 

0.76 

Older  (age  36  and  above) 

12 

20 

1.70 
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Major 

Open  Distance  Learning 

14 

29 

2.07 

Other  majors 

15 

4 

0.27 

GPA* 

Low  (3.0-3.69) 

8 

0 

0.00 

Medium  (3.70-3.99) 

9 

17 

1.89 

High  (4.00) 

11 

16 

1.45 

♦There  is  one  datum  missing  in  GPA. 


The  Most  Needed  Facilitation  Type  within  Each  Student  Characteristic 

The  most  requested  facilitation  type  within  each  student  characteristic  was  analyzed.  Overall,  the  results 
showed  consistently  that  the  most  requested  types  of  facilitation  were:  1)  Network  access,  2)  Assignments  and 
grades.  However,  further  observation  reveals  that  the  most  requested  facilitation  may  differ  in  different  student 
characteristics.  It  can  be  observed  that  students  with  prior  distance  learning  experience  needed  facilitation  in 
assignments  and  grades  the  most,  while  non-experienced  students  desired  facilitation  in  network  access  the  most. 
The  most  needed  facilitation  for  female  students  was  assignments  and  grades,  whereas  for  male  students  was 
network  access.  While  younger  students  (years  19  - 35)  needed  facilitation  in  assignments  and  grades  the  most, 
older  students  (years  36  and  above)  requested  facilitation  in  network  access  the  most.  Open  Distance  Learning 
majors  needed  facilitation  in  network  access  the  most,  and  other  major  students  desired  facilitation  in  assignments 
and  grades  the  most.  Both  high  and  medium  GPA  students  needed  facilitation  in  network  access  the  most.  However, 
low  GPA  students  did  not  request  any  facilitation.  See  Table  3 for  the  most  needed  type  of  facilitation  in  each 
learner  characteristic. 


Table  3.  The  Most  Needed  Facilitation  Type  by  Student  Characteristic 


Student  Characteristic 

Facilitation  Type 

Experience  in 
distance  learning 

No 

Assignments  & grades 

Yes 

Network  access 

Gender 

Female 

Assignments  & grades 

Male 

Network  access 

Age 

19-35 

Assignments  & grades 

36  and  above 

Network  access 

Major 

ODL 

Network  access 

Other  majors 

Assignments  & grades 

GPA 

Low  (3.00-3.69) 

None 

Medium  (3.70-3.99) 

Network  access 

High  (4.00) 

Network  access 

Learning  Achievement  and  Online 

The  final  course  grade  was  categorized  into  three  clusters:  low  grade  (2.19-3.46),  medium  grade  (3.47- 
3.82),  and  high  grade  (3.83-4.00)  with  each  cluster  having  similar  number  of  students..  The  total  number  and  the 
average  of  the  messages  that  requested  facilitation  in  each  cluster  of  the  final  course  grade  are  listed  in  Table  4. 
Table  4 shows  that  low-grade  and  high-grade  students  posted  an  average  of  1 message  per  person  while  medium 
grade  students  posted  an  average  of  1 .4  messages  per  person. 


Table  4.  Learning  Achievement  and  the  Amount  of  Online  Facilitation 


Final  Course  Grade 

Low  (2.19-3.46) 

Medium  (3.47-3.82) 

High  (3.83-4.00) 

Total  number  of  students 

10 

10 

9 

Frequency  of  requesting  facilitation 

10 

14 

9 

Average  Frequency  per  student 

1.00 

1.40 

1.00 

The  average  of  request  online  facilitation,  then,  was  calculated  under  each  student  characteristic  within 
each  cluster  of  the  final  course  grade.  The  results  showed  0 message  posted  by  non-experienced  students  in  all  grade 
clusters.  The  result  also  showed  similar  averages  of  messages  posted  by  experienced  students  in  all  grade  clusters, 
1.25,  1.56  and  1.29  messages  per  experienced  student  for  low,  medium,  and  high-grade  students,  respectively.  For 
female  students,  medium-grade  students  posted  the  most  (1.43)  messages  per  student;  and  for  male  students,  the 
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high-grade  students  posted  the  most  (1 .50)  messages  per  student.  For  younger  students,  high-grade  students  posted 
the  most  (1.50)  messages  per  student;  and  for  older  students,  medium-grade  students  posted  the  most  (2.00) 
messages  per  student.  For  Open  Distance  Learning  students,  mediumgrade  students  posted  the  most  (3.25) 
messages  per  student;  and  for  other  major  students;  low-grade  students  posted  the  most  (1 .67)  messages  per  student. 
The  results  showed  0 message  posted  by  low  GPA  students  in  all  grade  clusters.  For  medium  GPA  students,  low- 
grade  students  posted  the  most  (3.33)  messages;  and  for  high  GPA  students,  mediumgrade  students  posted  the  most 
(2.00)  messages. 

Overall,  The  result  showed  that  among  experienced,  female,  older.  Open  Distance  Learning  major,  and 
high  GPA  students,  mediumgrade  students  requested  more  facilitation  than  either  low  or  high-grade  students.  The 
result  also  showed  that  among  other  majors  and  medium  GPA  students,  low-grade  students  requested  more 
facilitation  than  either  medium  or  high-grade  students.  In  addition,  the  result  showed  that  among  male  and  younger 
students,  high-grade  students  requested  more  facilitation  than  either  low  or  mediumgrade  students.  Table  5 lists  the 
average  of  request  online  facilitation  by  student  characteristics  within  final  course  grade  clusters. 


Tahi^SJlhe  Average  of  Request  Online  Facilitation  by  Student  Characteristics  within  Grade  Clusters 


Grade 

FacilitatioiT 

Low 

Medium 

High 

(3.83-4.00) 

Student  Characteristic 

(2.19-3.46) 

(3.47-3.82) 

DL  Experience 

No 

0 

0 

0 

Yes 

1.25 

1.56 

1.29 

Gender 

Female 

1.14 

1.43 

.0.86 

Male 

0.67 

1.33 

1.50 

Age 

Younger  (19-35) 

1.14 

1 

1.50 

Older  (36  and  above) 

0.67 

2.00 

0.60 

Major 

Open  Distance 

Learning 

0 

3.25 

1 

Other  ma  jors 

1.67 

0.17 

1.00 

GPA 

Low  (3.0-3.69) 

0 

0 

0 

Medium  (3.70-3.99) 

3.33 

1.33 

1 

High  (4.00) 

0.00 

2.00 

1.20 

*DL  experience:  experience  in  distance  learning 


Discussion 


Types  of  Online  Facilitation 

Five  types  of  online  facilitation  were  identified.  They  were,  in  the  order  of  descending  frequency  of 
request:  1)  Assignments  and  grades  2)  Network  access  3)  Online  discussion  4)  Group  activities  5)  Other  course 
materials  access.  This  result  suggests  that  from  students’  viewpoints  the  most  desired  facilitation  is  assisting  them  to 
complete  the  assignments  and  to  obtain  high  grades.  The  second  most  needed  facilitation  is  the  assistance  in  network 
access  to  course  materials  and  participating  in  the  activities  in  the  course  site. 

This  finding  of  assignments  and  grades  as  the  most  needed  type  of  facilitation  shows  strong  implications  in 
web-based  course  design  and  teaching.  For  web-based  course  design,  instructional  system  designers  should  make 
the  description  of  assignments  and  grading  criteria  particularly  clear  so  that  it  can  thoroughly  communicate  with 
students  about  the  expectations  of  the  course.  For  web-based  course  teaching,  although  the  descriptions  of 
assignments  and  grading  criteria  may  be  clearly  stated  in  the  course  site,  students  may  overlook  them.  Instmctors 
should  instruct  students  to  the  location  of  specific  pages  or  even  demonstrate  to  students  how  to  access  to  this 
particular  information  containing  assignments  and  grading  criteria  in  the  course  site. 

The  implication  of  this  finding  concerning  network  access  as  the  second  most  requested  type  of  facilitation 
is  that  both  web-based  course  designers  and  instructors  should  take  students’  computer  literacy  into  consideration 
while  designing  and  teaching  web-based  courses.  Although  telecommunication  technology  is  very  promising  most 
of  the  time,  it  seems  also  very  discouraging  sometimes.  Network  disconnection  occurs  irregularly  if  not  frequently 
in  everyday  practice.  There  are  so  many  factors  that  may  affect  the  network  connection  and,  consequently,  may 
cause  communication  barriers  between  students  and  instructors.  Some  of  these  problems  may  not  be  controllable  or 
fixable  by  students.  Flexibility  of  schedule  should  be  built  within  web-based  course  design.  Due  dates  of 
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assignments  and  activities  should  not  be  too  rigid  in  order  to  allow  time  for  solving  unexpected  telecommunication 
problems.  Instructors  should  seek  prompt  support  from  the  system  support  team,  who  is  responsible  for  the  course 
delivery  system,  so  that  any  telecommunication  problems  can  be  solved  as  soon  as  possible. 

Response  Messages 

About  two  thirds  (64%)of  the  total  response  messages  was  posted  by  students,  while  the  instructor  together 
with  the  TA  provided  about  one  third  (37%).  This  finding  implied  that  peers  contribute  a great  deal  of  facilitation. 
The  willingness  of  providing  help  among  peers  in  this  web-based  learning  environment  appears  to  be  strong.  This 
result  suggests  that  interaction  among  students  plays  an  important  role  in  distance  learning  environments.  Peer 
interaction  should  be  included  as  an  essential  instructional  strategy  in  web-based  course  design.  Web-based  course 
instructors  should  build  rich  learning  communities  in  course  sites  and  encourage  students  to  actively  provide 
facilitation  to  peers. 

Student  Characteristics  and  Online  Facilitation 

The  most  important  finding  from  this  study  was  that  all  33  messages  that  request  for  online  facilitation 
were  posted  by  experienced  distance  students.  In  contrast,  no  message  was  posted  by  non-experienced  distance 
students.  The  reason  why  this  result  produces  such  a strong  contrast  may  be  because  experienced  students  know  how 
to  request  online  facilitation  whereas  non-experienced  students  may  not  know.  Although  it  is  speculated  that  non- 
experienced  students  may  need  and  request  more  facilitation  than  experienced  students  do,  the  data  seem  not  able  to 
support  such  a speculation.  The  data  can  only  be  explained  by  the  fact  that  non-experienced  students  have  problem 
learning  to  use  online  communication  tools  to  express  their  needs  for  online  facilitation.  The  online  tools  of 
communicating  via  the  course  site  are  new  to  non-experienced  students  and  they  may  be  less  comfortable  in  using 
these  new  tools. 

Although  this  finding  indirectly  supports  previous  findings  that  experienced  students  spend  more  time 
online  than  non-experienced  students  do  (Vaske,  1985;  Hiltz,  1993;  Grandgenett,  1996;  Althaus,  1997),  the 
hypothesis  that  non-experienced  students  will  request  more  online  facilitation  was  not  supported  in  this  study.  How 
to  reveal  the  need  for  online  facilitation  from  non-experienced  student  will  be  an  important  research  challenge  in  the 
future. 

The  average  frequency  of  requesting  facilitation  is  higher  in  females  (1.29  per  female)  than  in  males  (0.75 
per  male).  If  requesting  for  facilitation  is  an  indication  of  anxiety,  this  finding  indirectly  supports  previous  studies 
that  feminine-identity  students  were  found  to  appear  more  computer  anxious  than  masculine-identity  students  do 
(Rosen  & Sears,  1987).  Given  that  existing  studies  involving  gender  effects  in  computer-related  learning  have 
produced  inconsistent  findings,  this  finding  of  significant  gender  effects  should  be  considered  as  tentative.  Further 
research  in  examining  the  relationship  between  learner’s  gender  and  the  need  for  online  facilitation  is  still  needed. 

The  average  number  of  messages  posted  by  older  students  (1.70  per  student)  to  request  for  online 
facilitation  was  higher  than  those  posted  by  younger  students  (0.76  per  student).  This  finding  supports  some  studies 
that  older  students  are  found  to  be  more  computer  anxious  than  younger  students  (Rosen,  Sears,  & Weil,  1987;  Jones 
& Wall,  1989). 

The  average  number  of  messages  posted  by  Open  Distance  Learning  major  students  (2.07  per  student)  was 
higher  than  other  major  students  (0.27  per  student).  This  finding  indirectly  supports  a previous  result  that  computer 
anxiety  consistently  correlated  with  students’  majors  (Rosen,  Sears,  & Weil,  1987).  Once  again  that  further  research 
is  desired  for  directly  addressing  the  relationship  between  student’s  major  and  the  need  of  online  facilitation. 

The  average  number  of  messages  that  requesting  online  facilitation  posted  by  high  and  medium  GPA 
students  (1 .45  per  high  GPA  student,  1 .89  per  medium  GPA  student)  were  higher  than  low  GPA  students  (0.00  per 
low  GPA  students).  Low  GPA  students  did  not  even  request  any  online  facilitation  at  all.  They  seem  either  not  know 
how  to  request  facilitation  or  have  low  learner  motivation  to  solve  problems  that  are  pointed  out  in  the  online 
messages.  This  finding  may  suggest  that  the  learner  motivational  factors  should  be  built  in  both  web-based  course 
design  and  teaching.  If  the  web-based  course  design  can  gain  and  sustain  students’  attention  to  the  focus  of  the 
course  contents,  enhance  relevance  between  course  contents  and  the  learners’  concerns,  build  students’  confidence 
about  themselves,  and  generate  students’  satisfaction  about  the  course,  students  may  be  more  motivated  to  solve 
problems  and  to  request  necessary  facilitation. 

The  Most  Needed  Facilitation  Type  wi  thin  Each  Student  Characteristic 
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The  finding  indicated,  for  each  student  characteristic,  the  most  desired  facilitation  type  was  either  network 
access  or  information  of  assignments  and  grades.  Further  data  analysis  reveals  that,  on  one  hand,  female,  younger, 
other  majors,  non-experienced,  and  medium  and  high  GPA  students  requested  facilitation  in  assignments  and  grades 
the  most.  On  the  other  hand,  male,  older.  Open  Distance  Major,  and  distance  learning  experienced  students  needed 
facilitation  in  network  access  the  most.  Female,  younger,  other  majors,  non-experienced,  and  medium  and  high  GPA 
students  may  be  less  sure  about  their  learning  achievement,  so  they  are  more  concerned  about  assignments  and 
grades.  On  the  contrary,  male,  older,  Open  Distance  Major,  and  distance  learning  experienced  students  have  more 
confidence  on  their  online  skills  but  may  need  more  help  in  fully  utilizing  online  resources.  Such  a difference  in 
facilitation  preference  between  student  characteristics  deserves  further  study  to  provide  more  explanation. 

With  respect  to  grades,  this  study  found  that  low  GPA  students  did  not  even  request  any  facilitation.  The 
possible  explanation  is  that  low  GPA  students  may  not  have  enough  motivation  to  learn. 

Overall,  the  implication  of  the  above  findings  is  that,  from  students’  viewpoints,  assignments,  grades,  and 
network  access  should  be  the  central  foci  of  web-based  courses.  Expectations  of  assignments,  grading  criteria,  and 
the  guidelines  of  course  site  access  should  be  well  stated  in  course  sites  and  thoroughly  communicated  with  students 
by  the  instructor. 

Learning  Achievement  and  Online  Facilitation 

The  result  showed  that  among  experienced,  female,  older.  Open  Distance  Learning  major,  and  high  GPA 
students,  medium-grade  students  requested  more  facilitation  than  either  low  or  high-grade  students.  It  implies  that  if 
low-grade  students  had  requested  sufficient  facilitation,  they  may  achieve  better  grades.  High-grade  students  may 
already  have  the  ability  to  learn  by  themselves  without  much  facilitation. 

The  result  also  showed  that  among  other  majors  and  medium  GPA  students,  low-grade  students  requested 
more  facilitation  than  either  medium  or  high-grade  students.  This  finding  indicates  that  other  majors  and  medium 
GPA  students  have  experienced  learning  difficulties  in  the  course.  It  not  only  suggests  that  web-based  instructors 
should  provide  more  facilitation  to  other  majors  and  medium  GPA  students,  but  also  propose  a further  research 
question  - what  kind  of  difficulties  that  other  majors  and  medium  GPA  student  may  have  experienced  and  how  to 
assist  them  for  better  learning  experiences. 

In  addition,  the  result  showed  that  among  male  and  younger  students,  high-grade  students  requested  more 
facilitation  than  either  low  or  medium-grade  students.  High-grade  students  may  have  stronger  motivation  to  solve 
problems  and  concern  more  about  learning  achievement  so  they  asked  for  more  facilitation  than  low  or  medium 
grade  students  do. 

Conclusion 

This  study  identifies  five  types  of  online  facilitation  requested  by  students . Among  them,  the  two  most 
requested  types  of  online  facilitation  are  1)  assignments  and  grades  2)  network  access.  This  study  also  makes 
suggestions  for  web-based  course  design  and  teaching.  The  following  summarizes  these  suggestions: 

1 . Make  due  dates  of  assignments  and  activities  flexible  and  explicitly  clear 

2.  Thoroughly  communicate  with  students  about  the  expectations  of  assignments  and  grading  criteria 

3.  Seek  immediate  supports  from  system  support  team  to  solve  problems  in  the  event  that  network 
access  problems  occur 

4.  Design  instructional  strategies  emphasizing  peer  interaction  in  web-based  courses  and  encourage 
peer  interactions  in  teaching  web-based  courses 

5.  Employ  learner  motivational  strategies  in  order  to  encourage  low  GPA  students  and  non- 
experienced  students  to  request  facilitation  online 

The  major  limitation  of  this  study  is  the  small  size  of  the  research  sample.  There  are  29  students 
participated  in  this  study.  With  such  a small  number  of  participants  and  3 to  5 categories  in  each  variable  to  be 
analyzed,  it  is  very  difficult  to  employ  statistical  tests  that  will  show  meaningful  results.  Therefore,  the  findings 
from  this  study  should  be  considered  as  tentative,  and  generalization  from  the  conclusions  of  this  study  should  be 
very  limited. 

A repetition  of  this  study  with  larger  sample  size  is  recommended.  With  a large  number  of  participants, 
sufficient  data  can  be  collected  for  statistical  analysis  in  order  to  generalize  meaningful  conclusions. 

More  variables  should  also  be  incorporated  to  develop  the  concept  of  facilitation  and  better  understand  the 
nature  of  facilitation.  It  is  still  unclear  that  lack  of  online  skills  needs  more  facilitation  or  is  just  not  able  to  post 
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request  for  facilitation.  It  is  also  uncertain  about  the  relationship  between  conputer  anxiety  and  request  for 
facilitation. 

Overall,  this  study  identifies  five  types  of  online  facilitation  that  students  requested.  Further  research  may 
be  conducted  to  further  verify  these  five  types  of  online  facilitation  and  to  explore  their  relationships  with  learner 
characteristics  and  learning  achievement. 
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Abstract 

The  main  purpose  of  this  Naturalistic  inquiry  study  is  to  explore  four  ethnically  diverse  Web-based 
Computer  Supported  Collaborative  Learning  (CSCL)  students'  perceptions  of  and  experiences  in 
knowledge  construction  and  group  collaboration.  Findings  suggest  that  individual  and  group  successes 
are  interconnected  and  rely  on  successful  negotiation  and  construction  of  shared  knowledge  among  group 
members.  While  effective  collaboration  and  knowledge  construction  among  people  of  diverse  backgrounds 
may  be  challenging,  Web-based  collaborative  learning  among  similar  populations  may  pose  additional 
challenges  than  in  face-to-face  settings  due  to  the  absence  or  limited  non-verbal  communication  cues. 
Educators  interested  in  designing  online  courses  for  students  of  diverse  backgrounds  should  not  only 
consider  the  cognitive  but  also  the  social  and  cultural  aspects  of  communication.  Future  studies  should 
focus  on  exploring  effective  course  design  to  facilitate  both  cognitive  and  social  aspects  of  learning  and  to 
optimize  individual  as  well  as  group  learning. 

Introduction 

Assisted  by  technological  advances,  Web-based  instruction  is  widely  employed  in  preparing  learners  for 
“future  responsibilities”  and  “success  in  life”  (Dewey,  1938).  To  meet  individual  learner’s  needs  while  optimizing 
their  strengths  and  talents,  Confucius  said,  instruction  should  be  tailored  to  meet  individual  learner’s  needs. 

A plethora  of  distance-learning  research  studies  have  been  conducted  and  Conputer  Supported 
Collaborative  Learning  (CSCL)  has  been  one  of  the  major  focuses.  However,  most  CSCL  studies  have  addressed 
issues  regarding  tools,  design,  course  impact,  and  evaluations  from  instructors’,  administrators’,  or  instructional 
designers’  observations.  Few  of  these  studies,  however,  have  addressed  these  issues  through  the  lenses  of  students.  I 
became  aware  of  the  importance  of  studying  human  interaction  (as  opposed  to  focusing  solely  on  technical 
functionality)  from  participating  in  the  design  of  the  CSCL  course. 

Conducted  within  the  constructivist  paradigm,  this  research  focused  on  four  ethnically  diverse  students’ 
perceptions  of  and  experiences  in  a computer-supported  collaborative  learning  class  where  a major  aspect  of  the 
course  design  was  based  on  constructivist  theory.  The  purpose  of  the  study  is  to  explore  students’  perspectives  on 
group  collaboration  and  knowledge  construction  in  a CSCL  class  offered  in  Fall  2000  at  University  of  Texas. 

Naturalistic  inquiry  was  employed  as  the  research  strategy  for  this  study.  Egon  Cuba  referred  to  this 
inquiry  method  as  the  “constructivist  inquiry.  As  described  by  Erlandson  et  al.(1993),  naturalistic  paradigm 
assumes  that  there  are  “multiple  realities;”  affirms  the  “mutual  influence  that  researcher  and  respondents  have  on 
each  other;”  and  assumes  that  “total  generalization  is  never  possible.” 

In  order  to  avoid  framing  interviewees,  this  study  was  guided  solely  by  one  focus  question,  “How  did  you 
perceive  the  group  collaboration  and  knowledge  construction  work  in  your  CSCL  2000  class?”  Group  collaboration 
and  knowledge  construction  were  focused  upon  because  they  were  the  two  essential  course  objectives. 

Study  Context 

CSCL  is  derived  from  CSCW  (Computer  Supported  Collaborative  Work),  which  was  named  after  “Office 
Automation”  (a  system  that  aimed  at  facilitating  businesses  for  efficient  and  effective  work).  The  main  differences 
between  CSCL  and  CSCW  are  their  contexts  and  purposes.  CSCW  focuses  on  getting  work  done  and  is  mainly 
employed  in  business  settings  where  sectors,  departments,  and  companies  collaborate  to  complete  projects  and  make 
decisions.  CSCL  focuses  on  learning  and  is  employed  in  educational  settings  where  collaborators  in  the  learning 
community  construct  knowledge  and  accomplish  joint  projects.  Resta,  et  al.  (1999)  said  CSCL  maybe  used  as  the 
“catalyst  for  changing  teacher  practice”  as  teachers  are  “continually  revising  their  curriculum  design”  based  on  their 
experiences  and  “emergent  instructional  needs”  (p.  492).  Such  curriculum,  rather  than  relying  on  a set  plan  to 
achieve  predetermined  outcomes,  provides  a platform  for  discourse  and  interaction  between  and  among  students, 
peers,  and  course-content  experts.  The  CSCL  curriculum  enables  knowledge  construction  and  problem-solving 
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among  collaborators  of  diverse  backgrounds.  The  instructors’  role  is  transformed  from  that  of  a “sage  on  the  stage,” 
to  that  of  a “guide  on  the  side.” 

A major  goal  of  this  CSCL  2000  course  was  to  assist  learners  to  understand,  create,  and  reflect  through  the 
engagement  in  projects  similar  to  real-life.  A variety  of  instructional  material,  resource  links,  and  task  instructions 
were  available  on  the  Web  via  WebCT  courseware.  The  course  content  was  divided  into  seven  modules,  a course 
handbook,  a course  tool  page,  and  a resource  link  page  (Graph  #1).  Each  module  contained  tasks  requiring  learners 
to  work  both  individually  and  collaboratively  in  completing  tasks.  Collaborative  tasks  included  writing  a topic 
paper,  designing  a tour  in  a MOO  (Multi-user  Object  Oriented)  virtual  environment  where  users  log  onto  a site  to 
experience  a text  -based  virtual  reality  environment,  designing  a WebQuest  and  working  collaboratively  with  cross- 
team members  to  develop  a final  project  utilizing  a schedule  planning  tool  for  coordination  (jointplanning.com).  A 
WebQuest  is  an  inquiry  activity  that  is  based  on  realistic  and  engaging  tasks  and  the  resources  drawn  by  learners  are 
from  the  Web.  Its  focus  is  on  “using  information  rather  than  looking  for  it,  and  to  support  learners'  thinking  at  the 
levels  of  analysis,  synthesis  and  evaluation”(Dodge,  Bemie). 


Graph  # 1;  Course  Web  Site 
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Module  One  provided  an  overview  of  the  course  goals,  objectives,  required  entry  skills,  technology 
requirements,  course  activity  schedule,  and  other  information  helpful  to  the  students  in  preparing  to  complete  the 
course.  Module  Two  provided  opportunities  for  online  socialization  through  introduction  of  the  course  mission,  the 
online  environment,  and  technological  tools,  while  encouraging  students  to  socialize  with  peers  through  a class-wide 
introduction  activity.  Modules  Three,  Four,  and  Five  enabled  students  to  exchange  information  and  construct 
knowledge  through  online  communication,  search  and  exchange  information,  perform  assigned  tasks,  and  provided 
mutual  support  and  cooperation  among  online  team  members.  Students  worked  collaboratively  to  navigate  and 
explore  various  network  environments,  utilizing  collaborative  tools  to  plan  group  projects,  schedule  meetings, 
negotiate  tasks,  develop  ideas  and  concepts,  make  decisions,  and  edit  finished  projects  as  a group.  Participants 
engaged  in  building  a collaborative  team,  utilizing  collaborative  writing  strategies,  exploring  synchronous  online 
collaborative  learning,  and  inquiring  skills  to  develop  a WebQuest,  a Web-based  activity  that  involves  searching, 
reading,  analyzing,  and  dialoging  with  peers  to  solve  problems  and  create  Web  pages.  The  course  handbook  offered 
a virtual  office  tour,  an  organization  chart  about  this  virtual  company,  a directory  of  both  students  and  staff,  tips  on 
working  collaboratively,  topics  for  collaborative  work,  and  project  examples. 

To  accomplish  course  requirements,  online  socialization  and  communication  were  essential.  Extensive 
cooperation  and  collaboration  among  learners  was  necessary.  A course  discussion  virtual  workspace  (Graph  #2) 
offered  students  the  opportunity  to  exchange  information,  discuss  tasks,  upload  files,  work  collaboratively,  and 
socialize  both  asynchronously  and  synchronously.  CSCL  Technology  Company  was  employed  as  a metaphor  to 
indicate  a real  world  professional  setting.  Members  of  the  class  were  divided  into  five  virtual  teams  located  in 
suites.  There  are  two  to  three  offices  within  each  suite.  Two  to  three  students  shared  the  same  office  and  about  six 
to  seven  students  shared  the  same  suite.  Diversity  (ethnic  background,  gender,  and  on-campus  or  tele -campus 
access)  was  considered  when  assigning  students  their  suites  at  the  beginning  of  the  semester. 
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There  were  five  Webcasts  (video-conferences)  throughout  the  semester.  On -campus  students  had  the 
opportunity  to  attend  on  campus  while  tele  -campus  students  attended  through  the  Web.  Additional  modes  of 
participation  included  telephone  call-in  and  synchronous  chat. 

The  product  of  the  course  was  the  accumulated  contributions  from  learners  through  an  emphasis  on 
collaborative  learning.  Learners  engaged  in  a variety  of  projects  in  order  to  understand  the  CSCL  environment, 
create  projects  utilizing  tools  for  collaborative  learning,  and  reflect  their  own  learning  through  personal  journaling. 
Hence,  course  success  resided  completely  on  the  students’  success  in  collaboration. 

Method 

Naturalistic  Inquiry 

Naturalistic  inquiry  was  employed  as  the  research  strategy  for  this  study.  Egon  Cuba  (1981)  referred  this 
as  the  “constructivist  inquiry.”  Cuba  pointed  out  that  the  “naturalistic  inquiry  is  not  equivalent  to  qualitative 
inquiry.”  Naturalistic  inquiry  utilizes  “human  instrument”  - the  researcher,  with  findings  “created”  through  the 
interaction  between  the  researcher(s)  and  participants  (Erlandson,  et.al.,  1993  p.  ix-xv).  The  naturalistic  paradigm: 

1 . Assumes  that  there  are  “multiple  realities”  - While  the  Positivist  view  holds  that  there  is  a 
single  objective  reality,  the  naturalistic  view  assumes  that  “all  the  ‘parts’  of  reality  are  interrelated. 

2.  Affirms  the  “mutual  influence  that  researcher  and  respondents  have  on  each  other”  - 
Naturalistic  theorists  asserts  that  “mutual  simultaneous  shaping”  (the  interaction  between 
researchers  and  respondents)  is  unavoidable  and  the  researcher  must  find  ways  to  “control  the 
biases  that  do  not  inhibit  the  flow  of  pertinent  information  (p.  15). 

3.  Assumes  that  “total  generalization  is  never  possible”-  Unlike  most  scientific  studies  which 
seek  to  generalize  findings,  naturalistic  inquiry  seeks  to  include  “thick  description  of  one  set  of 
interrelationships  in  one  social  context”  to  allow  “transfer  of  understanding”  for  readers  across 
social  contexts  (Cuba,  1981). 

Cuba,  (1981)  said  that  quantitative  and  qualitative  methods  can  both  be  used  in  a naturalistic  paradigm  but 
qualitative  research  method  is  “generally  preferred.”  He  said  that  while  “relevance”  and  “rigor”  are  both  important, 
“relevance”  is  more  important  in  the  naturalistic  paradigm.  In  the  naturalistic  paradigm,  “emergent  theory”  is 
preferred,”  according  to  Cuba.  While  all  researchers  use  a variety  of  instruments  to  obtain  data,  in  the  naturalistic 
research,  the  “primary  research  instrument  is  the  researcher”  and  is  conducted  in  a “natural  setting”  rather  than  a 
“laboratory  or  controlled  setting.” 

Constructivism 

The  philosophy  behind  the  design  of  the  CSCL  course  is  Constructivism.  The  philosophy  behind  the 
design  of  the  current  naturalistic  study  is  also  Constructivism.  Fosnot  (1996)  said  Constructivism  is  a theory  about 
knowledge  and  learning  that  explores  both  “what  knowing  is  and  how  one  comes  to  know.”  This  theory,  he  said, 
describes  knowledge  as  “temporary,  developmental,  nonobjective,  internally  constructed,  and  socially  and  culturally 
mediated.”  Constructivists  view  learning  as  a self-regulatory  process  in  which  learners  constantly  make  meanings 
to  construct  and  gain  new  knowledge  and  insights. 

Jonassen  (1994)  identified  the  following  elements  of  constructivist  design:  (1)  multiple  representations  of 
reality,  (2)  representing  the  complexity  of  the  real  world,  (3)  emphasizing  on  knowledge  construction  instead  of 
knowledge  reproduction,  (4)  emphasizing  authentic  tasks  in  a meaningful  context  rather  than  abstract  instruction  out 
of  context,  (5)  preferring  real  world  settings  to  predetermined  sequences  of  instruction,  (6)  encouraging  thoughtful 
reflection  on  experience,  (7)  knowledge  construction  based  on  both  content  and  context,  (8)  knowledge  construction 
arrived  at  collaboratively  rather  than  competitively. 

As  Duffy  and  Jonassen  (1992)  indicated,  “Constructivism  provided  a very  important  vehicle  for 
establishing  the  dialogue  ...  the  information  age  and  the  technological  capabilities  have  caused  us  to  re- 
conceptualize  the  learning  process  and  to  design  new  instructional  approaches.”  (p.  ix)  Constructivists  believe  that 
“there  are  many  ways  to  structure  the  world,  and  there  are  many  meanings  or  perspectives  for  any  event  or  concept. 
Thus  there  is  not  a correct  meaning  that  we  are  striving  for."  (p.  3) 

The  study  of  human  settings  and  interactions  is  a complex  task.  No  two  humans  hold  identical  realities  or 
experiences.  It  is  my  purpose  to  better  understand  what  CSCL  students  have  encountered,  how  they  reacted  or 
responded  to  certain  situations,  how  they  interacted  with  each  other,  what  they  perceived  as  worth  knowing  and 
learning,  what  particular  incidences  they  perceived  as  worth  discussing,  and  any  other  reflections  they  have  had. 
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Participants 


Purposeful  sampling  method  to  achieve  as  maximum  a variation  as  possible  within  the  study  population 
was  employeded  in  recruiting  participants.  As  described  by  Merriam  (1998),  purposeful  sampling  is  based  on  the 
assumption  that  the  researcher  wants  to  discover,  understand  and  gain  insight  from  participants.  Therefore,  selecting 
a sample  that  researcher  can  learn  the  most  from  is  important  (p.  48). 

Study  participants  were  recruited  on  a volunteer  basis.  This  study  focused  on  four  female  graduate  students 
including  a 47 -year-old  Caucasian,  a 24-year-old  Asian,  a 36 -year-old  Hispanic  American,  and  a 32-year-old  Indian 
who  were  all  enrolled  in  a graduate  Web-based  CSCL  (Computer  Supported  Collaborative  Learning)  course  offered 
in  Fall  2000  at  the  University  of  Texas  (UT)  College  of  Education.  These  participants,  registered  through  UT-Austin 
(on-campus  students)  and  UT-Brownsville  (Tele -campus  students),  participated  in  two  interview  sessions  and 
numerous  e-mail  communications  with  the  researcher  from  mid-October  to  mid-December,  2000. 

Data  Collection 

For  data  triangulation,  three  data  sources  were  obtained:  transcripts  of  interviews;  participants'  concept 
maps  and  explanations;  online  course  data;  and  transcripts  of  follow-up  telephone  or  e-mail  communications. 
Additional  data  were  derived  from  the  researcher's  observation  notes  and  the  reflective  journal.  Primary  data 
sources  include  interview  transcripts,  participants'  explanations  of  their  concept  maps,  and  documents  obtained 
directly  from  participants.  Secondary  data  include  online  course  data  chosen  by  participants,  researcher's  transcript 
summaries,  and  researchers'  observation  notes  and  research  journal. 

Six  students  were  initially  requested  to  participate.  Four  students  among  the  eight  agreed  to  participate. 
These  participants  represented  divergent  levels  of  commitment  in  class  as  measured  by  students’  online  interaction 
and  attendance  from  September  1®^  to  October  1**2000.  Informed  consent  forms  were  sent  to  participants  via  e-mail 
with  the  request  that  they  read  the  form  carefully.  The  consent  form  that  explained  the  purpose  of  the  study, 
participation  procedures  and  methods,  study  timeline,  benefits  and  any  known  risks  of  participation,  and  estimated 
time  involved  in  the  two  interview  sessions  and  e-mail  communications.  The  researcher  collected  and  securely  filed 
these  forms  for  future  reference. 

Prior  to  the  first  interview,  participants  were  provided  the  study  focus  question:  “How  did  you  perceive  the 
group  collaboration  and  knowledge  construction  work  in  your  CSCL  2000  class?”  The  participants  were  ensured  of 
their  anonymity  in  the  study  and  the  right  to  withdraw  at  any  time  during  the  study;  participants  were  continuously 
asked  if  they  would  like  to  add,  delete,  or  make  comments. 

Participants  were  interviewed  separately.  To  ensure  that  participants’  stories  remained  the  sole  focus,  I 
explained  the  basis  of  naturalistic  inquiry  prior  to  the  interview,  reminded  them  that  their  perceptions  and 
experiences  were  the  sole  focus  of  my  study,  and  assured  them  that  they  could  express  themselves  freely.  To  assure 
that  my  understanding  matched  what  they  had  hoped  to  express,  I asked  for  clarifications  and  explanations.  I asked 
them  to  expand  and  provide  examples  based  solely  in  response  to  what  they  had  said  and  refrained  from  framing 
their  thoughts  by  using  any  pre -determined  questions.  Throughout  the  interview,  I also  summarized  what  they  have 
said.  This  emergent  interview  method  shared  certain  similarities  with  the  active  interview  process  described  by 
Holsten  and  Gobrium  (1995). 

To  further  supplement  my  understanding,  I sometimes  drew  pictures  based  on  what  participants  were 
saying  and  used  to  check  for  clarity  and  understanding.  Holsten  and  Gobrium  discussed  a goal  of  “creative 
interviewing”  techniques  as  being  “a  set  of  techniques  for  moving  past  the  mere  words  and  sentences  exchanged  in 
the  interview  process.”  I made  sure  that  the  participants  could  “share  their  own  thoughts  and  feelings”  not  only 
through  words,  but  also  symbols  (p.  12). 

Data  Analysis 

Naturalistic  inquiry  utilizes  the  researcher  as  “human  instrument.”  The  benefit  of  this  is  that  data  can  be 
collected  and  analyzed  non-linearly  and  interactively.  In  this  study,  data  analysis  occurred  even  during  the 
interview.  Two  interviews  per  informant  were  conducted  at  agreed-upon  locations,  approximately  2-3  weeks  apart. 
Each  interview  took  approximately  one  hour.  The  interviews  were  audio  -taped,  transcribed,  and  organized  into 
summaries.  At  first  interview,  participants  were  presented  a printed  version  of  their  class  online  messages  and  were 
asked  to  select  messages  of  their  choice  and  explain  the  significance.  During  the  first  interview,  participants 
expressed  their  perceptions  and  experiences  based  on  the  focus  question  and  probing  questions  for  explanation, 
expansion;  examples  were  drawn  from  participants’  responses.  At  first  interview,  each  participant  was  asked  to 
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draw  a concept  map  based  on  the  focus  question  and  to  present  this  drawing  at  the  second  interview.  The  first 
interview  summary  was  sent  to  participants  prior  to  the  second  interview. 

Interview  tapes  were  first  transcribed  word-for-word.  They  were  then  unitized  and  coded  from  either 
words  or  phrases  used  directly  by  participants  or  from  representative  words  for  the  unit  information.  Two  types  of 
coding  methods  were  utilized  for  each  interview  transcript.  The  first  method  involved  detailed  frequency  counting 
of  particular  ideas  and  subsequent  grouping  of  these  ideas  into  sub-themes.  The  second  coding  method  was  based 
on  concepts  that  emerged.  These  two  codes  were  then  compared  to  discern  cross-participants’  shared  themes.  Six 
shared  themes  emerged  and  were  defined  based  on  the  context. 


Trustworthiness  & Authenticity 

Lincoln  and  Cuba  (1985)  refer  to  trustworthiness  and  authenticity  as  two  essential  sets  of  criteria  in 
constructivist  research.  Trustworthiness  (the  truth -value,  or  quality  of  the  findings)  refers  to  the  validity  and 
reliability  of  research  issues.  It  ensures  the  research  methods  are  correctly  conducted.  Authenticity  refers  to 
fairness,  educative,  and  resulting  actions  arising  from  the  research  (p.  213). 

Various  methods  are  often  employed  to  ensure  “one  has  carried  out  the  research  process  correctly” 
(Manning,  1997)  and  increase  “the  degree  of  confidence  in  truth”  (Erlandson  et  al.,  1993).  The  methods  I employed 
to  ensure  trustworthiness  included:  writing  a “researcher  as”  statement,  purposefully  sampling  study  participants, 
engaging  with  participants  for  an  extensive  period  of  time,  obtaining  more  than  one  data  source  for  data 
triangulation,  forming  a peer-debriefing  group,  and  obtaining  verifications  of  participants’  interpretations. 

To  ensure  authenticity  in  my  study,  informed  consent  was  provided  to  participants  prior  to  the  first 
interview.  In  the  consent  form,  participants  were  informed  of  the  risks  of  and  benefits  from  participation  in  the 
study,  as  well  as  their  right  to  withdraw  at  any  time.  Prior  to  the  interview,  I explained  to  participants  the  nature  of 
the  study  and  their  rights  to  freely  express  any  opinions  and  concerns  they  may  have. 

Prior  to  conducting  this  study,  I composed  a “Researcher-as-Instrument  Statement”  that  reflected  upon  the 
possible  preconceptions  or  experiences  that  may  possibly  have  influenced  my  conduct  of  this  research.  In  this 
statement  I stated  that  I was  aware  of  my  past  notions  regarding  group  work,  my  personality  that  may  unconsciously 
influence  my  expectations  and  reactions,  and  my  experiences  with  online  learning  that  may  be  potentially  create 
bias.  Nevertheless,  I stated  that  I expected  to  remain  open  and  flexible  to  reassessment  of  any  assumptions  that  may 
arise  throughout  the  inquiry  process.  Cognizant  of  my  interpretations  that  create  sense  of  my  world,  I was  equally 
cognizant  of  my  duty  to  refrain  from  using  words  to  “frame”  participants  during  the  entire  research  process.  I 
stressed  my  research  position  to  informants  prior  to  interview.  As  a sherpa,  I had  the  opportunity  to  extensively 
observe  online  activities,  participate  in  a few  group  chats,  and  study  the  same  course  materials  as  all  CSCL 
members.  I observed  and  experienced  participants’  frustrations,  confusions,  stresses,  and  joys. 

As  suggested  by  Erlandson  (1993),  “The  researcher  should  step  out  of  the  context  being  studied  to  review 
perceptions,  insights,  and  analyses”  (p.  31).  Prior  to  forming  my  focus  question,  a peer-debriefing  group  was 
formed.  The  group  composed  of  three  peers  who  were  knowledgeable  of  naturalistic  inquiry  and  who  shared  similar 
interests  in  topics  of  inquiry.  They  helped  me  reconsider,  refine,  and  revise  my  study,  which  added  to  the  credibility 
of  this  study.  We  also  read  each  other’s  transcripts,  assisted  in  evaluating  codes  and  themes  during  the  data  analysis 
process,  read  the  draft  version  of  the  study,  and  provided  feedback  for  revision  prior  to  completion  of  the  study. 

With  their  assistance,  some  decisions  and  actions  were  altered  through  insights  gained.  We  provide  each  other  with 
both  task  and  emotional  support.  Often  acting  as  devil’s  advocates,  we  met  weekly  to  challenge  and  assist  each 
other.  Detailed  minutes  of  the  meetings  were  kept  and  served  as  a resource  to  be  referred  to  during  our  inquiry 
processes. 

To  ensure  that  my  understandings  and  interpretations  were  not  biased,  three  levels  of  member  checking 
were  conducted  throughout  the  inquiry  process.  The  first  member  check  was  conducted  during  the  interview  process 
when  participants  were  asked  to  clarify,  explain,  expand,  and  give  examples.  The  second  member  check  was 
conducted  within  a few  days  of  each  interview  when  participants  were  ask  to  verify  the  summaries  of  interviews 
transcripts.  The  grand  member  check  was  conducted  after  completion  of  case  reports  but  prior  to  submission  when 
participants  were  asked  to  confirm  the  final  product.  In  addition,  member  checks  were  also  conducted  through 
informal  conversations  over  the  telephone  and  through  e-mail.  During  the  entire  inquiry  process,  I also  maintained  a 
reflexive  journal  to  record  decisions  made,  reasons  for  those  decisions,  actions  taken,  questions  that  arose  during  the 
process,  possible  emerging  patterns  of  analysis,  and  reactions  to  particular  situations. 

Findings 
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To  ensure  confidentiality,  names  and  locations  are  changed  to  preserve  the  anonymity  of  the  participants. 
The  social  relationships  documented  within  remain  true  to  life.  The  following  cases  are  based  on  the  data  obtained 
from  participants  in  Fall,  2000. 

Participant  information 

Agnes,  a 47  year-old  Caucasian  doctoral  student,  works  part-time  at  a computer  company.  She  was  bom 
and  raised  in  America  and  has  a theater  and  art  background.  She  received  her  Master’s  degree  in  England. 

Although  Agnes  had  a few  small -scaled  online  collaborative  learning  experiences  prior  to  entering  CSCL  2000 
class,  she  had  never  collaborated  online. 

Nancy,  a 24  year-old  Taiwanese  master’s  student,  works  full-time  at  a primary  school  in  Austin.  She  was 
bom  in  Taiwan,  but  was  raised  in  South  America.  She  moved  to  America  alone  when  she  entered  college.  She  is 
interested  in  Computer-Based  Training  and  Assistive-Technology.  She  had  one  online  collaborative  learning 
experience  prior  to  entering  CSCL  2000  class  but  she  thought  it  was  a lot  less  intensive  than  this  CSCL  class. 

Elizabeth,  a 36  year-old  Hispanic -American  master’s  student,  works  full-time  at  a primary  school  in  Austin 
as  an  Assistant  Principal.  She  was  bom  in  Columbia  and  was  raised  in  America.  She  is  married  with  a kid  and  is 
expecting  a baby  in  6 months.  She  had  some  pervious  online  learning  experience  prior  to  entering  CSCL  2000  class 
but  have  never  collaborated  online  with  a group  of  people.  She  said  she  has  lots  of  opportunities  to  collaborate  with 
colleagues  in  the  face-to-face  settings  at  work. 

Angel,  a 32  year-old  Indian  master’s  student,  studies  full-time  at  the  University  of  Texas  (U.T.),  Austin. 

Fall  2000  is  the  first  semester  for  her  at  U.T.  and  in  America.  She  already  received  her  Economics  master’s  degree 
in  India  prior  to  entering  U.T.  and  had  worked  as  a high  school  teacher.  She  had  no  previous  online  learning 
experience  and  not  much  collaborative  learning  experience. 

Emerged  Themes  Across  Case 

Six  themes  emerged  through  the  process  of  data  generation,  analysis  and  comparison  of  participants: 
relationships;  environment;  communication;  roles  and  identities;  feelings;  and  personal  variables.  Emerged  themes 
were  listed  in  the  order  of  importance  as  defined  by  the  frequency  being  discussed. 

Relationships 

Relationship  received  the  most  attention  among  six  emerged  themes.  For  Agnes,  group  collaboration  is  a 
“process.”  Agnes’  group  had  six  people.  “You  are  looking  for  connections  with  people,”  said  Agnes.  She  thought 
that  building  relationships  online  was  different  because,  “connection  is  often  established  one  way.”  Agnes 
conceived  that  “interpersonal  relationships”  were  important  in  the  dynamics  among  her  knowledge  construction, 
group  collaboration,  and  self-identity  (as  illustrated  in  her  concept  map.  Graph  #3). 

Similar  to  Agnes,  Elizabeth  also  emphasized  much  on  relationships.  During  the  interviews,  Elizabeth 
repeatedly  emphasized  the  importance  of  “being  responsible  for  others.”  She  said  she  did  not  want  to  “let  down” 
her  group  mates.  She  thought  if  a person  does  not  have  “strong  sense  of  responsibility  for  others”  s/he  would  not 
succeed  in  class.  Elizabeth  thought  it  is  important  to  build  relationships  and  friendships  in  a collaborative  learning 
class  where  members  are  held  accountable  for  each  other  and  they  had  to  rely  on  each  other  in  accomplishing  tasks. 
She  described  her  excitement  at  the  beginning  of  the  semester  when  she  first  “met”  her  virtual  officemate  online. 
“Every  time  I get  online,  I would  check  to  see  if  she  is  there,”  said  Elizabeth.  She  said  the  friendship  she  established 
with  her  officemate  would  last  beyond  the  class. 

Nancy  and  Angel  also  mentioned  about  relationships  but  not  emphasized  as  much  as  Agnes  and  Elizabeth. 
Angel  thought  everyone  has  different  needs  for  relationships.  She  thought  that  in  a collaborative  learning  course 
like  CSCL,  learners  are  seeking  both  intellectual  as  well  as  emotional  connections,  however,  when  there  are  too 
many  trivial  tasks,  students  tend  to  just  focusing  on  getting  the  tasks  done  rather  than  learning  socio -emotional 
aspects  of  learning.  Angel  said  that  she  was  too  busy  trying  to  learn  different  tools  and  getting  use  to  a new  learning 
environment,  she  did  not  spend  much  time  socializing  with  others. 

Similar  to  Angel,  Nancy  did  not  perceive  interpersonal  relationships  as  important  for  her  success  in  class. 
“The  tasks  are  short,  and  they  have  no  implications  for  the  future,”  said  Nancy.  She  said  the  relationship  in  class  is 
temporal  rather  than  long-term.  Nancy  thought  interactions  in  temporal  relationships  are  usually  superficial  rather 
than  deeper  understanding  of  each  other.  She  thought  that  people  usually  react  to  things  differently  if  they  know 
they  are  going  to  have  the  work  relationship  for  only  a few  months  (a  semester). 
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Communication 


The  second  most  discussed  theme  is  communication.  Agnes  perceived  that  in  online 
communication  context,  people  do  not  “have  a sense  of  each  other  as  people.”  She  said  she  had  to  “make  efforts  to 
individualize”  her  messages  according  to  the  different  perceptions.  Agnes  saw  herself  as  the  “focus  of  the  group” 
and  her  group  did  not  “see  itself’  except  in  relation  to  her.  She  said  she  would  intentionally  “disappear  for  a while” 
so  that  her  group  would  put  her  “in  perspective  again.” 

Agnes  thought  that  an  over-reliance  on  interpretations  and  assumptions  may  “mess  us  up  in  a terrible  way” 
because  in  the  online  situation,  they  often  “try  to  make  meaning  immediately  by  transferring  facts  and  information 
and  make  assumptions.”  She  said  some  classmates  do  not  express  much  and  they  just  use  some  “little  symbols”  to 
show  their  emotions  and  sometimes  it  is  difficult  to  really  know  how  they  feel.  As  an  example,  Agnes  recalled 
when  she  mistakenly  thought  a female  group  member  was  a man  based  on  the  name  and  the  reason  that  she 
communicated  more  fact  than  emotions.  Prior  to  confirming  this  “faceless”  individual’s  gender,  Agnes  said  she  had 
perceived  this  team  member  as  having  a very  strong  personality  and  was  not  moved  to  confirm  this  person’s  gender 
until  that  person  issued  an  apology  which  was  not  consistent  with  the  image  Agnes  held  for  that  person. 

Elizabeth  perceived  positive  feedback  as  an  important  method  for  online  communication.  She  said  she 
found  herself  transferring  her  communication  style  in  real  life  to  the  online  learning  environment.  She  said  because 
her  role  as  an  assistant  principal  at  work,  she  often  stressed  the  importance  of  “team  spirit”  and  showed  her 
appreciation,  encouragement,  supportive  by  giving  personalized  notes  to  her  group  mates  via  private  mail.  Elizabeth 
thought  that  people  build  relationships  through  communication  and  this  relationship  is  reciprocal.  “1  may  not  go  that 
extra  mile  for  someone  who  1 feel  does  not  go  that  extra  mile  for  the  group,”  said  Elizabeth.  Similar  to  Elizabeth, 
Agnes  also  mentioned  about  the  reciprocal  communication,  she  thought  asynchronous  online  interaction  really  lacks 
immediacy  and  intimacy. 

Communication  was  Nancy’s  major  focus  among  the  six  themes  emerged  across  participants.  Nancy 
thought  the  main  difference  between  online  and  face-to-face  collaborative  learning  was  the  “sharing  of  personal 
lives”  and  the  chances  for  misunderstandings.  She  said  limitations  include:  the  need  for  clarifications  seemed 
exponential;  miscommunications  seemed  rampant  due  to  individual  interpretations  and  assumptions;  group  members 
seemed  to  lack  an  awareness  of  each  others,  communication  methods  seemed  to  have  limitations;  misunderstandings 
seemed  to  arise  from  missing  cues  (verbal,  facial,  and  tone  of  voice);  communication  styles  and  skills  seemed  to 
vary  widely  among  members;  and  communication  tools  seemed  to  present  challenges  of  functionality. 

Nancy  recalled  an  example  where  her  group  was  chatting  online  and  some  members  of  the  group  kept 
talking  about  football  rather  than  tasks.  As  the  team  leader  for  that  particular  task,  she  tried  to  get  their  attention  by 
typing,  “Hello,  there....!”  Surprisingly,  one  of  the  group  members  said  she  was  shouting  and  the  other  interpreted 
the  situation  as  the  “cultural  gap.”  “It  had  nothing  to  do  with  culture,  1 understood  what  they  were  talking  about  and 
1 have  been  exposed  to  football  all  my  life,  but  let’s  just  get  back  to  the  subject,  will  we?”  said  Nancy.  “Everything 
you  type  in  chat  is  in  the  middle  of  my  conversation  with  the  others,”  Nancy  added.  She  said  she  did  not  want  to 
spend  many  hours  online  chatting  without  getting  tasks  done  and  as  the  project  leader,  she  felt  responsible  to  keep 
the  meeting  effective  and  efficient. 

Angel  also  mentioned  about  people  misunderstood  her  but  she  said  it  is  important  to  clarify  and  negotiate 
meaning.  “It  may  take  some  time  to  get  your  message  across  or  to  understand  what  others  have  to  say,”  said  Angel. 
But  she  thought  if  everyone  is  sincere  and  sensitive  to  others’  needs,  the  trust  that  build  in  the  group  will  save  a lot 
of  time  in  later  tasks.  Similar  to  Nancy,  Angel  also  observed  different  types  of  communication  styles  in  this  CSCL 
environment.  They  both  mentioned  that  in  the  online  environment,  some  people  are  more  active  while  the  others  are 
more  passive;  some  are  assertive  while  the  others  just  go  with  the  flow;  some  feel  more  comfortable  revealing  their 
feelings  while  others  do  not  reveal  their  personal  lives  in  the  online  settings;  some  were  formal  while  the  others  were 
very  casual  and  made  jokes  or  communicated  with  others  through  art  forms. 

Roles  and  identities 

How  members  of  the  group  identify  themselves  in  the  group  affect  how  they  act  and  communicate.  For 
Agnes,  knowledge  construction  and  group  collaboration  are  interrelate  to  herself  as  a person.  Identity  is  “who  I am 
to  myself  and  what  1 would  take  with  me  no  matter  what,”  said  Agnes.  She  said  that  there  were  “very  strong 
identities”  in  her  group  and  for  any  given  collaborative  projects,  group  members  would  “jump  right  in  and  grab”  the 
tasks  they  prefer  doing.  Agnes  thought  it  was  good  to  rotate  leadership  role  in  collaborative  teams,  but  guidelines 
for  group  leaders  should  be  directed.  She  stressed  that  being  “assertive”  was  one  of  the  most  important 
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characteristics  that  online  leaders  should  possess.  She  also  said  these  leaders  should  be,  “very  organized,” 
consistent;  “responsible,”  and  “active  in  participation,”  and  one  who  “keeps  others  organized  to  a certain  level.” 

Nancy  said  that  roles  and  responsibilities  within  the  group  should  be  well-defined.  “Collaboration  should 
be  done  in  a way  that  everyone  knows  their  roles,”  said  Nancy.  Nancy  said  there  were  some  “overlapping  work  and 
gaps”  in  her  group.  She  thought  that  it  is  very  important  to  “align  people  and  their  roles  very  carefully.”  Nancy  said 
that  in  a group,  “people  are  from  different  kinds  of  backgrounds”  but  “diversity  works  well  only  if  everyone  knew 
his  or  her  roles  and  responsibilities”  and  if  they  have  “positive  attitudes.”  She  said  that  a successful  group  should 
have  strong  leadership.  Nancy  recalled  her  experience  being  the  project  leader  for  one  module  and  said  one  of  the 
most  challenging  things  was  that  “everyone’s  concept  of  time  is  different”  and  being  at  the  forefront  having  certain 
responsibilities  in  “making  sure  everything  is  fine,”  she  said  it  required  certain  skills.  She  said  if  a leader  tried  to 
“control”  and  hold  “power”  rather  than  “lead,”  then  that  person  was  “not  doing  a leader’s  job.”  Rather,  the  person 
was  “doing  authoritarian  work.”  Similar  to  Nancy,  Angel  thought  it  was  important  to  have  well-defined  roles  and 
responsibilities.  She  said  the  course  did  not  give  clear  guidelines  because  the  course  expect  students  to  take  turns 
performing  leadership  roles  while  everyone  holds  different  concepts  and  expectations  toward  leadership  role.  She 
said  that  one  of  the  big  challenges  in  any  group  collaboration  is  the  distribution  of  work.  Those  who  are  leaders  will 
naturally  take  on  more  work  while  others  may  just  laid  back  and  become  less  involved. 

Elizabeth  said  she  identified  herself  as  a part  of  the  whole  group.  She  described  her  feeling  of 
“togetherness”  and  having  mutual  goals  helped  her  get  focused.  “Our  suite  is  a little  society  and  out  chat  is  kind  of 
like  the  town  meeting,”  said  Elizabeth.  She  said  that  because  the  group  “struggled  together”  in  completing  tasks 
throughout  the  semester,  they  build  that  sense  of  “togetherness”  and  “team  cohesiveness.”  Elizabeth  thought  the 
term,  “leader”  should  be  called  something  else  in  the  online  collaborative  learning  environment  because  there  could 
be  multiple  leaders  in  a group.  She  said  some  members  are  better  at  making  decisions  while  others  may  be  good  at 
communicating  with  others;  some  are  good  at  finding  resources  while  others  may  be  good  at  organizing  various 
ideas;  some  are  good  at  keeping  members  on  track  while  others  may  bring  the  group  joy  and  cheer  everyone  up 
when  the  group  is  stressed. 

Environment 

Participants  referred  “environment”  as  the  course  virtual  workspace  and  the  people  involved.  Agnes 
thought  that  the  “patterns  of  relationships”  have  strong  connection  with  the  environment  they  find  themselves  in  and 
that  this  CSCL  environment  was  largely  crafted  by  the  instructor  and  technology.  She  said,  “Life  imitates  art.” 
Agnes  believed  that,  “all  thought  internal  is  first  modeled  through  the  relationships  occurring  externally.”  She 
thought  our  “consciousness  comes  after  relationships”  and  “dialogue  comes  before  internal  monologue.”  Agnes 
suggested  that  CSCL  instructors,  when  designing  their  courses,  should  “create  a safe  environment”  so  that  students 
would  not  “just  project  their  meanings  on  everything”  but  would  “take  risks”  and  “experience  the  new  meaning  and 
new  structures.”  Instructors  should  also  consider  their  own  “values”  because  instructors  are  “part  of  the  big 
environment”  and  their  values  impact  how  students  experience  the  process,  she  said. 

Elizabeth  perceived  the  course  environment  as  a place  where  people  “come  together”  to  make  friends  and 
accomplish  tasks.  Due  to  the  various  tools  used  in  the  course,  Elizabeth  perceived  environment  as  multiple  places. 
She  said  she  was  often  confused  by  too  many  places  to  go  to  interact  with  others  and  to  work  through  assignments. 
Angel  perceived  the  environment  as  a very  structured  place  where  students  learn,  interact  with  others,  and 
accomplish  tasks.  Nancy  viewed  the  environment  as  a “big  open  communication  space”  where  people  just  come 
together  and  collaborate  on  a variety  of  projects.  She  said,  “supervision  is  required.”  Students  need  to  “have 
enough  guidance”  rather  than  just  being  “left  alone”  and  there  should  be  “some  kind  of  facilitator  or  mediator  to 
solve  conflicts”  and  to  “encourage  people  to  be  more  honest  with  each  other.” 

Angel  said  the  course  was  very  well-structured  and  there  were  many  intensive  tasks  they  had  to 
accomplish.  The  course  expected  students  to  log-on  everyday  and  arranged  one  “sherpa”  in  each  team.  However, 
“sherpas”  have  different  styles,  knowledge,  status,  and  access  to  specifications  about  the  course.  When  members  of 
the  group  encounter  problem,  they  look  up  to  “sherpas”  to  answer  their  questions  immediately  and  take  the 
leadership  role  to  guide  their  work.  She  said  her  group  “sherpa  was  very  encouraging  and  cheerful”  but  she  said 
most  class  members  expect  to  have  more  frequent  and  consistent  feedback  from  sherpas  and  they  need  feedback  on 
their  progress  from  the  instructor.  The  course  designed  many  interactive  activities  for  members  to  collaborate,  but 
there  seemed  to  be  lack  of  interaction  between  students  and  sherpas  and  between  students  and  the  instructor. 

Feelings 
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Participants  expressed  feelings  of  uncertainty,  stress,  frustration,  and  alienation.  Agnes  said  she  felt 
“alienated  from  others”  and  “unbelievably  stressed”  because  of  the  “tight  schedule”  and  the  “nature  of  the  work.” 
Agnes  said  she  likes  to  “be  around  people”  but  her  frustration  came  from  needing  “to  be  with  people”  and  “to  do 
works  that  have  meaning.”  Agnes  expressed  her  hopes,  fears,  and  worries:  “I  worry  that  we  are  bringing  our  old 
solutions  to  the  new  forms,”  She  challenged  Web-based  instructors  to  deliberate  on  a question:  “Will  people  realize 
their  potential  for  humanitarian  action  or  be  seduced  by  the  flavor  of  power  and  reduce  further  our  chances  to  create 
a joyful  world?” 

All  participants  shared  similar  feelings  of  stress  and  uncertainty  in  this  Web-based  CSCL  class.  Agnes  said 
that  she  did  not  know  what  others  were  thinking  and  feeling  because  feedback  was  either  delayed  or  non-existent. 
“There  are  many  things  that  you  can’t  predict,”  said  Nancy.  She  said  she  was  not  sure  that  “true  feelings”  were 
routinely  expressed  online.  “People  would  be  greeting  each  other  or  whatever,  but  you  don’t  know  if  they  meant  it  or 
just  typed  it,”  Nancy  added. 

Elizabeth  and  Angel  expressed  feelings  of  confusion  and  frustration.  Elizabeth  said,  “You  have  to  go  to 
different  places  to  get  one  piece  of  information  sometimes,”  She  suggested  that  the  navigation  should  be  as  simple 
as  possible  for  users  rather  than  offering  multiple — and  confusing-paths  for  users.  Angel  recalled  her  difficulty 
accessing  the  computer.  She  said  because  it  was  her  first  semester  in  a foreign  country,  she  did  not  have  a computer. 
The  University  libraries  often  close  on  holidays  so  she  could  not  do  the  work.  Angel  said  she  sometimes  became 
frustrated  when  some  TeachNet  functions  did  not  access  from  the  Web  or  when  the  server  would  go  down. 

Personal  variables 

Agnes  thought  personal  factors  influence  understanding,  thinking,  and  experiences  in  any  given  setting 
because,  “Everything  is  a system.  A person  is  a system.  When  they  enter  a relationship  or  an  environment,  they 
bring  in  wherever  they  are  at.”  Agnes  described  her  group  as  a meeting  place  where  “the  full  personality  drama” 
occurred  and  where  everyone  diverged  in  their  “working  and  writing  styles.”  Agnes  thought,  “some  people  are 
more  comfortable  with  patriarchal  hierarchies”  which,  she  said,  is  not  based  on  ethnicity  or  gender.  Like  Agnes, 
Nancy  also  perceived  that  personal  variables  influenced  online  collaboration.  She  observed  differences  in 
personalities,  preferences,  interests,  values,  cultural  backgrounds,  communication  styles,  and  working  styles.  Nancy 
said  that  she  tried  to  “guess”  what  others’  personalities  were  and  attempted  to  “accommodate”  their  personalities 
based  on  “their  contributions.”  She  observed  that  everyone  in  her  group  differed  in  their  priorities,  in  their 
reactions,  and  their  modes  of  thinking,  collaborating,  communicating,  and  writing.  Nancy  thought  her  family 
upbringing  also  affects  how  she  interacted  with  others.  “I  prefer  people  do  what  they  are  comfortable  with,”  said 
Nancy.  She  thought  that,  “there  should  be  one  person  taking  the  lead  and  others  follow.” 

Elizabeth  and  Angel  also  expressed  how  their  family  upbringing  affected  their  online  behavior.  Elizabeth 
said  her  family  emphasizes  responsibility  and  the  importance  of  maintaining  public  decorum.  She  said  living  with 
extended  family  members  is  part  of  her  culture,  and  from  being  exposed  to  various  family  members  with  multiple 
perspectives,  she  learned  to  accept  and  respect  others’  point  of  views  more  readily.  Angel  said  in  India,  men  and 
women  are  normally  separated  throughout  primary  and  secondary  education  and  the  society  is  structured 
hierarchically.  She  said  that  because  she  often  had  to  solve  problems  by  herself  while  growing  up,  every  time  she 
encountered  difficulties  in  the  CSCL  class  she  would  attempt  to  solve  problems  by  herself  rather  than  seek  help. 

Conclusion  and  Future  Implications 

The  findings  of  this  study  suggest  that  participants  perceived  communication  and  interpersonal 
relationships  as  two  major  areas  of  concern  in  the  online  group  collaborative  learning  environment.  Future  studies 
should  focus  on  the  social  aspects  of  interactions  in  the  Web-based  computer  supported  learning  environment. 
Further,  the  implementation  process  should  be  closely  examined  in  order  to  gain  a better  understanding  of  how 
various  learners  define  knowledge,  how  knowledge  is  communicated  and  negotiated  among  learners  of  diverse 
backgrounds,  and  how  knowledge  is  constructed  as  the  result  of  communication  and  negotiation. 

Due  to  the  nature  of  collaboration  and  the  necessity  to  negotiate  toward  shared  meanings,  online 
collaborators  reported  spending  a considerable  amount  of  time  attempting  to  understand,  check,  confirm,  coordinate, 
and  negotiate  with  group  members.  Future  studies  should  focus  on  strategies  utilized  by  experienced  on-line 
collaborative  learners  to  facilitate  and  ensure  effective  and  efficient  communication.  In  this  study,  participants 
suggested  that  in  order  to  make  the  group  decision-making  process  easier  and  results  more  meaningful,  individuals 
should  clearly  state  their  expectations,  be  cognizant  of  the  various  cultural  and  individual  notions  of  time  and 
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responsibility,  be  sensitive  to  others’  perspectives  and  needs,  and  communicate  frequently  with  group  members  to 
resolve  conflicts. 

Based  on  my  findings  and  observations,  the  following  is  a list  of  eight  suggestions  for  Web-based 
instructors  to  consider  as  they  design  online  collaborative  learning  courses: 

1 . Assumptions  and  Interpretations:  Both  students  and  the  instructor  should  be  cognizant  of  their 

* preconceptions,  expectations,  and  possible  assumptions.  Our  focus  of  attention  and  our  behavior 

are  both  heavily  influenced  by  what  we  know  and  how  we  know  it. 

2.  Social  Aspects  of  Learning:  The  social  aspects  of  learning  should  be  emphasized  as  much  as  the 
cognitive/  intellectual  aspects  of  learning.  The  course  instructor  should  understand  the  importance 
of  designing  activities  that  facilitate  social  interactions  as  well  as  the  importance  of  constantly 
providing  to  and  requesting  feedback  fro  m learners. 

3.  Positive  Attitude : Members  of  the  community  should  maintain  positive  attitudes  and  strive  to  be 
open  and  flexible  while  understanding  that  learning  is  an  ever-evolving  process. 

4.  Modeling:  The  instructor  should  guide,  coach,  and  facilitate  learning  by  providing  instant 
feedback.  If  the  instructor  expects  learners  to  check-in  everyday,  he/she  should  do  the  same. 
Instructor  and  staff  policies  should  be  consistently  applied  to  all  members  of  the  community. 

5.  Real  and  Humanized  Instruction : The  main  emphasis  of  learning  tasks  should  be  on  humans, 
not  tools.  Tasks  that  require  students  to  mechanically  perform  procedures  may  not  offer 
meaningful  learning  to  learners. 

6.  Process  and  Product:  The  process  of  knowledge  acquisition  and  group  interactions  should  be 
evaluated  as  stringently  as  the  final  product.  Since  the  human  mind  evolves  through  a process  of 
perceptions  altered  by  experience,  how  learners  undergo  knowledge  construction  and  group 
collaboration  are  as  important  as  the  resultant  product. 

7.  Performance  Assessment:  Learners  should  be  held  accountable — but  accountability  should  be 
consistent  and  justifiable.  How  process  can  be  evaluated  should  be  incorporated  and  considered  in 
performance  assessments. 

8.  Safe  Environment:  An  environment  of  fear,  envy,  and  anger  is  detrimental  to  learning. 
Conversely,  an  environment  of  joy,  mutual  respect,  and  sympathy  is  conducive  to  learning. 
Learning  is  promoted  in  an  environment  where  students  are  ensured  free  expression,  are  enabled 
to  make  mistakes  because  mistakes  are  explicitly  viewed  as  acceptable  inevitabilities  in 
knowledge  construction,  are  inspired  and  encouraged  by  peers  and  instructor,  and  are  intellectually 
challenged. 
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Abstract 


In  the  online  learning  environment,  communication  and  collaboration  occur  from  a distance.  In  the 
absence  of  non-verbal  communication  cues  (such  as  facial  expressions,  gestures,  and  tone  of  voice), 
understanding,  communication,  and  interaction  rest  largely  on  individual  assumptions  and  interpretations. 
The  magnitude  of  these  communication  challenges  increases  as  cultural  differences  among  communicators 
widen. 

The  purposes  of  this  study  are  to  (1)  discuss  the  role  of  effective  intercultural  communication  and 
collaboration  in  the  Web-based  collaborative  learning  community,  (2)  examine  helpful  strategies  utilized 
by  both  instructors  and  learners  in  a Web-based  curriculum  model  where  diversity  is  treated  as  a fact  and 
collaboration  among  learners  of  diverse  backgrounds  as  the  norm,  and  (3)  discuss  future  implications  for 
integrating  technology  into  curriculum  targeted  to  global  learning  populations. 


Introduction 

The  Internet  has  provided  the  possibility  to  erase  geographic  and  interpersonal  boundaries  among  people  of 
diverse  backgrounds,  has  created  opportunities  for  widespread  electronic  delivery  of  news,  information,  and 
curriculum,  and  has  altered  the  way  we  communicate,  share  knowledge,  deliver  education,  and  conduct  business. 
Globally,  market  expansion  and  course  offerings  are  increasingly  assisted  by  use  of  the  Internet.  In  education,  the 
Internet  has  been  widely  used  in  preparing  learners  for  “future  responsibilities”  and  “success  in  life”  (Dewey,  1938, 
p.  17).  Cross-nation  collaborative  projects  on  the  Internet  across  disciplines  have  exponentially  increased 
internationally  in  recent  years. 

Amidst  the  hype  of  cutting-edge  technology,  however,  it  is  often  overlooked  that  computers  do  not  think, 
only  humans  do;  computers  do  not  have  agency,  humans  do;  technological  difficulties  do  not  limit  intellectual 
advancement,  humans  do.  Technology  potentially  provides  an  array  of  resources,  but  also  presents  constraints. 
Indeed,  humans  ultimately  decide  how  to  utilize  new  technologies,  and  these  decisions  are  often  based  on  both 
proven  (or  unproven)  and  discovered  strategies.  To  integrate  technology  into  instruction  and  learning,  it  is  essential 
to  focus  on  human  needs,  strategies,  perceptions,  and  experiences  communicating  and  collaborating  in  the  cyber- 
leaming  environment  — rather  than  solely  on  the  functionality  of  technological  tools. 

As  Web-based  learning  communities  and  online  school  partnership  projects  exponentially  expand  globally, 
effective  intercultural  communication  and  collaboration  is,  as  never  before,  of  vital  significance.  Web-based 
instruction  provides  opportunities  for  a diverse  body  of  individuals  to  interact.  In  order  to  suit  individual  learner’s 
needs  while  optimizing  their  strengths  and  talents,  Confucius  said,  instruction  should  be  tailored  to  meet  individual 
learner’s  needs.  While  instructors  are  challenged  to  understand  and  be  sensitive  to  the  needs  of  learners  in  their 
design  and  implementation  of  Web-based  courses,  learners’  success  hinges  not  only  on  their  willingness,  attitude, 
and  devotion  to  achieve  shared  goals  in  a sea  of  information,  but  also  on  how  they  handle  the  challenges  of 
consolidating  fellow  learning  community  members*  multiple  views,  perspectives,  and  approaches. 

Based  on  my  Web-based  collaborative  learning  curriculum  model  experiences  as  a student,  researcher, 
moderator,  instructional  designer,  and  teaching  assistant,  this  paper  intends  to  explore  how  effective  communication 
and  collaboration  among  members  of  diverse  backgrounds  may  be  encouraged  in  the  design  and  implementation 
process  and  to  explore  what  intercultural  communication  techniques  may  be  or  have  been  utilized  by  members  in  the 
Web-based  learning  community  in  bridging  differences,  achieving  group  goals,  and  optimizing  individual  learning. 

Web-Based  Instruction:  Opportunities  and  Challenges 

The  development  of  computer  network  technology  provides  opportunities  for  dynamic  human  contact  and 
collaboration.  Teachers  at  all  instructional  levels  are  learning  and  integrating  technology  into  curriculum  and 
instruction.  Internet  access  in  public  schools  and  instructional  sites  in  America  has  increased  from  35  percent  in 
1994  to  89  percent  in  1998  and  is  expected  to  grow  further.  Student  computer  use  has  also  increased  from  27.3 
percent  in  1984  to  68.8  percent  in  1997.  These  changes  represent  increases  of  10.3  percent  at  the  Pre-K  level,  10.4 
percent  in  grades  1-8,  12.3  percent  in  grades  9-12,  9.5  percent  at  the  college  level,  and  3.4  percent  at  the  graduate 
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school  level  (U.S.  Department  of  Commerce,  1998).  Major  reasons  for  this  rapid  growth  include  technology’s 
capacity  for  reaching  remote  locations  while  providing  immediate  and  engaging  communication,  its  promise  of 
collaborative  opportunities,  and  the  human  need  for  connection  and  interaction. 

The  Internet  is  a global  society  involving  users  of  diverse  backgrounds,  and  this  global  society  is  comprised 
of  various  communities  with  cultures  of  their  own.  This  Internet  society  is  constantly  evolving  due  to  the  evolution 
of  tools  and  stakeholders  involved.  In  this  society  where  people  communicate  to  collaborate  the  methods  of 
communication  and  collaboration  evolve  with  the  evolution  of  technological  tools  and  the  dynamics  of  human 
interactions. 

Web-based  instruction  has  often  been  implemented  to  supplement,  enhance,  and  transform  existing 
curriculum.  Some  instructors  use  the  Web  to  post  syllabus  online,  to  supplement  face-to-face  class  teaching,  to 
broaden  the  scope  of  their  instruction,  or  to  deliver  courses  in  their  entirety.  Many  distance-learning  courses, 
however,  have  encountered  high  attrition  rates  (Abdul-Rahman,  1994;  Galusha,  1997),  unequal  participation  and 
individual  commitment,  role  ambiguity  in  group  contexts,  absenteeism,  inattention  to  social  relationships,  and 
students  feeling  overwhelmed  (O’Hara-Devereaux&Johnasen,  1994).  The  viability  of  effective  communication  and 
collaboration  in  the  absence  of  face-to-face  interaction  has  been  questioned  (Handy,  1995).  When  students  find 
course  content  irre  levant,  perceive  the  environment  as  impersonal,  can  not  obtain  immediate  technical  or 
instructional  assistance  or  timely  instructor  feedback,  and  feel  disconnected  from  fellow  learners,  they  may  feel 
isolated,  unmotivated,  or  unchallenged.  These  shortcomings  originate  from  a lack  of  understanding  and  sensitivity 
to  the  dynamics  of  human  interaction  and  the  nature  of  human  learning. 

In  the  following  section,  1 will  first  explore  topics  related  to  community-building,  cooperative  and 
collaborative  learning,  culture  and  communication,  and  intercultural  communication  as  they  relate  to  the  curriculum 
model  examined  and  the  purpose  of  this  study.  Then,  1 will  examine  a Web-based  curriculum  model  based  on  my 
two  years  experience  as  student,  researcher,  moderator,  instructional  designer,  and  teaching  assistant  within  this 
model.  Finally,  1 will  discuss  implications  for  future  global  Web-based  learning  curriculum. 

Community  Building  & Web-Based  Learning 

Despite  Thoreau’s  fervent  wish  for  solitude  while  at  Walden  Pond  (Thoreau,  1996),  the  effects  of  isolation 
are  so  powerful  that  isolation  is  a potent  punishment  technique.  Parents  and  teachers  isolate  misbehaving  children. 
Hostages  are  held  in  isolation,  as  are  miscreant  prisoners.  Most  of  us  cannot  tolerate  enclosure  without  contact  for 
very  long;  we  crave  company  to  belong  to  and  share  with,  and  love  is  as  necessary  as  the  air  we  breathe.  Similarly, 
contact  is  often  necessary  to  accomplish  our  goals;  communication  is  essential  to  our  pursuit  of  achievement  as 
social  beings. 

In  “My  Pedagogic  Creed”  (1929),  Dewey  said,  “The  educational  process  has  two  sides,  one  psychological 
and  one  sociological.”  Dewey  emphasized  the  social  aspect  of  learning.  Dewey  thought  some  schools  failed 
because  they  neglected  “the  school  as  a form  of  community  life.”  Educators  are  aware  of  and  have  been  utilizing 
the  power  of  people  working  together  to  build  a "community  of  learners."  Increasingly,  this  notion  has  been  re- 
emphasized with  the  advance  in  technology  and  global  network.  "The  community  is  something  that  happens.  It 
happens  when  people  speak  to  one  another  and  listen  to  one  another  in  an  effort  to  discern  the  truth  and  to  discover 
themselves  in  the  process.  It  happens  only  in  an  environment  of  freedom  and  openness.  Community  happens  only 
in  an  atmosphere  of  honesty  and  tolerance.  Community  happens  when  people  care  about  one  another  and  when  they 
are  willing  to  take  responsibility  for  themselves  as  well  as  for  each  other,”  said  Robert  Berdahl,  the  chancellor  of  the 
University  of  California  at  Berkeley  (Berdahl,  1998).  It  can  be  inferred  from  Berdahl’s  commentary  that 
communication  and  collaboration  are  essential  for  community  building. 

Communities  are  essential  in  human  history.  We  have  constantly  sought  to  survive  collectively,  to  belong, 
to  care,  and  to  support  each  other.  Various  forms  of  communication  and  culture  are  often  established  in  the 
community  to  maintain  cooperation  and  collaboration.  Maurice  Friedman  (1983)  distinguished  two  kinds  of 
community:  a community  of  affinity,  which  refers  to  a group  of  “like-minded”  people  who  have  come  together  for 
security,  and  a community  of  otherness,  which  refers  to  when  members  of  the  group  are  not  alike,  but  when  they 
share  common  concerns.  Within  a particular  virtual  community  (a  form  of  network  organization  through  the  use  of 
technological  tools),  people  share  “common  concerns."  They  are  not  necessarily  "like  -minded"  and  while  the  same 
beliefs  and  values  may  not  be  commonly  shared,  certain  visions,  concerns,  or  goals  are  shared  (Friedman,  1983). 

Many  educators  believe  that  learning  is  more  effective  when  collaborating  students  encounter  conflicts 
(Piaget,  1977;  Doise  & Mugny,  1984;  Savery  & Duffy,  1996).  According  to  these  educators,  educational,  social, 
and  cultural  background  differences  are  not  necessarily  negative,  but  rather,  through  interactions  among  different 
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people  new  thoughts  and  critical  thinking  and  analysis  are  inspired  and  opportunities  for  individuals  to  gain  insights 
are  provided. 

The  richness  of  multiple  perspectives  in  the  Web-based  learning  environment,  if  fully  nurtured,  can  help 
spur  the  formation  of  rich  and  innovative  perspectives.  Conflicts  are  positive  when  collaborators  communicate  and 
negotiate  to  reach  a shared  understanding,  discover  new  paths,  and  construct  new  thoughts  and  ideas  together. 
Sharing,  exchanging,  and  negotiating  provide  learners  opportunities  to  deepen  their  understanding.  During  the 
process  of  communication  and  collaboration  in  knowledge  acquisition,  negotiation,  decision-making,  and 
knowledge  construction,  learners  hopefully  slough  off  their  unconscious  reliance  on  preconceptions,  biases, 
unproven  interpretations,  and  assumptions. 

Cooperative  and  Collaborative  Learning 

According  to  the  1999  U.S.  Department  of  Commerce  report,  ”2 1st  Century  Skills  for  21st  Century  Jobs,” 
the  proportion  of  skilled  in  the  workforce  increased  from  20  percent  in  1950  to  60  percent  in  1997  while  the 
proportion  non-skilled  laborers  in  the  workforce  decreased  from  60  percent  to  20  percent  illustrating  that  the 
demand  for  skilled  workers  has  dramatically  increased  over  the  past  half  century  while  the  demand  for  unskilled 
laborers  has  dramatically  decreased.  In  this  new  economy.  Fortune  500  companies  regard  “teamwork,”  as  the  most 
critical  job  in  the  21®*  century  skill.  Concomitantly,  the  ability  to  cooperate,  collaborate  and  work  well  with  others  is 
considered  more  important  than  basic  skills  (such  as  reading  and  writing)  that  traditional  schools  have  long  defined 
as  being  most  important  (2 1 St  Century  Skills,  1999,  p.  1). 

Studies  consistently  indicate  that  cooperative  and  collaborative  learning  are  not  only  effective  in  generating 
positive  outcomes  in  academic  performance,  but  are  also  influential  in  affective  and  social  aspects  of  learning 
(Johnson,  Johnson,  & Smith,  1998;  Slavin,  1991,  Harasim,  1990).  “When  the  classroom  is  structured  in  a way  that 
allows  students  to  work  cooperatively  on  learning  tasks,  students  benefit  academically  as  well  as  socially,”  Slavin 
(1987)  said  in  extolling  the  benefits  of  cooperative  and  collaborative  learning. 

Cooperation  creates  possibilities.  As  Fosnot,  (1996)  said,  “As  ideas  are  shared  within  a community,  new 
possibilities  are  suggested  to  the  individual  for  consideration.  These  multiple  perspectives  may  offer  a new  set  of 
correspondences,  and  at  times  even  contradictions,  to  individual  constructions.”  (Fosnot,  1996,  p.  27)  Johnson  and 
Johnson  (1990)  concluded  that,  “Achievement  is  higher  in  cooperative  situations  than  in  competitive  or 
individualistic  ones  and  cooperative  efforts  result  in  more  frequent  use  of  higher-level  reasoning  strategies,  more 
frequent  process  gain,  and  higher  performance  on  subsequent  tests  taken  individually  (group-to-individual  transfer) 
than  do  competitive  or  individualistic  efforts”  (Johnson  & Johnson,  1991). 

“Groups  pull  and  tug.  They  pull  between  tasks  to  accomplish  and  work  to  produce,  but  they  also  tug  to 
maintain  cohesiveness  and  an  optimal  level  of  morale.”  (Schmuck  & Schmuck,  1997,  p.  263)  Working  in  a group 
can  be  irritating  and  frustrating.  While  differences  in  knowledge  and  skills  may  influence  overall  results,  major 
challenges  and  problems  to  cooperation  and  collaboration  come  from  differences  in  attitudes  and  in  individuals’ 
willingness  to  communicate,  negotiate,  change,  commit,  and  take  responsibility — rather  than  superficial  differences 
in  gender,  ethnicity,  race,  class,  religion,  or  culture. 

Culture  and  Communication 

Mirroring  the  American  community,  public  school  students  represent  an  array  of  cultural  backgrounds. 
Globalization  has  widened  the  need  for  intercultural  communication  and  collaboration.  Dodd  & Montalvo  (1987) 
identified  intercultural  effectiveness  as  a desirable  skill  essential  in  creating  a productive  and  non-threatening 
communication  climate  mariced  by  non-dogmatic  openness  and  innovation.  Researchers  have  noted  the  limitations 
of  and  misunderstandings  in  cross-cultural  dialogue,  the  importance  of  cross-cultural  learning,  and  the  necessity  of 
being  culturally  sensitive  and  responsive  (McLoughlin,  1999). 

With  ample  knowledge  and  strategies,  technological  tools  are  the  means  rather  than  ends  to  goals  we  want 
to  achieve.  Given  the  importance  of  global  communication  and  collaboration,  it  is  essential  to  assess  what  we  can 
do  to  bridge  the  differences  among  members  in  the  community  and  to  establish  culturally  sensitive  curriculum 
suitable  for  learners  of  diverse  backgrounds. 

Culture  and  communication  are  intertwined  and  interdependent.  Culture  influences  the  way  we  make 
meaning,  interpret,  and  communicate  messages.  Culture  is  complex  and  involves  a plethora  of  systems.  As 
described  by  Carley  H.  Dodd,  "Culture  is  like  a kaleidoscope  with  similar  shapes  but  different  colors,  or  at  other 
times,  with  different  shapes  but  similar  colors"  (Dodd,  1991,  p.  12).  Culture  is  often  described  as  the  dynamic 
interplay  between  the  experiences  of  people  and  the  social  structure  at  large  (Giroux,  1997;  Collier,  1998;  Grossberg 
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1992;  Grossberg,  1997;  Marcus  and  Shannon,  1997).  Culture  is  the  societal  situation  strained  through  the  mesh  of 
individual  perception.  This  dichotomy  is  almost  paradoxical  in  that  culture  is,  in  some  ways,  unique  to  the 
individual  yet  shared  by  a community.  One's  identification  with  a culture  comes  from  the  "sameness"  that  is  shared 
with  others. 

Cultural  differences,  as  referred  to  in  this  paper,  are  not  limited  merely  to  differences  in  ethnicity  or 
nationality.  Cultural  differences  indicate  "patterns  of  thinking,  feeling,  and  acting."  (Hofstede,  1997,  p.  5)  A group 
holding  the  same  "shared  constructions"  (Erlandson,  et.al.,1993,  p.  24)  represents  a culture  of  its  own.  Those 
unfamiliar  with  the  online  learning  "culture"  may  easily  get  lost  in  the  cyber  environment  and  feel  isolated.  Those 
ignorant  of  their  own  and  other’s  culture  may  feel  isolated  within  a collaboration  environment.  Those  unwilling  to 
be  open  to  and  respectful  of  multiple  perspectives,  accepting  of  ambiguities  or  uncertainties,  flexible,  and  sensitive 
and  mindful  of  cultural  communication  differences  may  encounter  difficulties  in  the  collaborative  learning 
environment. 

Collier  (1998)  characterized  cultural  identities  as  historical,  contextual,  and  relational  constructions  that 
have  "enduring"  and  "changing"  properties.  Cultural  identities  emerge  in  everyday  discourse  and  in  social  practices, 
as  well  as  by  rituals,  norms,  and  myths  that  are  handed  down  to  new  members  (Collier  1998).  Because  of  mobility, 
greater  accessibility  of  information,  and  expanded  multi-group  membership,  the  stereotypical  generalizations  has 
become  increasingly  invalid.  Most  people  belong  to  multiple  groups  (both  voluntary  and  not).  Dodd  suggests  that 
the  "macrocultural  system"  needs  to  be  considered  as  well  as  the  "subcultural  differences"  (Dodd  1991)  when  we 
look  at  individuals  interacting  in  a cultural  context.  While  the  society  in  which  an  individual  is  situated  largely 
influences  one’s  "macrocultural"  perspective,  the  individual  nature  of  culture  lies  in  the  uniqueness  of  the 
"subcultures"  to  which  one  belongs. 

In  the  online  learning  environment,  knowledge  acquisition  and  group  collaboration  differ  from  personal 
encounters.  In  this  environment,  learners  map  their  way  through  the  process  as  they  communicate  from  remote 
sites.  When  nonverbal  communication  cues  such  as  facial  expressions,  gestures,  and  tone  of  voice  are  largely 
absent,  the  frequency  of  communication  and  the  reliance  on  interpretation  increases.  Effective  communication  and 
collaboration  in  the  cyberspace  requires  patience  and  tolerance  as  members  work  to  understand  one  another  in 
accomplishing  group  and  individual  goals.  Gudykunst  (1994),  in  describing  effective  in-group  communication,  said, 
"When  there  is  a lack  of  connection  between  the  participants,  community  cannot  develop.  For  community  to 
develop,  dialogue  is  necessary.  Dialogue  involves  communication  between  individuals.  In  a dialogue,  each 
participant's  feeling  of  control  and  ownership  is  minimized;  each  participant  confirms  the  other,  even  when  conflict 
occurs"  (p.  223).  Interpersonal  relationships  are  essential  for  effective  communication  among  members  of  the 
group.  D.W.  Johnson  & R.T.  Johnson  (1975)  suggested  a few  interpersonal  processes  and  patterns  of  interaction 
within  the  structure  of  a cooperative  learning  environment:  mutual  liking;  effective  communication;  utilization  of 
fellow  students  as  resources,  division  of  labor,  and  divergent  and  risk-taking  thinking.  They  also  added  that  high 
interactions,  trust,  mutual  influence,  acceptance  and  support,  emotional  involvement,  and  coordination  of  effort  are 
also  essential. 

Collis  (1999)  proposed  that  cultural  sensitivity  should  be  a cornerstone  in  the  design  of  Web-based  courses 
because  culture  influences  the  acceptance,  use,  and  impact  of  learning  resources.  Culture  affects  the  individual's 
response  to  computer-related  system  (Collis  1999).  Among  Collis’  suggestions,  I find  the  following  points 
important  in  designing  a culturally  sensitive  course:  (1)  creating  an  atmosphere  of  non-critical  acceptance  of  ideas; 
(2)  moving  from  deficit-model  thinking  where  the  students  begin  with  a deficiency  in  terms  of  a lack  of  pre -defined 
knowledge  to  a socially  participative  model  where  the  students  learn  through  communicative  interaction  with  others 
that  includes  horizontal  communication  among  students  as  well  as  vertical  communication  among  the  facilitators 
and  students;  (3)  accepting  tone  and  style  differences  in  communication;  (4)  grouping  of  users  to  facilitate 
competence  and  comfort;  (5)  appropriately  allocating  responsibilities  among  students  and  facilitators;  (6)  accepting 
results  of  trial  and  error  in  course  design;  and  (7)  discarding  the  idea  that  learning  or  acting  differently  is 
unacceptable.  Collis  (1999)  concludes  that  institutions  and  instructors  will  not  only  need  to  improve  their  insight 
into  cultural  differences  that  affect  teaching  and  learning,  but  will  also  need  to  form  these  insights  into  manageable 
ways  to  respond  to  multiple  cultures. 

Bridging  the  Differences:  A Web-Based  Curriculum  Model 

This  paper  examined  a Web-based  Computer  Supported  Collaborative  Learning  (CSCL)  curriculum  model 
at  the  University  of  Texas  at  Austin.  This  model  treats  diversity  as  a fact  and  collaboration  as  fundamental  for 
knowledge  construction  through  the  provision  of  rich  information  and  resources  to  learners.  In  this  model,  multiple 
perspectives  are  valued.  Students  engaged  in  activities  where  knowledge  is  constructed  through  negotiation  among 
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collaborators  from  diverse  backgrounds  within  the  community.  Group  diversity  is  also  carefully  considered  in 
forming  collaborative  virtual  teams.  Students  were  first  asked  to  indicate  their  preferences  based  on  their  interests. 
Factors  such  as  ethnicity,  gender,  computer  skills,  and  geographic  locations  were  then  considered  in  building  virtual 
teams. 


This  curriculum  model,  initiated  by  Dr.  Paul  E.  Resta  at  the  University  of  Texas  at  Austin,  invited  a group 
of  graduate  students  from  diverse  backgrounds  with  various  strengths  and  talents  to  participate  in  the  course  design, 
development,  implementation,  evaluation,  and  revision  processes.  Reflecting  the  diverse  student  body  - and  the 
collaborative  nature  of  the  course  - these  graduate  students  from  diverse  cultural  backgrounds  and  expertise 
collaborate  to  design  and  improve  the  course  throughout  the  curriculum  development  and  implementation  process. 
The  course  is  based  on  an  on-going  revision  process  where  researchers  meet  on  a regular  basis  to  share  observations 
in  order  to  improve  the  course. 


Graph  #7 


Graph  #2 


Courses  in  this  model  are  situated  in  the  virtual  environment  where  students  “meet”  through  online 
communication  and  interaction.  A virtual  environment  with  collaborative  tasks  was  created  based  on  the  metaphor 
of  a hypothetical  technology  company,  school  district,  or  educational  technology  institute  (See  Graph  #7  for 
example  of  virtual  workspace).  Students  are  divided  into  groups  representing  a department  within  a company,  a 
school  within  the  district,  or  an  educational  sector  within  an  institute.  Problem-  and  project-based  approaches  to 
learning  were  employed.  In  these  courses,  students  must  collaboratively  complete  a technology  plan,  write  a grant 
proposal,  or  design  a CSCL  course  based  on  many  structured  tasks.  To  accomplish  course  requirements,  online 
socialization  and  communication  are  essential,  as  are  extensive  cooperation  and  collaboration,  among  learners  of 
diverse  backgrounds. 

Many  online  tools  are  provided  to  students  to  support  the  development  of  their  projects.  The  course 
content  with  rich  information  and  resource  links  is  provided  on  the  Web  via  a courseware  (WebCT,  Vcampus,  or 
Prometheus),  while  class  discussions  and  interactions  take  place  at  a virtual  platform  on  TeachNet  via  FirstClass 
groupware.  For  synchronous  interaction,  the  online  chat  function  is  employed.  Learners  participate  in  monthly 
Webcast  (videoconference)  on  campus  (face-to-face)  or  by  way  of  network  accessing  (see  Graph  #2  for  example), 
which  serve  as  a monthly  forum  for  guest  experts’  discussion  of  relevant  topics,  teams  to  share  their  work,  and  the 
instructor  to  answer  student  questions  or  give  advice.  Courses  are  divided  into  modules;  module  tasks  progress  from 
simple  to  complex.  Learners  are  either  on-campus  students  at  the  University  of  Texas  at  Austin  or  distance-learning 
students  from  across  Texas  and  the  United  States.  On -campus  students  have  the  option  of  meeting  the  instructor, 
staff,  and  peers  face-to-face  or  through  the  Internet,  while  off-campus  students  can  interact  only  via  the  network  and 
phone  communication. 

Communication:  Challenges  and  Strategies  Employed 

Communication  in  the  Web-based  collaborative  learning  environment  includes  both  task  and  social  aspects. 
Due  to  communication  limitations,  online  collaborators  easily  spend  a large  portion  of  time  understanding,  checking. 
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confirming,  coordinating,  and  negotiating  to  obtain  mutual  understandings  and  to  construct  shared  knowledge.  With 
the  limitations  in  Web-based  communication  tools  and  the  resultant  reliance  on  interpretations  and  assumptions, 
effective  communication  among  learners  of  diverse  backgrounds  is  very  challenging.  A few  examples  of  Web 
design  that  may  be  open  to  cultural  interpretation  include:  various  expectations  for  communication  (some  learners 
may  expect  daily  communication  from  others  in  the  team  while  others  are  not  accustomed  to  communicating 
online),  the  amount  and  type  of  information  desired  and  required  (some  learners  may  expect  explicit  information 
while  others  who  are  accustomed  to  implicit  expression  do  not  feel  comfortable  being  direct),  communication  styles 
and  preferences  (some  prefer  to  take  turns,  some  like  to  jump  to  conclusions,  while  others  like  to  wait  for  cues), 
level  of  task  orientation  (some  are  task-driven  high  achievers  while  others  take  a laid-back  position),  variation  in 
understandings  (some  are  from  the  culture  where  the  course  is  situated  while  others  may  situate  in  a completely 
different  culture),  group  orientations  (some  prefer  and  have  experience  working  in  groups  while  others  prefer 
competition  rather  than  collaboration),  and  group  dynamics  (some  cultures  may  look  to  a leader  for  direction  while 
others  work  better  when  there  is  no  single  leader  in  the  group). 

As  opposed  to  face-to-face  settings  where  communicators  are  recognized  as  human,  in  the  Web 
environment,  people  are  often  objectified  and  largely  recognized  by  name.  As  opposed  to  face-to-face  settings  that 
are  conducted  naturally,  in  the  Web  environment,  there  is  usually  a 45  seconds  time  lag  between  event  and 
broadcast.  As  opposed  to  face-to-face  settings  where  it  is  possible  to  drag  a person  aside  and  whisper  in  privacy,  in 
online  chat,  this  is  not  possible  because  everyone  shares  the  same  space.  As  opposed  to  face-to-face  settings  where 
communicators  can  choose  their  focus  of  attention  (or  inattention),  in  the  video-conferencing  environment,  the 
camera  setting  dictates  viewers’  attention. 

In  my  previous  study  entitled,  “Multiplicity  and  flexibility  as  design  features  - A case  study  of  a Web- 
based  collaborative  learning  community  for  diverse  learners”  (to  be  published  in  the  Computer  Supported 
Collaborative  Learning  2002  conference  proceeding),  I identify  multiplicity  and  flexibility  as  two  key  course  design 
features  essential  for  meeting  the  needs  of  students  from  diverse  backgrounds.  As  a teaching  assistant,  I worked 
closely  with  the  instructor  and  learners  and  observed  the  course  while  assisting  in  its  implementation.  A few  course 
aspects  were  highlighted  to  illustrate  multiplicity  and  flexibility:  course  content  and  structure;  communication  tools, 
channels,  and  types;  support,  accessibility,  and  feedback;  and  performance  assessment.  Multiplicity  refers  to  the 
multiple  ways  of  presenting  and  delivering  course  material,  channels  of  communication,  activity  offerings,  and 
learning  strategies.  Flexibility  refers  to  the  welcoming  of  and  openness  to  questions  and  suggestions,  timely 
support,  options  for  learning  tasks,  and  provision  of  individualized  feedback  throughout  the  course. 

The  provision  of  multiple  learning  options  is  critical  to  enable  deeper  understandings  among  students  from 
various  cultural  and  social  backgrounds  who  hold  widely  divergent  individual  values  and  interests.  Multiple  course 
representation  methods  were  employed  to  meet  the  needs  of  multiple  users.  These  representations  included  text, 
audio,  video,  and  simulation,  as  well  as  multiple  learning  contexts  and  strategies.  This  curriculum  model  provided 
options  in  some  assignments.  For  example,  in  one  of  the  classes  offered,  students  had  the  options  of  type  of  medium 
and  their  focus  in  an  assignment  regarding  leadership  visions.  Some  students  chose  film,  others  chose  print,  while 
others  drew  from  their  own  experiences.  A Korean  student  chose  to  write  about  her  president  while  another 
Caucasian  student  chose  a female  leader  because  she  said  that  female  leaders  have  been  largely  unheralded.  The 
provision  of  options  gave  students  of  diverse  cultures,  needs,  and  values  the  opportunity  to  situate  their  learning  in 
the  context  of  their  culture  or  values  in  order  to  achieve  deeper  understandings.  As  no  plan  is  perfect,  adjustments  to 
the  course  content  and  schedule  were  necessary  throughout  the  implementation  process.  Reflecting  the  flexibility  of 
this  course  to  meet  the  needs  of  diverse  learners,  approaches,  methods,  and  schedules  were  adjusted  and  alternatives 
provided  when  technological  failures  or  technical  setbacks  occurred  or  to  accommodate  learners'  unforeseen 
personal  problems  and  difficulties. 

In  the  required  end-of-module  learning  reflections,  learners  in  this  curriculum  model  indicated  a few 
effective  strategies  for  online  collaboration  within  this  model.  These  strategies  include:  ensuring  the  technical 
aspects  (such  as  necessary  software  downloads)  are  solved  upon  initial  entry;  employing  frequent  and  cumulative 
check-ins  to  ensure  optimal  participation;  ensuring  adequate  time  for  ice-breaking  and  team-building  to  facilitate 
team  coherence  and  success;  providing  positive,  constructive  feedback  and  encouragement  to  peers;  establishing 
multiple  leadership  roles;  and  setting  meeting  agendas  for  effective  decision-making  and  collaboration.  Students 
also  indicated  that  equally  essential  strategies  essential  for  online  collaborative  success  were  taking  initiatives  on 
tasks,  honestly  and  sincerely  communicating  needs  and  intentions,  and  encouraging  risk-taking  by  celebrating 
multiple  perspectives.  A few  learners  indicated  that  language  is  often  the  major  challenge  for  students  whose  native 
language  is  not  English  because  culturally  related  idioms  and  slang  used.  These  non -native  English-speaking 
students  suggested  that  language  deficiency  is  an  invalid  basis  forjudging  other  because  an  individual’s  talents  are 
simply  masked  by  language  deficiencies.  Indeed,  many  non-native  English-speaking  students  reported  gaining 
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confidence  in  their  communication  abilities  as  the  course  progressed,  illustrating  the  positive  effect  of  collaboration 
in  bringing  out  the  multiple  talents  of  diverse  learners.  Many  students  suggested  that  it  is  important  to  be  sensitive 
to  other’s  needs,  encourage  participation,  and  to  provide  positive  feedback.  Learners  further  suggested  that  regular 
contributions  from  all  group  members  is  essential  to  members*  feelings  of  connection  and  sense  of  community,  as 
well  as  to  the  completion  of  tasks. 


In  light  of  the  global  community  spawned  by  the  World-Wide-Web  and  the  increased  preponderance  of 
global  Web-based  learning  communities  through  cyber  instruction,  this  paper  discusses  why  culture  and 
communication  are  important  considerations  when  designing  and  implementing  Web-based  collaborative  learning 
communities.  Based  on  a Web-based  CSCL  curriculum  model,  this  paper  discusses  some  strategies  employed  by 
the  instructor  and  instructional  designers  to  enhance  the  design  and  implementation  of  the  course  to  optimize 
individual  and  group  learning  among  learners  of  diverse  backgrounds;  to  facilitate  class  communication  as  well  as 
by  learners  of  the  community  to  better  collaborating  with  peers  with  various  patterns  of  thinking,  frames  of  mind, 
and  school  of  thoughts. 

Communication  among  people  of  congruent  cultures,  societies,  and  backgrounds  - or  even  from  the  same 
family  - is  challenging.  Communication  among  people  of  different  cultures,  societies,  and  backgrounds  is  even 
more  challenging.  Collaborative  communication  across  cultures  in  the  online  learning  environment  requires  the 
willingness  of  community  members  to  listen,  to  respect,  and  to  accept  different  perspectives;  to  accommodate  and 
negotiate  in  order  to  reach  shared  meanings;  to  be  flexible  in  their  acceptance  of  ambiguities;  to  provide  mutual 
respect,  trust,  and  support;  to  develop  cultural  sensitivity  and  to  understand  the  value  of  multiple  perspectives;  to 
negotiate  shared  meanings;  to  obtain  mutual  understanding,  and  to  reach  consensus  for  the  achievement  of  the 
shared  goals  and  needs. 

Given  the  diversity  of  the  global  Internet  society,  cultural  sensitivity  and  flexibility  are  essential  to 
collaborative  virtual  classroom  success.  Future  studies  should  focus  on  the  needs  of  learners  from  diverse  cultures 
in  the  design  and  implementation  of  virtual  curriculum  and  in  virtual  classroom  intercultural  communication  and 
collaboration.  Web-based  instructional  designers,  instructors,  and  moderators  should  employ  multiple  approaches 
and  strategies  in  designing,  developing,  and  implementing  their  courses  and  in  assessing  students  - while  always 
keeping  in  mind  the  needs  of  learners  from  diverse  backgrounds  - to  inspire  and  encourage  constructive  work. 
Cultural  sensitivity  may  assist  in  bridging  the  cultural  diversities  and  contributing  to  overall  course  success. 

Building  global  communities  of  diverse  learners  requires  that  courses  not  simply  represent  an  autocratic  instructor's 
curriculum  in  the  absence  of  consideration  of  multiple  needs  and  resources;  the  ideal  curriculumshould  consider 
multiple  perspectives,  provide  multiple  communication  and  learning  styles,  and  allow  a high  level  of  flexibility. 

As  the  Chinese  philosopher  and  educator  Confucius  said,  "Education  should  be  provided  indiscriminately. 
Teaching  should  be  tailored  discriminatelyi"  This  implies  the  duel  notions  of  equal  opportunity  among  learners 
under  instruction  tailored  to  the  needs  of  individual  students.  For  optimum  learning  to  occur,  instructors  should  be 
cognizant  of  cultural  differences,  but  also  should  also  cognizant  of  the  challenges  to  intercultural  communication 
and  the  need  to  be  sensitive  to  the  diverse  needs  of  learners  during  the  process  of  instruction.  For  optimum  learning 
to  occur,  experienced  experts’  strategies  of  tailoring  instructions  to  learners’  needs  and  facilitation  of  group 
interaction  need  to  be  shared. 

Web-based  instructors  of  diverse  student  bodies  face  the  same  challenge  as  Chinese  cooks  — it’s  all  in  the 
mix.  Just  as  Chinese  cooks  need  to  know  the  functions  of  various  ingredients,  instructors  need  to  know  the  strengths 
and  weaknesses  of  various  learners.  Just  as  Chinese  cooks  need  to  know  the  combined  effect  of  various 
components,  instructors  need  to  know  how  to  maximize  the  divergent  abilities  of  learners.  Just  as  Chinese  cooks 
need  to  temper  the  use  of  spices,  temperature,  and  cooking  time,  instructors  need  to  temper  the  pace  and  demands  of 
instructional  strategies.  Without  these  sensitivities,  optimal  performance  cannot  be  attained. 
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Abstract 

This  paper  describes  a constructivist  and  collaborative  approach  to  transforming  a traditional  face-to-face 
on-campus  course  into  an  entirely  Web-based  course.  The  major  objectives  of  this  Computer-Supported 
Collaborative  Learning  (CSCL)  course  are  for  students  to  experience,  learn  and  design  online 
collaborative  learning.  This  paper  is  based  on  the  multiple  experiences  and  perspectives  of  the  course 
instructor,  a course  designer  who  also  served  as  one  of  the  moderators,  and  a student  in  the  course 
The  purposes  of  this  paper  are  to  (I)  describe  the  design,  implementation,  and  evaluation  strategies  used  in 
this  Web-based  collaborative  learning  course,  (2)  examine  the  characteristics  of  the  authentic  environment 
designed  to  enable  students  to  experience  the  strategies,  opportunities,  challenges  and  benefits  of  online 
collaborative  learning,  (3)  discuss  the  roles  and  benefits  of  using  "'e-sherpas  ” as  a unique  support  system 
for  online  learning  teams,  (4)  explore  students'  perspectives,  reflections,  and  suggestions,  and  (5)  discuss 
course  evaluation  results,  lessons  learned,  and  implications  for  design  of  online  collaborative  learning 
environments. 

Introduction 

Just  as  companies  are  expanding  their  markets  globally  through  the  Internet,  so  are  universities  increasingly 
offering  courses  globally  through  the  Internet.  There  has  been  rapid  growth  in  the  number  of  courses  being  offered 
either  entirely  online  or  as  a supplement  to  a face-to-face  course  (Underwood  et  al.,  2000;  McLoughlin,  2000; 
Mugler  & Landbeck,  2000;  Graham  & Scarborough,  1999).  Online  courses  span  a broad  spectrum  of  pedagogical 
practices  from  highly  controlled,  linear,  and  teacher-centered  online  curriculum  to  situated  and  learner-centered 
curriculum.  Most  courses  are  driven  by  the  functionality  and  availability  of  tools  while  others  are  inventing  new 
tools  or  new  approaches  to  meet  the  specifications  of  a particular  curriculum  structure  and  goals. 

Many  online  courses  simply  transfer  face-to-face  classroom  lecture -based  content  onto  online  platforms 
without  considering  interactive  and  human  factors.  Instructors  who  transfer  lecture -based  face-to-face  instruction 
models  to  web-based  environments,  often  tend  to  create  online  courses  with  little  interactivity  or  opportunities  for 
students  to  engage  in  discourse,  collaboratively  solve  problems  and  construct  their  own  knowledge.  Although  the 
value  of  collaborative  learning  is  recognized,  many  instmctors  are  unaware  of  the  strategies  and  processes  for 
building  viable  virtual  learning  teams.  Instructors  and  instructional  designers  need  to  understand  the  nature  of  online 
interactions  and  communications,  the  dynamics  of  group  collaboration,  and  strategies  for  facilitating  online 
interactions  among  diverse  learners  in  order  to  design  effective  web-based  learning  activities. 

The  paper  examined  the  design  and  implementation  of  a Web-based  collaborative  learning  course  that  was 
offered  at  the  University  ofTexas  at  Austin  in  fall,  2000  from  the  perspectives  of  the  course  instructor,  a member  of 
the  instructional  design  team  who  also  participated  in  this  course  as  an  e-sherpa  (e -moderator),  and  a student  who 
completed  the  course.  The  paper  describes  the  specific  design  features  of  the  course  and  examines  the  effectiveness 
of  the  e-sherpa  as  a support  system  for  virtual  learning  teams  based  on  feedback  from  the  students,  instructor  and  the 
e-e-sherpas.  It  also  discusses  the  lessons  learned  and  the  implications  for  design  of  online  collaborative  learning 
environments. 

The  Course  Context 

A major  goal  of  this  CSCL  2000  course  was  to  help  learners  understand,  create,  and  reflect  upon  online 
collaborative  learning.  A variety  of  instructional  material,  resource  links,  and  scaffolding  guidelines  were  made 
available  to  learning  teams  via  the  WebCT  courseware  (Graph  #1).  The  course  content  was  divided  into  seven 
modules,  a course  handbook,  a course  tool  page,  and  a resource  link  page  (Graph  #2).  Each  module  contained  tasks 
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in  which  learners  had  to  work  both  individually  and  collaboratively  to  complete  learning  tasks.  Collaborative  tasks 
included  writing  a topic  paper,  designing  a MOO  (Multi-user  Object  Oriented)  virtual  environment  where  users  log 
onto  a site  to  experience  a text  -based  virtual  reality  environment,  designing  a WebQuest,  and  working 
collaboratively  with  cross-team  members  to  develop  a final  project  utilizing  a schedule  planning  tool  for 
coordination. 

Module  One  provided  an  overview  of  the  course  goals,  objectives,  required  entry  skills,  technology 
requirements,  course  activity  schedule,  and  other  information  helpful  to  the  students  in  preparing  to  complete  the 
course.  Module  Two  provided  opportunities  for  online  socialization  by  introducing  the  mission  of  the  course,  the 
online  environment  and  tools,  and  encouraging  students  to  socialize  with  peers  through  a class-wide  introduction 
activity.  Module  Three,  Four,  and  Five  required  students  to  exchange  information  and  construct  knowledge  through 
online  communication,  collaborative  inquiry,  dialogue  and  discussion,  and  team  building.  Modules  helped  students 
understand  the  unique  aspects  of  online  communication  and  collaboration  and  the  need  for  respect,  honesty  and 
mutual  support.  Students  worked  collaboratively  to  navigate  and  explore  various  network  environments,  utilizing 
collaborative  tools  to  plan,  schedule,  negotiate,  develop,  make  decisions,  and  edit  finished  projects  as  a group.  The 
course  involved  developing  complex  collaborative  documents,  WebQuests  as  well  as  other  intellectual  products 
designed  to  engage  the  students  in  a variety  of  online  collaborative  activities  with  a range  of  collaborative 
technologies.  In  addition  to  the  content  in  each  module,  a course  handbook  was  developed  to  provide  information 
such  as  a virtual  office  tour,  an  organization  chart  about  this  virtual  company,  a staff  directory  which  contained 
information  about  students  in  the  course,  tips  on  working  collaboratively,  topics  for  collaborative  work,  and  project 
examples. 


Gf'ciph  # I:  Course  Web  Site 
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To  accomplish  course  requirements,  online  socialization,  collaboration,  and  communication  were  essential. 
Extensive  cooperation  and  collaboration  among  learners  was  necessary.  Web-based  groupware  tools  offered 
students  opportunities  to  exchange  information,  discuss  tasks,  upload  files,  work  collaboratively  on  the  same 
document,  and  socialize  both  asynchronously  and  synchronously  (Graph  #3).  In  order  to  provide  an  authentic 
environment,  the  metaphor  of  a high  technology  company,  CSCL  Technology  Corporation^  was  employed  to 
simulate  a real  world  professional  setting.  Members  of  the  class  were  divided  into  five  virtual  teams  located  in 
suites.  There  were  two  to  three  offices  within  each  suite.  Two  to  three  students  shared  the  same  office  and  six  to 
seven  students  shared  the  same  suite  (Graph  #4).  Diversity  (ethnic  background,  gender,  and  on-campus  or  tele- 
campus access)  was  considered  when  assigning  students  into  suites  at  the  beginning  of  the  semester. 

Students  in  this  CSCL  course  had  to  interact  with  peers,  the  instructor,  and  e-sherpas.  The  use  of  the  e- 
sherpa  is  one  of  the  key  design  features  of  this  CSCL  class.  There  was  one  e-sherpa  in  each  suite  of  from  5-7 
members.  The  term  sherpa  is  used  to  describe  a skilled  Tibetan  mountain  climber  on  the  southern  slopes  of 
Himalayas.  The  function  of  a sherpa  on  a mountain  climbing  expedition  is  to  familiarize  the  expedition  with  the 
local  terrain  and  to  help  members  of  the  group  carry  their  load.  In  this  course,  the  e-sherpa  was  used  to  describe 
experienced  online  mentors  who  worked  with  online  students  in  groups  and  helped  them  become  self-directed  and 
self-managed  and  to  assist  students  with  any  challenges  they  encountered. 

A monthly  newsletter  was  provided  at  the  course  Web  site.  Students  could  obtain  information  about  the 
regularly -scheduled  Webcast  (a  video  conference  session  where  on-campus  students  can  attend  at  the  campus  site 
and  tele -campus  students  could  attend  through  the  Web).  There  were  a total  of  five  Webcast  sessions  in  this 
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semester  long  course.  During  the  Webcast,  students  could  also  participate  through  the  dial-in  phone  conference 
from  home  or  through  the  synchronous  online  class  chat. 


Graph  # 3:  Course  Discussion  Virtual  Workspace 
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Graph  # 4:  Online  Virtual  Team  - Offices  and  Suites 
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The  intellectual  products  developed  by  the  learning  teams  were  based  on  the  individual  and  collective 
contributions  of  members  of  the  group.  In  accomplishing  collaborative  tasks  such  as  creating  the  topic  paper, 
navigating  a MOO  tour,  producing  a WebQuest,  and  writing  a cross-team  “White  Paper.”  With  the  assistance  of 
various  tools,  learners  not  only  learned  how  to  exploit  the  functionality  of  the  tool  in  working  collaboratively, 
solving  problems,  making  decisions,  and  producing  products.  Throughout  the  course,  learners  reflected  on  their 
learning  at  the  end  of  each  module  through  personal  journaling. 

To  assess  and  improve  students'  performance  and  ensure  learners  are  held  accountable  to  each  other, 
evaluation  rubrics  were  used  throughout  the  course.  Rubrics  are  a type  of  scoring  guide,  used  to  assess  more 
authentic  and  complex  performances  and  products.  According  to  Mary  Rose  (1999),  rubrics  provide  “authentic 
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assessment”  so  the  evaluations  of  student  performance  are  “closer  to  the  challenges  of  real  life  than  isolated  tests.” 
Rubrics  that  “communicate  detailed  explanations  . . . benefit  students  in  making  them  more  conscious  about  their 
own  learning  outcomes  and  process,”  but  also  benefit  instructors  by  providing  them  with  an  objective  basis  for 
assigning  grades  and  involving  students  in  the  evaluation  process. 

The  Medium  is  the  Message 

Two  of  the  major  challenges  in  Web-based  courses  are  faculty  understanding  the  pedagogies  for  engaging 
learners  in  the  online  learning  process  and  the  heavy  demands  on  the  instructor  to  monitor  and  manage  the  course. 
Many  distance-learning  courses  experience  high  attrition  rates  that  result  from  a complex  of  factors  such  as  learners 
feeling  isolated,  unmotivated,  overwhelmed,  or  unchallenged  in  distance  learning  courses.  This  results  in  the  32 
percent  dropout  rate  for  online  courses  as  compared  to  4 percent  for  regular  classroom  classes  (Viers,  Robert). 

In  addition,  it  is  easy  for  faculty  to  get  overwhelmed  by  the  amount  of  work  it  requires  in  developing  Web- 
based  courses.  This  work  involves  developing  the  initial  idea  of  the  online  course  and  re -conceptualizing  the 
instructional  practice  in  order  to  integrate  technology  into  curriculum.  It  involves  extensive  work  in  designing  and 
developing  the  course  content  and  activities,  and  further,  to  implementing  and  evaluating  the  course  so  students  feel 
motivated,  satisfied,  and  engaged.  The  entire  evolutionary  process  is  very  costly  in  terms  of  faculty  and  staff  time 
in  developing  the  course. 

Course  Design:  Constructivist  Theory  for  Authentic  Environment  Design  and  e-sherpas 

The  philosophy  behind  the  design  of  the  CSCL  course  is  Constructivism.  Constructivism  can  be  defined  as 
a theory  of  learning  in  which  knowledge  is  constructed  in  the  mind  of  the  learner.  Jonassen  (1994)  identified  the 
following  elements  of  constructivist  design:  (1)  multiple  representations  of  reality,  (2)  representing  the  complexity 
of  the  real  world,  (3)  emphasizing  on  knowledge  construction  instead  of  knowledge  reproduction,  (4)  emphasizing 
authentic  tasks  in  a meaningful  context  rather  than  abstract  instruction  out  of  context,  (5)  preferring  real  world 
settings  to  predetermined  sequences  of  instruction,  (6)  encouraging  thoughtful  reflection  on  experience,  (7) 
knowledge  construction  should  be  both  content  - and  context-dependent,  (8)  knowledge  construction  should  be 
through  collaboration  rather  than  competition  among  learners. 

Reflecting  Jonassen’s  points  on  constructivist  theory,  various  design  features  were  employed  in  this  CSCL 
course.  As  an  example,  a rich  learning  environment  with  multiple  forms  of  representations  were  provided  to  allow 
learners  with  multiple  intelligences  to  feel  comfortable  in  the  environment  and  to  exhibit  a broader  range  of  their 
talents  and  expertise.  The  provision  of  an  authentic  environment  is  one  of  the  major  messages  this  course  was 
intended  to  deliver.  Both  cognitive  and  social  aspects  of  learning  were  emphasized  in  the  course.  Students  worked  in 
teams  throughout  the  semester  to  complete  tasks.  Multiple  perspectives  were  encouraged.  Authentic  tasks  and 
activities,  reflecting  real-world  settings,  built  in  a coherent  framework  allowed  learners  to  solve  problems 
collaboratively  and  develop  critical  thinking  skills. 

Constructivist  classrooms  are  different  from  traditional  class  settings.  In  this  environment,  students  work 
in  groups  while  individual  contributions  are  highly  valued.  Activities  require  students’  contributions.  Curricula  data 
generated  in  the  environment  are  based  on  dialogue  and  interactions  among  learners.  The  instructor,  rather  than  a 
sage  on  the  stage,  seeks  students’  points  of  views  and  facilitates  student  learning.  The  understandings  obtained  from 
students  are  used  as  references  for  on-going  course  revisions.  The  learners,  rather  than  passive  knowledge 
absorbers,  are  viewed  as  thinkers  and  active  learners.  The  assessment  of  student  learning  is  usually  based  on 
students’  work  and  progress  throughout  the  process  rather  than  paper-pencil  tests  predetermined  by  the  instructor. 
The  features  of  constructivist  classrooms  were  deliberately  built  in  the  online  CSCL  curriculum. 

Brooks  and  Brooks  (1993)  have  identified  characteristic  teacher  and  student  behaviors  that  are  commonly 
found  in  constructivist  classes:  (1)  encouraging  student  autonomy  and  initiative,  (2)  asking  open-ended  questions 
and  allowing  time  for  responses,  (3)  encouraging  higher  level  thinking,  (4)  encouraging  dialogues  between  the 
instructor  and  students  and  among  students,  (5)  challenging  students’  existing  knowledge  and  encouraging 
discussions,  (6)  encouraging  the  use  of  raw  data,  primary  sources,  and  manipulative  and  interactive  materials. 

The  authentic  environment  in  this  Web-based  CSCL  course  engaged  students  in  thoughtful  and 
collaborative  activities.  There  were  a variety  of  ways  team  members  could  communicate  creatively  and  flexibly 
with  each  other  and  with  the  instructor.  As  an  example,  students  collaborated  to  produce  projects  and  make 
decisions  in  pairs,  in  small  groups,  and  across  groups.  The  asynchronous  nature  of  communication  allowed  learners 
to  have  opportunity  for  reflection  and  revise  their  answers  rather  than  reacting  immediately.  The  authentic  activities 
presented  learners  with  various  problems  and  engaged  them  in  various  forms  of  collaboration  and  cognitive 
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challenges.  Further,  multiple  perspectives  among  diverse  learners  challenged  learners’  existing  knowledge  and 
encouraged  them  to  actively  participate  in  conversations  in  order  to  achieve  group  success. 

The  CSCL  class  involved  learners  in  the  cognitive  process  in  three  ways:  (1)  The  course  content  involved 
learning  how  to  use  collaborative  tools,  (2)  The  course  organization  involved  learning  how  to  work  collaboratively 
with  others  in  complex  learning  tasks,  (3)  The  course  location  involved  learning  how  to  work  at  a distance, 
synchronously  as  well  as  asynchronously. 

For  better  course  management  and  learner  support,  the  concept  of  the  “e-sherpa,”  coined  by  the  instructor. 
Dr.  Paul  Resta,  was  employed  to  function  similarly  to  the  e-moderators  described  by  Salmon  (2000).  Gilly  Salmon 
used  a few  case  studies  to  describe  the  e-moderators’  role  and  responsibility.  As  an  example,  Quantas  College 
Online  (QCO),  an  online  course  established  in  1996  for  the  purpose  of  training  all  staff  within  Quantas  Airways, 
expected  e-moderators  to  welcome  and  encourage  participants,  monitor  learning  progress,  provide  feedback, 
facilitate  learning  sessions,  moderate  discussions,  provide  input,  assess  learning  outcomes,  maintain  records,  and 
update  status  of  participants. 

In  this  Web-based  CSCL  course,  an  e-sherpa  represents  a non-authoritarian  support  person  who  climbs  the 
mountain  with  students  to  help  them  accomplish  their  goals  rather  than  directing  and  leading  students.  The  role  of 
the  e-sherpa  differs  significantly  from  the  traditional  authoritative  leadership  figure.  Much  like  real  Tibetan  sherpas, 
the  e-sherpa  in  this  course  context  is  neither  that  of  serving  as  the  leader  of  the  team  nor  as  the  authoritarian  figure 
of  the  group.  Their  role  was  to  simply  accompany  learners  and  to  help  the  online  team  carry  the  load  and 
accomplish  its  learning  goals.  In  this  course,  e-sherpas  were  a group  of  graduate  students  who  had  prior  knowledge 
of  the  similar  course  structure,  experience  in  working  in  online  teams,  good  communication  skills,  and  who  were 
interested  in  the  online  learning  environment.  Some  of  the  e-sherpas  were  employed  at  the  CSCL  course  instructor- 
directed  computer  lab  while  others  participated  on  a volunteer  basis  online. 

The  e-sherpas’  roles  and  responsibilities  included  unobtrusive  approach  to  participating  and  supporting 
learners,  assisting  learners  when  they  encounter  technical  difficulties,  clarifying  assignments  when  necessary, 
providing  emotional  support  as  needed,  communicating  and  mediating  conflict  within  the  team,  helping  carry  the 
load  when  emergencies  arise,  providing  positive  feedback  and  encouragement,  monitoring  learning  progress  and 
sharing  with  the  instructor  and  fellow  sherpas. 

Students’  Perspectives:  Issues  related  to  course  design  and  implementation 

The  design  and  implementation  of  the  course  required  extensive  monitoring,  feedback  and  on-the-fly 
revision  of  any  uncovered  problems  such  as  broken  link.  Course  revision  was  an  on-going  process  and  was  based  on 
the  analysis  on  progress  in  course  by  the  instructor  and  instructional  designers  as  well  as  understanding  learners’ 
perspectives,  experiences,  reflections,  and  suggestions.  The  following  sections  provide  a summary  of  the  feedback 
from  students  in  end-of-semester  course  evaluations  which  were  based  on  a 5-point  scale  measurement  and  issues, 
perspectives,  experiences,  and  reflections  discussed  from  the  perspective  of  one  of  the  authors,  who  was  a student  in 
the  course. 

General  Survey  Results 

• 1 am  very  satisfied  with  the  course:  3.95 

• 1 have  learned  a lot  in  the  course:  4.29 

• 1 enjoyed  the  course:  4.05 

• The  course  was  very  stimulating  for  me:  4.33 

• My  cooperative/  collaborative  skills  have  improved:  4.33 

• 1 would  like  to  participate  in  other  online  courses:  4.71 

• By  participating  in  CSCL  activities,  I developed  new  perspectives  on  learning:  4.38 

A Students’  Reflections 
1.  Procedural  issues 

The  flexibility  of  the  FirstClass  groupware  to  provide  e-mail,  collaborative  documents,  chat,  and  individual 
and  group  folders  made  it  an  ideal  tool  for  use  in  collaborative  learning.  The  FirstClass  collaborative  environment 
was,  however,  new  to  most  students,  and  the  business  metaphor  of  suites  and  offices  was  also  unfamiliar  and  at 
times  confusing.  As  part  of  course  requirements,  students  were  expected  to  locate  or  deposit  reflections, 
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collaborative  and  individual  project  contributions,  completed  projects,  introductory  exercises,  and  work  in  progress 
documents  in  specific  places  within  the  virtual  office.  When  students  needed  clarification  to  negotiate  the 
complexities  of  the  site,  the  sherpa  was  contacted  and  proved  useful  in  clarifying  these  locations  for  the  teams, 
especially  at  the  beginning  of  the  course. 

In  addition,  the  office  metaphor  was  extended  with  the  addition  of  a CSCL  Announcements  bulletin  board 
icon,  an  icon  for  Tech  Assistance,  and  a CSCL  Cafe  icon  for  informal  comments  unrelated  to  specific  course 
concerns.  The  service  provided  by  the  Announcements  feature  was  evident  to  all  students,  but  the  purposes  of  the 
Cafe  and  Tech  Assistance  services  were  sometimes  misunderstood.  On  occasion  the  professor  or  the  e-sherpa  was 
sent  questions  that  should  have  been  directed  to  technical  support,  and  the  questions  had  to  be  forwarded  to  the 
proper  technical  support  person.  In  the  case  of  the  Cafe,  discussions  posted  there  were  sometimes  course-specific 
and  the  class  was  reminded  to  exchange  these  comments  in  another  location  within  their  suite  or  office. 

2.  Third-party  issues 

The  CSCL  course  employed  several  collaborative  tools.  As  already  mentioned.  First  Class  trouble-shooting 
questions  were  sometimes  misdirected  to  other  persons  instead  of  to  the  technical  support  folder.  In  the  case  of  the 
other  collaborative  tools,  in  particular  the  various  MOOs,  Zebu,  and  the  WebQuest  building  site,  there  was  no  clear 
source  of  technical  support  and  students  often  spent  more  time  trying  to  negotiate  the  pitfalls  of  the  sites  than  they 
did  working  on  some  of  the  assignments.  Some  students  experienced  frustration  and  the  pressure  of  meeting 
deadlines  as  they  struggled  with  these  sites.  Of  necessity,  students  established  informal  networks  of  assistance  in 
which  solutions  to  common  problems  were  shared.  A FAQ  icon  within  the  office  graphic  or  one  or  more  e-sherpas 
with  special  expertise  would  have  been  of  great  assistance  here  as  expert  resources  to  consult  about  the  sites  and 
their  potential  problems  so  students  could  more  quickly  use  the  resources. 

3.  Assignment  clarification 

Consistent  with  a constructivist  philosophy  the  assignments  were  open-ended,  varied,  and  not  cumulative, 
and  so  each  one  had  specific  and  unique  features  that  the  team  members  had  to  establish  and  clarify  among 
themselves  before  and  during  their  collaborative  work.  Students  had  to  determine  not  only  the  specific  content  the 
assignment  was  requesting,  but  also  the  format  it  was  to  take,  how  it  was  to  be  organized,  which  parts  were  to  be 
generated  collectively,  and  which  were  to  be  generated  individually  and  then  collaboratively  refined.  They 
employed  informal  on-going  formative  evaluations  as  their  work  evolved  and  became  more  complex.  There  were 
project  rubrics,  which  were  ambiguous  at  times  as  assignment  requirements  were  modified,  as  well  as  peer 
evaluation  rubrics  against  which  assessed  students’  contributions  to  the  project.  E-sherpas  or  the  instructor  were 
asked  questions  regarding  the  assignment  components,  and  they  responded  with  further  explanations  for  teams  or 
individuals. 

4.  Interpersonal  issues 

Students  were  placed  in  learning  teams  to  assure  diversity  and  balance  in  culture,  knowledge  of  the  content, 
and  technology  expertise.  Occasionally,  conflicts  among  team  members  arose  related  to  differing  expectations  with 
regard  to  work  or  to  interpersonal  interactions.  In  order  to  successfully  complete  assignments,  it  was  important  for 
group  members  to  take  responsibility  for  participation  in  group  work  sessions  and  to  meet  deadlines.  Sometimes 
work  issues  related  to  differences  in  quality  or  quantity  of  work  arose  in  teams . Individuals  and  the  group  as  a whole 
were  challenged  to  negotiate  mutually  satisfactory  conventions  that  enabled  cooperation,  compromise,  and  the 
giving  and  receiving  of  constructive  criticism.  These  skills  were  vital  for  the  development  of  quality  team  products 
and  projects  as  well  as  timely  completion  of  the  project.  The  course  design  provided  for  the  rotation  of  leadership 
roles  among  team  members..  This  allowed  each  team  to  find  its  own  way  of  working  best  while  giving  each 
individual  an  opportunity  for  carrying  greater  responsibility  for  a while. 

It  was  additionally  important  for  group  morale  for  members  to  be  understanding,  supportive,  and 
encouraging  of  the  work  of  other  team  members.  In  a few  instances,  individuals  were  unable  or  unwilling  to  serve  as 
emotional  as  well  as  intellectual  partners  with  their  team  members.  In  those  instances,  the  team  had  greater  difficulty 
in  accomplishing  its  learning  tasks.  For  the  team  to  be  successful,  it  had  to  find  ways  to  resolve  all  of  these  issues 
and  reach  consensus.  Over  the  entire  period  of  the  course,  a general  ecology  of  support  emerged  with  each  member 
contributing  from  his/her  strength  as  roles  emerged  for  leading  in  a variety  of  ways.  The  e-sherpas  were  consulted 
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less  and  less  and  usually  asked  for  help  only  when  there  was  an  impasse.  At  this  stage,  the  problems  were  then 
addressed  and  resolved  by  the  instructor. 

Suggestions 

E-sherpas  have  the  potential  to  serve  several  useful  functions  in  developing  and  supporting  online  learning 
communities.  The  responsibilities,  however,  may  require  special  knowledge  or  training  in  order  for  them  to  meet  all 
of  the  challenges  in  helping  virtual  learning  teams  be  successful. 

1.  Course  assignments  should  be  made  as  clear  as  possible,  taking  into  consideration  that  in  real 
world  settings  a considerable  amount  of  ambiguity  is  inevitable.  The  e-sherpa  must  have  a clear 
understanding  of  the  course  assignment  components  in  order  to  clarify  them  to  the  team  while 
avoiding  offering  editorial  comments  on  the  work  products  themselves.  Obviously  this 
responsibility  requires  that  the  e-sherpas  be  kept  up  to  date  on  any  changes  to  the  course  and  its 
assignments  that  are  made  as  the  course  progresses. 

2.  A source  of  practical  information  about  the  courses’  collaborative  tools  would  be  an  important 
element  in  the  technical  support  network  and  would  fill  a need  not  met  by  the  courseware 
technical  support  staff.  Team  members  could  avoid  frustration  and  lost  time  if  they  could 
consult  a FAQ  or  e-sherpa  with  the  ability  to  facilitate  their  use  of  the  collaborative  tools. 

3.  For  many  students  new  to  online  collaborative  learning,  some  direct  instruction  or  required 
reading  about  forming  collaborative  teams  might  ease  their  transition  from  the  tradition  of 
individuals  working  competitively  to  teams  working  collaboratively  in  a constructivist 
classroom.  An  e-sherpa  with  skills  in  mediating  collaborative  working  interactions  would  be 
useful  early  in  the  course  to  assist  the  learning  team  in  working  through  the  awkward  period  of 
establishing  individual  identities  within  the  group  ecology.  Such  assistance  might  forestall  the 
need  for  more  critical  guidance  later  by  establishing  a well- functioning  team  from  the  start.  If 
difficulties  arose  later,  a working  relationship  with  a trusted  and  skilled  e-sherpa  might  make 
resolution  of  the  problems  easier. 

4.  Much  of  what  the  teams  devoted  themselves  to  might  more  correctly  be  thought  of  as 
collaborative  work  rather  than  collaborative  learning  although  all  boundaries  between  the  two 
are  often  unclear.  Through  guiding  conversations  throughout  the  course,  an  instructor  or  e- 
sherpa  could  provide  a way  for  team  members  to  become  aware  of  how  the  course  is  structured 
to  facilitate  collaborative  learning,  consider  what  other  collaborative  strategies  they  might  try, 
and  examine  the  features  of  effective  communication  within  a collaborative  endeavour.  Difficult 
decisions  would  need  to  be  made  to  prioritise  existing  course  goals,  content,  and  activities  to 
build  in  the  time  and  opportunity  for  these  interactions  with  team  members  to  take  place. 

5.  Feedback  in  the  course  for  team  projects  as  well  as  for  individual  contributions  and  reflections 
was  relatively  sparse.  One  feature  of  constructivist  theory  is  time  for  reflection  on  the  learning 
process.  While  academic  feedback  is  the  provenance  of  the  instructor,  if  the  e-sherpas  were 
deeply  engaged  in  observing  the  interactions  within  a team,  he/she  would  be  uniquely  situated 
to  give  astute  feedback  to  the  team  on  the  nature  of  its  collaborative  interactions  and  to  draw 
members  into  conversation  about  what  they  were  learning.  Using  the  e-sherpa  merely  to  convey 
observations  on  the  workings  of  the  learning  teams  and  their  academic  products  is  useful  but 
does  not  capitalize  on  the  full  potential  of  the  e-sherpa  in  supporting  the  learning  of  the 
members  of  the  learning  community. 

6.  At  its  best  collaborative  learning  can  produce  a learning  experience  and  artifacts  of  that 
experience  that  are  greater  than  what  an  individual  student  could  achieve  working  alone.  The 
time  and  opportunity  for  risk-taking  are  essential  for  reaching  this  higher  level  of  achievement. 
In  this  particular  course  risk-taking  on  the  part  of  individuals  or  teams  was  thwarted  in  a number 
of  ways.  The  time  allowed  for  assignment  completion  did  not  provide  a cushion  for  false  starts 
or  revamping  common  in  risk-taking  situations.  Fewer  assignments  or  more  relaxed  time 
requirements  might  alleviate  this  problem.  Instructor  feedback  and  its  associated  reassurance 
and  encouragement  are  also  an  intangible  support  for  risk-taking  attitudes  among  students.  The 
course  needs  a faculty  presence  through  feedback  communication  with  teams  and  with 
individuals  more  frequently  and  more  specifically  to  encourage  students  to  test  their  potential. 
Finally,  the  frequent  peer  assessment  feature  of  the  course  discouraged  vigorous  debate  or 
discussion  that  might  have  resulted  in  the  group  undertaking  a project  that  contained  more  risk. 
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Team  members  were  cautious  in  order  not  to  be  misunderstood  and  evaluated  as  uncooperative 
or  unsupportive  of  the  group. 

To  assure  individual  accountability,  team  members  assessed  their  own  contributions  as  well  as  those  of  all 
other  team  members  at  the  end  of  each  learning  module.  The  positive  aspect  of  the  peer  evaluations  was  that  it 
helped  individual  team  members  who  were  late  in  contributing  their  work  or  participating  minimally  in  collaborative 
task  to  become  more  active  members  of  the  learning  team.  It  is  also  possible  that  the  frequent  peer  assessments 
discouraged  vigorous  debate  or  discussion  that  might  have  resulted  in  the  group  undertaking  a project  that  contained 
more  risk.  There  was  evidence  that  some  team  members  were  cautious  in  order  not  to  be  misunderstood  and 
evaluated  as  uncooperative  or  unsupportive  of  the  group. 

Lesson  Learned  and  Future  Implications 

Many  instructors  believe  that  online  learning  can  have  a very  bright  future  if  courses  are  designed  well. 

Yet,  merely  transferring  existing  curriculum  online  and  grouping  students  without  regard  to  interpersonal  dynamics 
will  not  achieve  ideal  student  participation  and  collaboration.  Instructors  and  instructional  designers  need  to  truly 
understand  the  nature  of  online  interactions  and  communications,  the  dynamics  of  group  collaboration,  and 
strategies  for  facilitating  online  interactions  among  diverse  learners  in  order  to  design  activities  for  effective 
interactions. 

Students  and  e-sherpas  made  various  design  and  implementation  suggestions  at  the  end  of  the  semester. 
Some  of  these  suggestions  included: 

• Building  trust  by  providing  ice-breaking  sessions  at  the  beginning  of  the  semester 

• Providing  e-sherpas’  training  prior  to  the  semester 

• Clarifying  e-sherpas’  roles  and  responsibilities  prior  to  the  semester 

• Standardizing  e-sherpas’  monitoring  process  of  learning  teams’  progress 

• Standardizing  e-sherpas*  observation  and  report  guidelines 

• Specifying  assignments  and  providing  immediate  clarifications  of  students’  questions 

• Linking  design  goals  and  implementation  processes  seamlessly 

• Valuing  the  social  aspects  of  learning  equally  to  cognitive  aspects 

• Clarifying  leadership  roles  by  providing  explicit  guidelines 

• Considering  the  number  of  tools  employed  carefully  to  avoid  overwhelming  students 

• Avoiding  holiday  assignment  deadlines  by  examining  the  course  schedule 

• Employing  tasks  as  building  blocks  toward  students’  final  projects 

• Improving  the  immediacy  of  feedback  for  technical  and  instruction-related  questions 

• Providing  frequent  and  constructive  instructor  feedback  on  students’  learning  progress 

• Assisting  students  by  conducting  virtual  office  hours  regularly 

• Providing  self-check  quizzes  in  assisting  learners  in  tracking  their  learning  progress 

• Surveying  and  identifying  students’  concerns  and  suggestions  regularly 

• Adjusting/revising  course  based  upon  identifying  student  concerns  and  suggestions 

As  one  of  the  pioneers  in  online  collaborative  learning  course  design,  we  still  have  much  to  learn  about  the 
design,  implementation,  and  evaluation  to  achieve  optimal  results  in  Web-based  learning.  The  dropout  rate  of  this 
CSCL  course,  however,  was  only  two  out  of  32  students  in  class  and  the  overall  course  evaluations  were  very  high.. 
We  understand  that  technology  tools  can  only  provide  high  quality  in  the  hands  of  good  teachers.  Through  the 
sharing  of  our  experiences,  good  practices,  perspectives,  reflections,  and  suggestions,  we  hope  to  enhance  online 
instruction  in  higher  education. 
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Abstract 

With  the  wide  acceptance  of  the  Internet  as  an  information  and  communications  source,  students  have 
come  to  expect  it  to  supply  supplemental  course  material  and  other  enhancements  to  their  traditional 
classroom  activities.  Students  have  grown  accustomed  to  the  'anytime  / anywhere'  capability  and 
flexibility  offered  by  the  World  Wide  Web.  This  media  will  become  a vital  exchange  mechanism  between 
educators  and  students.  However,  the  technologies  that  support  the  Web  are  very  different  from  those 
traditionally  used  in  the  preparation  and  delivery  of  course  content.  This  can  make  entry  into  the  Internet 
course  support  arena  a challenge  for  educators  not  familiar  with  its  inner  workings. 

The  same  Internet  acceptance  and  usage  expectation  that  is  forcing  educators  into  the  World  Wide  Web 
has  also  provided  us  with  some  simple  tools  and  techniques  that  can  be  employed  to  begin  the  integration 
of  the  Web  into  course  support.  The  Web  has  become  so  integrated  into  all  aspects  of  computer  usage  that 
most  personal  productivity  applications  provide  Web  tools  as  part  of  the  software.  The  degree  of  this 
integration  varies  with  the  application  and  the  vendor,  but  these  rudimentary  Internet  tools  exist  in  much  of 
the  software  educators  use  in  course  preparation  and  delivery.  Through  the  application  of  these  simple 
tools  to  the  documents,  spreadsheets,  communications,  and  presentation  currently  generated,  they  can 
easily  be  made  available  to  students  through  a Web  page.  Lecture  notes,  presentation  slides,  study  guides, 
practice  quizzes,  solutions,  and  homework  guides  are  just  a few  of  the  documents  educators  generate  with  a 
word  processor.  These  can  easily  be  converted  to  a format  that  is  viewable  over  the  Web.  Grade  postings 
and  other  data  generated  with  a spreadsheet  can  be  made  available  via  the  Web  in  a similar  manner. 

The  on-demand  availability  of  the  Internet  offers  several  advantages.  Students  can  access  information 
when  they  are  ready  to  study,  not  when  the  instructor  is  available  to  present  it.  Students  can  access  the 
specific  information  needed  to  finish  a specific  assignment.  As  a result  of  this  information  availability,  the 
instructor  will  spend  less  time  redistributing  lost  or  missed  information,  and  less  time  answering  the  same 
question  from  several  different  students. 

On-demand  information  availability  is  beneficial  to  both  traditional  and  non -traditional  learners. 
Traditional  students  maintain  the  structure  and  control  of  traditional  instructional  methodologies  while 
having  increased  access  to  support  information.  Non-traditional  students  are  provided  a mechanism  that 
places  them  personally  in  control  of  what  they  learn,  and  when  they  learn  it.  This  control  is  a necessary 
component  of  successful  learning  in  the  non-traditional  student  population. 

The  Internet  offers  the  student  more  timely  access  to  specific  course-related  information  with  less  effort  on 
the  part  of  the  instructor.  Add  in  the  wide  acceptance  and  availability  of  the  Web,  and  new  opportunities 
for  course  and  learning  improvement  become  evident.  With  minimal  effort,  the  Internet  can  enhance 
almost  any  course. 

Introduction 

The  Internet  is  a widely  accessible,  on-demand  source  of  information.  Students  have  come  to  rely  on  the 
Internet  for  most  of  their  information  gathering  activities.  They  go  to  the  Internet  for  information  on  anything  from 
the  configuration  of  a networking  cable  to  the  show  times  at  the  local  movie  theater.  Three  fourths  of  all  college 
seniors  use  the  Internet  in  their  job  search.  (Shelly  2000)  Given  this  wide  acceptance,  it  is  reasonable  to  assume  that 
students  will  also  look  to  the  Internet  for  course  related  materials  such  as  lecture  notes,  assignments,  grade  postings, 
and  even  general  communications. 

Many  institutions  have  a Web  presence  that  is  maintained  by  the  institution.  Frequently,  the  homepage  of 
the  institution’s  website  will  have  links  to  individual  department  pages.  These  departmental  pages  will  in  turn  have 
links  to  individual  course  pages.  The  institution  will  often  provide  a page  for  each  course  based  on  a common 
temple  that  is  used  for  all  courses  institution-wide.  This  common  template  often  includes  standard  links  to  common 
course  page  content  such  as  grades  and  class  notes.  The  faculty  associated  with  a specific  course  are  usually  given 
access  privileges  to  the  folders  that  contain  the  Web  pages  for  that  course,  thus  allowing  them  to  create  and  maintain 
course  related  Web  content. 
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The  Web  presence  for  the  course  is  in  place.  However,  course  information  is  available  on  the  World  Wide 
Web  only  if  the  course  instructor  makes  it  available.  The  tools  and  technologies  that  have  traditionally  been  used  to 
create  World  Wide  Web  content  are  different  from  those  used  to  create  normal  course  materials.  Learning  these 
new  tools  requires  an  investment  of  time  that  many  faculty  cannot  justify.  For  this  reason,  many  faculty  do  not 
support  their  courses  on  the  Web  even  though  a Web  presence  is  already  available. 

The  same  wide  acceptance  of  the  Web  as  the  de  facto  information  source  has  encouraged  many  software 
developers  to  include  Web  support  features  in  applications  that  have  not  traditionally  been  associated  with  Web 
content  development.  Currently,  many  of  the  applications  that  have  traditionally  been  used  in  course  material 
development  include  basic  Web  content  development  features.  Personal  productivity  tools  like  word  processors, 
spreadsheets,  and  presentation  software  now  provide  simple  tools  to  generate  Web  content.  With  a basic 
understanding  of  the  organization  of  a Web  page,  browser  operation,  and  the  tools  available  in  current  personal 
productivity  software,  most  faculty  can  develop  Web  content  to  support  their  courses. 

Web  Page  Organization 

The  World  Wide  Web  is  actually  a wide  area  network  of  over  4 million  loosely  connected  servers  that 
provide  information  upon  request.  (Goldman  1 997)  These  servers  hold  the  files  that  create  Web  pages.  When  the 
server  receives  a request  to  view  a Web  page,  the  server  downloads  the  files  needed  to  create  the  page  to  the 
requesting  computer. 

Each  Web  server  is  identified  by  a unique  name  called  a URL,  short  for  Uniform  Resource  Locator.  A 
URL  consists  of  protocol  specification,  a domain  name,  and  optional  path  to  a specific  Web  page.  URLs  are 
specified  using  the  familiar  format  http://www.domain.com/path/page.htm^  In  this  example  the  protocol  is  http, 
which  stands  for  hypertext  transfer  protocol.  Hypertext  transfer  protocol  is  the  communications  standard  (protocol) 
for  transfering  Web  pages.  The  domain  is  represented  by  www.domain.com,  and  /path/page.htm  illustrates  the 
format  for  the  optional  page  reference.  In  an  actual  Web  page  reference,  the  domain  (www.domain.com)  and  the 
optional  page  specification  (/path/page.htm)  would  be  replaced  with  the  domain  name  of  the  Website  being 
accessed  and  the  (optional)  path  and  file  name  of  the  desired  page. 

Each  domain  must  be  registered  with  an  organization  that  is  responsible  for  assigning  maintaining  domain 
information.  Each  domain  is  associated  with  a specific  location  on  the  World  Wide  Web  called  its  Web  address. 
These  addresses  are  composed  of  four  three-digit  numbers  separated  by  periods  (i.e.  230.195.013.212).  (Goldman 
2001)  Domain  information  is  kept  in  databases  at  specific  locations  accessible  to  the  World  Wide  Web.  When  a 
user  requests  to  view  a website,  browser  software  on  the  user’s  computer  references  the  databases  to  determine  the 
Web  address  of  the  website  in  question.  A request  is  sent  to  the  Web  server  at  the  address  obtained  from  the  domain 
database.  The  Web  server  at  that  address  equates  the  domain  to  a specific  folder  located  on  the  server.  It  locates  a 
file  in  the  designated  folder  titled  INDEX.HTM  and  sends  that  file  to  the  requesting  computer.  This  file  represents 
the  homepage  for  that  website.  The  Web  homepage  may  be  titled  HOME.HTM  on  some  system.^ 

Additional  pages  in  the  website  are  usually  arranged  in  folders  that  are  subordinate  to  the  folder  that 
contains  the  homepage.  The  folder  system  is  used  to  organize  additional  pages  that  are  referenced  through  the  home 
page  in  the  same  way  that  folders  are  used  to  organize  files  on  a drive.  System  administrators  frequently  use  this 
type  of  folder  arrangement  to  allow  certain  users  access  to  certain  areas  of  the  website,  without  giving  all  users 
access  to  all  pages  of  the  website.  Ideally,  the  instructors  are  given  access  to  folders  that  contain  information 
relevant  to  the  courses  they  teach.  This  access  allows  the  appropriate  faculty  to  post  course  materials  to  the  website. 

The  Web  presence  may  be  maintained  internally,  or  it  may  be  outsourced.  If  maintained  in-house,  the  Web 
files  frequently  reside  on  a server  within  the  institution.  If  the  folders  that  contain  the  website  files  are  accessible 
through  the  institution’s  network,  it  is  possible  to  add  content  to  the  website  directly,  provided  the  user  has  been 
granted  access  privileges  to  the  necessary  folders.  If  access  privileges  are  not  available  to  the  course  instructors  or  if 
the  website  is  maintained  off -site,  the  institution  will  designate  a specific  individual,  sometimes  call  the  Webmaster, 
to  post  all  files  to  Web  folders.  This  is  done  to  ensure  that  all  postings  adhere  to  institutional  policies  for  Web 
content,  and  to  maintain  security.  If  the  institution  allows  only  the  Webmaster  to  access  the  system,  then  all 
postings  to  the  Web  must  be  forwarded  to  the  Web  Master.  The  institution  should  have  a mechanism  in  place  to 
handle  posting  requests. 

The  URL  will  allow  a browser  to  find  a specific  server  on  the  Web.  By  including  the  file  and  path  in  the 
URL  any  file  within  that  Web  server  can  be  accessed,  even  if  there  are  no  links  from  existing  Web  pages  to  that  file. 
Faculty  can  place  information  on  the  Web,  by  either  directly  accessing  the  Web  folders  or  by  requesting  postings 
from  a Webmaster.  Once  the  file  is  posted  in  the  website,  students  can  be  given  the  complete  URL  to  the  file. 
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Students  can  access  the  file  by  entering  the  URL,  complete  with  the  path  and  file,  into  the  browser.  Links  from 
other  Web  pages  are  not  needed. 

For  example,  let  us  place  the  syllabus  for  a course  numbered  CPT250  on  the  Web.  The  original  syllabus  is 
saved  under  the  name  Syl250Fall01.htm  in  the  My  Documents  folder  on  the  local  C:/  drive.  The  Web  site  domain 
is  pua.edu.  The  files  used  to  generate  this  website  are  located  on  a network  drive  designated  W:/.  The  CPT250 
Web  page  is  located  in  the  /courses/cpt/cpt250/  folder.  Since  a folder  already  exists  for  CPT250,  we  will  place  the 
syllabus  in  it.  You  Webmaster  should  be  able  to  provide  information  on  the  folder  layout  for  your  Web  site.  Begin 
by  moving  the  syllabus  in  the  Web  folder.  Use  Windows  Explorer  to  copy  the  file  Syl250Fall01.htm  from  the  My 
Documents  folder  on  the  C:/  drive  to  the  /courses/cpt/cpt250/  folder  on  the  W:\  drive.  The  last  step  is  to  provide 
students  with  the  complete  URL  of  the  page,  http://www.pua.edu/courses/cpt/cpt250/Syl250Fall01.htm  Students 
can  access  the  syllabus  by  entering  this  URL  into  a browser. 

Many  institutions  provide  a common  template  for  course  Web  pages.  The  common  template  promotes  a 
consistent  look  and  feel  to  the  Website,  and  also  helps  keep  individual  course  pages  consistent  with  Web  policies. 
The  Webmaster  creates  a Web  page  for  each  course  from  the  template.  Links  are  provided  to  the  individual  course 
pages  from  the  home  page.  The  template  often  contains  links  to  such  common  components  as  grades,  syllabi,  and 
class  notes.  Exploiting  these  available  links  will  ease  student  access  to  the  information. 

The  key  to  using  a predefined  link  from  an  existing  page  is  knowing  what  the  link  accesses.  Links  are 
actually  instructions  to  the  browser  to  retrieve  and  display  another  Web  page.  Recall  that  Web  pages  are  contained 
in  files.  Thus,  a link  contains  the  name  of  the  file  to  be  accessed.  When  a file  of  that  name  is  placed  in  the 
appropriate  folder,  the  link  will  retrieve  and  display  the  information  from  that  file.  Predefined  links  can  be  used 
only  if  the  file  name  and  path  of  the  file  the  link  accesses  are  known.  The  Webmaster  should  be  able  to  provide  this 
information  or  post  files  to  the  proper  locations  for  you  if  access  is  restricted. 

The  file  name  is  defined  in  the  link  when  the  Web  page  is  created  and  is  not  easily  changed.  However,  it  is 
easy  to  change  the  name  of  a file.  Changing  the  name  of  a file  to  match  that  specified  in  the  link  will  cause  the 
browser  to  display  the  information  in  that  file.  As  an  example,  let  us  assume  that  a faculty  member  has  a syllabus 
for  CPT250  that  needs  to  be  posted  to  the  CPT250  Web  page.  The  original  syllabus  is  saved  under  the  name 
Syl250Fall01.htm in  the  My  Documents  folder  on  the  local  C:/  drive.  The  institution’s  Web  site  has  a page  for 
CPT250  with  a Syllabus  link  that  displays  a file  called  Syllabus.htm.  The  website  domain  is  pua.edu.  The  files 
needed  to  generate  the  website  are  located  on  a network  drive  designated  W:/.  The  CPT250  Web  page  is  located  in 
the  /courses/cpt/cpt250/  folder.  Again,  we  will  place  the  document  there.  First,  rename  the  Syl250Fall01.htm  file 
to  Syllabus.htm,  the  name  of  the  file  the  Syllabus  link  will  access.  Next,  copy  the  file  into  W:/courses/cpt/cpt250, 
the  folder  the  link  accesses.  Use  Windows  Explorer  to  complete  the  rename  and  copy  operations.  If  a file  named 
Syllabus.htm  already  exists  in  the  /courses/cpt/cpt250  folder  you  will  be  prompted  to  replace  it.  Once  the  file  has 
been  successfully  copied  into  the  /courses/cpt/cpt250  folder,  clicking  the  Syllabus  link  will  retrieve  and  display  the 
syllabus. 

The  Browser 

The  browser  is  the  application  that  allows  a user  to  view  information  downloaded  from  Web  servers. 
(Walters  2001)  The  user  enters  the  URL  of  the  website  to  be  viewed.  The  browser  locates  the  Web  address  from 
the  domain  database  and  sends  a request  to  the  appropriate  Web  server.  The  Web  server  downloads  the  requested 
files  to  the  browser  and  the  browser  generates  and  displays  the  web  page.  The  user  must  enter  the  exact  URL  in 
order  for  the  browser  to  find  the  desired  Web  server. 

Browsers  use  the  data  contained  in  the  downloaded  files  to  generate  the  Web  page.  This  data  must  be 
presented  to  the  browser  in  a format  the  browser  understands.  If  the  browser  cannot  understand  the  data  presented, 
it  cannot  generate  a Web  page  from  it.  The  most  common  file  format  for  generating  text  in  a Web  page  is  the 
HTML  format.  Hyper  Text  Markup  Language  is  a formalized  set  of  rules  for  specifying  how  text  will  be  displayed. 
HTML  adds  additional  information  (called  tags)  to  the  text  that  instructs  the  browser  on  how  the  text  should  be 
displayed.  As  the  text  is  downloaded,  the  tags  are  interpreted  and  the  text  is  displayed  accordingly. 

Browsers  can  also  display  images  in  a Web  page.  Again,  the  data  used  to  generate  the  image  must  be 
presented  to  the  browser  in  a format  that  it  understands.  Most  browsers  understand  many  graphical  formats.  Some 
of  the  more  frequently  encountered  graphical  file  formats  are  GIF,  JPG,  and  PNC.  Browsers  can  also  generate  audio 
from  files  using  the  WAV  or  MIDI  format,  and  full  motion  video  from  A VI,  MOV,  and  MPG  formatted  files. 
(Cadenhead  1999) 

The  browser  can  also  provide  limited  access  to  files  that  are  not  in  a format  that  it  can  understand.  When 
the  browser  is  asked  to  access  a file  in  a type  it  does  not  recognized,  it  will  offer  the  user  the  option  if  opening  the 
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file  or  saving  it  to  a local  drive.  If  the  user  opts  to  open  the  file,  the  browser  will  hand  the  task  over  to  the  local 
operating  system.  The  operating  system  will  check  the  file  extension  to  determine  if  it  is  a file  associated  with  a 
specific  application  on  the  computer.  If  a corresponding  application  is  found,  then  the  operating  system  will  start 
that  application  and  open  the  file  in  it.  If  the  operating  system  does  not  find  an  application  that  it  associates  with  the 
file  extension,  it  will  open  a dialog  box  and  ask  the  user  to  select  the  application  to  use  to  access  the  file.  If  the  save 
option  is  selected  instead  of  the  open  option,  the  operating  system  starts  a download  wizard  that  allows  the  user  to 
select  a location  and  name  on  a local  drive  where  the  file  will  be  saved. 

Generating  Web  Content;  Common  Personal  Productivity  Tools 

The  wide  acceptance  and  use  of  the  World  Wide  Web  has  affected  much  of  the  personal  productivity 
software  currently  available.  Many  of  the  tools  faculty  currently  use  to  generate  course  materials  now  have 
provisions  for  saving  data  in  formats  that  browsers  can  understand.  Word  processors,  spreadsheets,  database,  and 
presentation  software  now  allow  users  to  save  documents  in  HTML  format.  Existing  documents  can  be  re -saved  in 
HTML  format,  allowing  creation  of  Web  materials  without  additional  work  in  a separate  software  tool.  Once  in 
HTML  format,  these  files  can  be  made  available  to  students  by  placing  them  on  the  website.  Students  can,  in  turn, 
access  the  files  through  links  in  existing  Web  pages,  or  by  entering  the  path  and  file  name  in  the  URL.  All  this  is 
possible  with  a minimum  of  additional  effort,  using  software  the  user  is  already  familiar  with.  The  keys  to  success 
are:  1)  saving  the  documents  in  a format  browsers  understand  and  2)  placing  these  documents  in  the  website  where  a 
browser  can  find  them. 

Saving  in  HTML;  A Microsoft  Word  Example 

Microsoft’s  word  processor.  Word,  will  save  documents  in  HTML  format.  The  following  example  uses  the 
technique  described  above  to  save  an  existing  homework  assignment  document  in  HTML  format  and  place  it  on  the 
Web  for  student  access.  The  document  is  currently  saved  in  a file  called  Homeworkl0.doc  and  is  located  in  the  My 
Documents  folder  on  the  local  computer.  The  new  HTML  version  of  the  assignment  will  be  saved  in  the  webpage 
pua.edu  in  the  /courses/cpt/cpt250  folder.  The  Website  home  folder  is  designated  as  the  W;/  network  drive  on  the 
local  computer. 

Begin  by  retrieving  the  original  document  into  Word.  Once  the  document  is  retrieved,  save  it  in  HTML 
format  by  clicking  the  File  menu,  and  then  clicking  the  Save  as  Web  Page... option  in  the  dropdown  list.  A new 
Save  As  window  appears.  Note  the  Save  In:  dropdown  list  in  the  upper  left  comer  of  the  Save  As  window.  Clicking 
the  down  arrow  to  the  right  of  the  input  box  will  produce  a list  of  all  available  drives.  Navigate  through  this  list  and 
double  click  the  W;/  drive.  A folder  list  will  appear  below  the  W;/  drive  icon.  Next  double  click  the  /courses 
folder.  Double  click  the  /cpt  folder  in  the  list  that  appears.  Double  click  the  /cpt250  folder  in  the  folder  list  that 
appears  below  the  /cpt  folder.  This  will  select  the  /courses/cpt/cpt250  folder  to  receive  the  HTML  file.  Enter  the 
name  of  the  new  HTML  version  of  the  document  by  typing  the  new  name,  HomeWorklO,  in  the  File  Name:  input 
box  in  the  lower  center  of  the  Save  As  window.  Do  not  add  the  .htm  extension.  The  system  will  automatically 
append  the  appropriate  extension  when  it  saves  the  document.  Finally  click  the  Save  button  in  the  lower  right  comer 
of  the  Save  As  window.  The  HTML  version  of  the  homework  assignment  is  now  saved.  Since  the  file  was  saved  in 
a folder  that  is  part  of  the  Website,  students  are  able  to  access  the  homework  assignment  on  the  Web  by  typing 
http;//pua.edu/courses/cpt/cpt250/HomeWorkl0.htmin  their  browser’s  URL  input  box,  or  by  clicking  on  a link 
to  the  file  if  one  exists. 

If  direct  access  to  the  Web  folders  is  not  possible,  the  Webmaster  will  post  the  file  to  the  website.  First, 
save  the  Word  document  in  HTML  format  per  the  example  above  but  specify  a local  drive  instead  of  the  Web  folder. 
Next  send  a request  to  the  Webmaster  to  post  the  file  to  the  Web.  If  the  file  will  be  accessed  through  the  URL,  the 
Webmaster  should  provide  the  complete  URL  after  the  posting  is  complete.  If  the  file  is  to  be  accessed  via  an 
existing  link,  the  Webmaster  can  place  and  name  the  file  so  the  link  operates  properly. 

Saving  in  HTML;  A Microsoft  Excel  Example 

Microsoft  Excel,  a popular  spreadsheet  application,  will  also  save  in  HTML  format.  It  is  a common 
practice  to  post  grades  in  a public  place  so  students  can  monitor  their  status.  This  example  illustrates  how  the  Web 
can  be  used  to  post  grades  for  student  review  by  saving  an  existing  grade  spreadsheet  in  HTML  format  and  placing 
it  on  the  Web  where  students  can  access  it.  The  grade  spreadsheet  is  currently  saved  in  a file  named 
GSP250F01.xls.  It  is  located  in  the  My  Documents  folder  on  the  local  computer.  The  new  HTML  versbn  of  the 
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grade  posting  will  be  saved  with  the  name  grades.htm,  in  the  /courses/cpt/cpt250/  folder  of  the  website  pua.edu, 
designated  as  the  W:/  network  drive  on  the  local  computer. 

The  first  task  is  to  retrieve  the  spreadsheet  and  save  it  in  HTML  format.  Start  Excel  and  open  the 
GSP250F01.xls  spreadsheet.  Next,  save  the  spreadsheet  in  HTML  format  by  selecting  the  File  option  from  the 
menu  bar,  and  clicking  the  Save  as  Web  Page...  option  in  the  dropdown  list.  The  Save  As  window  appears.  Locate 
the  Save  In:  input  box  and  use  the  down  arrow  to  the  right  to  display  the  list  of  available  drives  and  folders. 

Navigate  through  this  list  until  the  W:/courses/cpt/cpt250  folder  is  located  and  double  click  that  folder.  Below  the 
Save  In:  input  box  is  a Sa^Je:  section.  This  consists  of  two  option  buttons  labeled  Workbook  and  Selection 

Sheet.  Click  the  Entire  Workbook  option.  In  the  File  Name:  input  box  enter  the  file  name  grades.  Do  not  include 
the  .htmfile  extension.  Excel  will  append  the  extension  onto  the  file  name  when  the  file  is  saved.  Verify  that  the 
Save  as  Type:  input  box  contains  Web  Page.  If  not,  open  the  list  by  clicking  the  down  arrow  to  the  right  of  the 
input  box  and  select  the  Web  Page  option.  Finally,  click  the  Save  button.  The  grade  posting  spreadsheet  will  be 
saved  as  W:/courses/cpt/cpt250/grades.htm.  Students  can  access  the  grades  on  the  Web  by  entering 
http://www.pua.edu/courses/cpt/cpt250/grades.htm  (the  complete  URL)  into  the  browser.  Clicking  a link,  if  one 
exists,  can  also  access  the  grades. 

If  direct  access  to  the  Web  folders  is  not  possible,  the  Webmaster  will  post  the  file  to  the  website.  First, 
save  the  Excel  spreadsheet  in  HTML  format  per  the  example  above  but  specify  a local  drive  instead  of  the  Web 
folder.  Next  send  a request  to  the  Webmaster  to  post  the  file  to  the  Web.  If  the  file  will  be  accessed  through  the 
URL,  the  Webmaster  should  provide  the  complete  URL  after  the  posting  is  complete.  If  the  file  is  to  be  accessed  via 
an  existing  link,  the  Webmaster  can  place  and  name  the  file  so  the  link  operates  properly. 

Generating  Web  Content:  Downloads 

The  Web  can  also  be  used  to  make  non-viewable  files  available.  The  browser  cannot  display  the  content  of 
these  types  of  files  because  they  are  saved  in  a format  that  the  browser  cannot  understand.  If  the  browser  does  not 
understand  the  format  of  a file,  it  will  offer  the  user  the  option  of  opening  the  file  using  another  application  or 
downloading  and  saving  the  file  to  a local  drive. 

If  the  Open  option  is  selected,  the  local  system  will  check  for  an  application  that  understands  the  format  of 
the  file.  If  the  system  finds  an  application  that  understands  the  format,  the  file  can  be  opened  and  manipulated  just  as 
any  other  local  file,  with  one  exception.  The  local  application  cannot  save  the  file  back  to  the  Web  site.  It  must  be 
saved  on  a local  drive.  If  the  system  cannot  find  an  application  that  understands  the  file  format,  it  will  present  a list 
of  all  application  available  on  the  system  and  allow  the  user  to  select  the  application  to  use. 

If  the  Save  option  is  selected,  the  user  is  prompted  to  enter  the  location  the  file  will  be  saved  to.  The  user 
selects  the  desired  location  and  initiates  the  download.  The  file  is  copied  from  the  website  to  the  local  machine. 

Downloading  and  Opening  a Word  Document:  A Microsoft  Word  Example 

This  example  illustrates  opening  a Microsoft  Word  document  file  from  a website.  The  file  is  in  Word’s 
.doc  format,  which  is  not  understood  by  the  browser.  We  assume  that  the  student  has  Microsoft  Word  available  on 
the  local  computer.  The  Web  file  is  a Word  document  named  Homeworkl0.doc.  It  is  saved  on  the  Web  site 
pua.edu  in  the  folder  /courses/cpt/cpt250/.  To  access  the  file  click  on  an  existing  link  or  enter  the  complete  URL 
and  path  into  the  browser:  http://www.pua.edu/courses/cpt/cpt250/homeworkl0.doc.  A Windows  Wizard  will 
open  offering  option  buttons  for  Open  this  file  from  its  current  location  or  Save  this  file  to  disk . Select  the  Open 
option  button.  The  local  computer  will  search  for  an  application  that  understands  files  in  the  .doc  format.  It 
identifies  Microsoft  Word  as  an  application  that  understands  .doc  files,  starts  Word  running  on  the  local  computer, 
and  downloads  Homeworkl0.doc  into  Word.  The  document  can  now  be  manipulated  with  Word  just  as  any  other 
local  document.  It  can  be  saved  to  a local  drive  using  the  normal  save  procedure  in  Word,  but  it  cannot  be  saved  to 
the  Web  site. 

Downloading  and  Saving  a Data  File 

The  following  example  shows  how  to  download  a data  file  from  a website.  The  file  we  want  to  transfer  to 
the  local  computer  is  a data  file  to  be  used  in  a C++  programming  assignment.  The  file  name  on  the  website  is 
InputData.dat.  The  .dat  extension  indicates  that  this  file  contains  data.  Files  with  the  .dat  extension  are  not 
usually  associated  with  any  specific  application  and  therefore  cannot  be  opened  from  the  website.  The  file  is  located 
on  the  pua.edu  website  in  the  folder  /courses/cpt/cpt250.  The  goal  is  to  change  the  file’s  name  to  Proj3Input.dat 
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and  save  it  in  the  /Documents  folder  of  the  local  C:/  drive.  Begin  by  starting  the  browser  and  entering  the  complete 
URL  and  path  of  the  file:  http://www.pua.edu/courses/cpt/cpt250/InputData.dat  The  browser  will  start  the  File 
Download  wizard.  The  first  screen  offers  the  user  two  option  buttons;  Open  this  file  from  its  current  location  and 
Save  this  file  to  disk . Select  the  Save  this  file  to  disk  option.  The  next  screen  is  the  Save  As  dialog  box.  Click  the 
down  arrow  to  the  right  of  the  Save  In:  input  box  and  double  click  the  C:/  drive.  A list  of  all  folders  available  on  the 
C:/  drive  will  appear.  Double  click  on  the  Documents  folder.  The  folder  will  open,  and  its  contents  will  be 
displayed  in  the  window.  Type  Proj3Input.dat  in  the  File  name  input  box  and  click  the  Save  button.  As  the 
download  begins,  the  wizard  will  display  a status  screen  that  shows  the  file  source  and  destination  information,  the 
estimated  time  to  complete  the  download,  and  the  data  transfer  rate.  When  the  download  is  complete,  the  screen 
will  show  statistics  on  the  download  process.  The  file  is  now  in  the  folder  C:/Documents/,  under  the  name 
Proj3Input.dat  Verify  this  by  opening  Windows  Explorer  and  viewing  the  contents  of  the  C:/Documents  folder. 

Conclusion 

The  Web  is  an  information  source  that  offers  many  faculty  - student  communications  opportunities.  Both 
faculty  and  students  can  benefit  from  the  on-demand  information  availability  the  Web  offers.  The  Web  support 
features  included  in  today’s  personal  productivity  software  will  allow  faculty  to  develop  basic  Web  content  with 
little  or  no  additional  effort.  These  features,  used  in  conjunction  with  the  existing  Web  presence  of  most  institutions 
provides  a framework  on  which  faculty  can  grow  course  support  with  little  or  no  Web  development  experience. 

This  places  Web  course  support  within  reach  of  most  faculty. 
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Abstract 

Last  year,  the  International  Society  for  Technology  in  Education  (ISTE)  worked  to  develop  and  improve 
teacher  skills  in  the  area  of  educational  technology  by  creating  technology  performance  profiles  for 
teacher  preparation.  These  standards  were  given  to  a group  of  PSU  instructional  systems  graduate 
students  who  developed  rubrics  to  assess  the  four  categories  of  standards.  The  process  and  methods  used 
in  the  design,  development,  and  creation  of  these  rubrics  will  be  presented  and  discussed,  and  the  rubrics 
will  be  available  for  use  by  the  participants. 

Introduction 

Assessment  has  always  been  an  important  part  of  both  the  education  and  training  fields.  Standards  are  also 
valuable  way  of  defining  expectations,  but  when  it  comes  to  assessing  whether  standards  have  been  achieved  the 
plot  thickens.  Issues  that  arise  concerning  the  reliability  and  validity  of  measurement  and  assessment  will  always  be 
present.  Assessment  instruments  are  not  always  easy  to  design  or  develop.  Some  assessment  instruments  such  as 
rubrics  can  be  of  some  help.  Rubrics  are  designed  to  “focus  on  measuring  a stated  objective  (performance,  behavior, 
or  quality),  use  a range  to  rate  performance,  and  contain  specific  performance  characteristics  arranged  in  levels 
indicating  the  degree  to  which  a standard  has  been  met.**  (Pickett)  A rubric  may  be  applied  in  many  situations  and 
score  criteria  that  is,  “summative,  by  providing  information  about  a student’s  knowledge,  formative,  by  providing 
information  about  a student's  strengths  and  weaknesses,  evaluative,  by  providing  ways  to  create  instruction  that 
better  fits  each  student's  needs,  and  educative,  by  providing  students  with  an  understanding  of  how  they  learn.” 

(Luft,  1997)  Rubrics  evaluate  and  measure  a learning  situation  using  a holistic  or  analytical  approach  that  ideally 
results  in  the  creation  of  an  easy  to  use  matrix  of  learning  targets  and  the  level  at  which  these  targets  are  achieved. 
The  number  of  categories  and  levels  of  mastery  is  always  up  to  the  creator(s)  of  the  rubric. 

This  presentation  discussed  the  use  of  rubrics  as  an  instrument  to  assess  standards.  The  National 
Educational  Technology  Standards  (NETS)  project  writing  team  has  created  and  defined  technology  performance 
profiles  for  teacher  preparation  across  four  categories  in  hopes  to  use  these  profiles  in  the  development  and 
improvement  of  teacher  skills  and  competencies  in  the  area  of  educational  technology 

(http://cnets.iste.org/pdf/nets_brochure.pdf).  This  project,  initiated  by  the  International  Society  for  Technology  in 
Education  (ISTE),  focused  on  the  development  of  national  standards  for  various  uses  of  technologies  that  are 
associated  with  school  improvement  in  the  United  States.  A group  of  graduate  students  at  Penn  State  University, 
taking  a class  based  on  analyzing  outcomes  and  learners,  given  these  standards  and  charged  with  developing 
assessment  instruments  based  on  the  ISTE  standards.  This  presentation  explored  the  design  and  development  of 
those  assessment  instruments. 

The  presentation  began  with  a brief  introduction  and  statement  of  objectives.  We  then  explored  the  work 
done  by  the  ISTE  NETS  writing  team  with  the  intent  to  engage  the  audience  in  the  specific  areas  addressed  by  these 
standards.  We  then  explored  the  four  categories  of  standards,  general  preparation,  professional  education,  student 
teaching/intemship,  and  first-year  teaching  and  the  suggestions  for  evaluation  provided  by  a class  well  versed  in 
analyzing  learners  and  outcomes.  We  looked  at  separate  rubrics  the  class  constructed  and  examined  how  they  relate 
to  the  original  standards.  We  discussed  the  design  and  development  of  these  rubrics  and  the  processes,  obstacles, 
and  methodologies  in  which  these  assessment  instruments  were  created.  The  presentation  ended  with  an  open  forum 
and  a brief  question  and  answer  period  between  the  audience  and  the  presenters. 
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The  format  of  this  presentation  recommends  a refined  approach,  used  for  creating  and  implementing 
rubrics,  as  an  appropriate  instrument  of  assessment,  specifically  when  evaluating  performance  based  on  the  ISTE’s 
National  Educational  Technology  Standards.  The  audience  was  expected  to  offer  constructive  criticism  and 
suggested  ways  to  improve  upon  the  processes  and  methods  used. 

The  NETS-T Project 

• The  National  Educational  Technology  Standards  for  Teachers  (NETS-T)  is  an  International 
Society  for  Technology  in  Education  (ISTE)  initiative,  which  seeks  to  prepare  today’s  teachers -in- 
training to  provide  technology-supported  learning  opportunities  and  interventions  for  K-12 
classrooms. 

• This  initiative  is  funded  by  a United  States  Department  of  Education’s  Preparing  Tomorrow’s 
Teachers  to  use  Technology  (PT^)  Grant,  a catalyst  program  designed  to  train  technology  literate 
teachers. 

• Accordingly,  NETS-T  defines  technology  knowledge  and  skills,  as  well  as  attitudes  new  teachers 
should  possess  as  they  proceed  through  their  teacher  preparation  programs. 

• Each  performance  standard  outlined  in  each  of  the  four  profiles  is  designed  to  assess  ISTE’s 
National  Education  Technology  Standards  (NETS)  and  their  corresponding  Performance 
Indicators  for  teachers. 

• To  facilitate  this  process,  ISTE  developed  four  profiles  of  technology  literate  teachers,  each  of 
which  details  the  required  technology  knowledge  and  skills  at  the  following  stages  of  preparation: 

o Completion  of  the  general  preparation 
o Prior  to  intemship/student  teaching  experience 
o After  completion  of  intemship/student  teaching 
o Upon  completion  of  first  year 


♦Additional  details  for  each  profile  are  available  at  http://cnets.iste.org/pdf/nets  brochure. pdf  and  at  the 
ISTE  website  http://www.iste.org. 

Why  We  Did  It 

• Kyle  Peck,  Ph.D.,  Professor  of  Education  in  Instructional  Systems,  assigned  the  assessment  of  the 
NETS-T,  in  partial  fulfillment  of  a required  graduate  course  in  assessments. 
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• He  felt  that  this  project  would  both  provide  a real-life  opportunity  as  well  as  be  of  practical  value 
to  ISTE’s  efforts,  which,  to  that  date,  had  not  created  a comprehensive  assessment  strategy  for  this 
initiative. 

How  It  Was  Done 

• For  the  most  part,  students  were  able  to  choose  the  Profile  that  they  wished  to  develop  assessments 
for. 

• We  were  given  the  freedom  to  organize  the  performance  standards  such  that  it  was  deemed 
manageable. 

• This  meant  combining,  re  -writing,  and  even  removing  performance  standards  that  did  not  fit  in 
with  the  respective  graduate  students  assessment  plan. 

• Accordingly,  it  is  safe  to  say  that  no  two -assessment  products  within  each  profile  were  the  same. 


Examples 

Profile  #2:  Professional  Preparation 
Description 

Learning  Target  III:  Teaching,  Learning  and  Curriculum  (ISTE  NETS  Performance  Indicators: 
I,  III,  IV, V) 

Performance  Objective:  The  teacher  will  be  able  to  design  lessons  and  peer  teach  with  technology  that  meets  the 
content  standards  and  reflects  the  best  practices  in  teaching  and  learning. 

Performance  Standard:  Having  observed  teaching  practices  in  various  schools,  video  demonstrations  and  other  peer 
teaching,  the  teacher  will  be  able  to  design  and  teach  lessons  that  reflect  the  best  practices  in  teaching  and  learning 
using  technology 

Assessment  Method:  Lesson  Plans  and  Teaching  Practice 

Performance  Task:  Design  and  teach  technology  enriched  learning  activities  that  maximize  student  leaning  with 
diverse  needs. 

Learning  Target  I 

Elements  of  Assessment:  Knowledge  of  Technology  Concepts,  Tools  and  their  Uses. 

This  is  a ‘to  do  activity’  for  your  professional  development. 

There  are  three  sections  numbered  A,  B,  C.  Please  follow  the  instructions  carefully. 

A.  Technology  Concepts. 

i)  Assessing  your  conceptualization  of  technology-related  concepts. 

Directions:  Work  through  this  activity  and  construct  your  own  personal  concepts  about  technology.  Circle  T or  F to 
indicate  what  is  True  or  False,  and  write  an  explanation  if  required. 

1.  The  terms  ‘medium  for  instruction’  and  ‘instructional  technology’  are  synonyms. 

TorF 

2.  There  is  a difference  between  interactive  technologies  and  presentation  technologies. 

TorF 

3.  Printed  textbooks  are  technology  tools  for  delivery. 

T or  F 


7 
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4.  There  is  no  significant  difference  in  learner  achievement  between  those  who  are  taught  with  the 
Internet  and  those  taught  with  traditional  methods. 

TorF 

Explain  your  answer. 


ii)  Assessing  Personal  Attitudes  Towards  Technology 

Directions:  Place  an  “x”  on  the  continuum  to  represent  your  feelings  towards  the  statements  below. 

1.  Technology  should  be  integrated  into  teaching  and  learning  from  the  cradle  to  the  grave. 

Agree  Uncertain  Disagree 

I ^ I 

2.  New  technologies  for  use  in  the  classroom  are  more  important  than  traditional  basic  supplies. 

Agree  Uncertain  Disagree 

I ^ I 

3.  Computer-mediated  teaching  will  soon  make  teachers  redundant. 

Agree  Uncertain  Disagree 

I ^ I 

4.  Today’s  learners,  who  are  more  literate  than  their  own  teachers,  threaten  me. 

Agree  Uncertain  Disagree 

I ^ I 

5.  I am  not  confident  with  my  skills  to  try  new  technologies. 

Agree  Uncertain  Disagree 

1 ^ I 

6.  Today’s  learners,  who  are  very  computer  savvy,  are  less  creative  than  learners  who  are  less  exposed  to 
computers. 

Agree  Uncertain  Disagree 
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C.  Technological  Tools 

Assess  your  knowledge  of  Technological  Tools. 

In  the  second  column  of  the  table  below,  place  a checkmark  next  to  the  technological  tools  that  you  are 
comfortable  using  and  make  a cross  by  those  you  would  like  to  be  able  to  use.  In  the  third  column,  give  your 
comments  on  how  you  plan  to  improve  your  skills  to  use  those  tools. 


Medium 

Technology  tools  for  Delivery 

Comments  for  Improvement 

Face-to-Face  contact 

Presentation  Aids: 
Black  or  white  board 
Overhead  projector 
Data  projector 
Slide  projector 

Text  (including  graphics) 

Printed  textbook 

Web-based  textbook 

Facsimile 
Computers 

(Including  a range  of  applications  such  as  e- 
mail,  electronic  databases.  World  Wide  Web 
hypertext  documents,  FTP  or  ASCII  documents, 
CD-ROM) 

Audio 

Audiocassette 
Radio  broadcast 
Telephone 

Computers  (with  related  applications) 

Video 

Television  Broadcast 

(Terrestrial,  satellite  or  cable  and  including 
narrowcast  educational  television"! 
Videocassettes 
Video  disc 

Videoconferencing 

Integrated  Multimedia 

Computer-mediated  learning  environments 
(This  could  be  stand  alone  or  networked  and 
with  numerous  possible  ways  to  access  all 
media  through  a single  technolo^) 

Knowledge  of  Computer  Operations 

Directions:  In  the  right-hand  column,  circle  the  letter(s)  that  indicate  your  knowledge  about  computer  uses. 
K = I Know  DK  = I Don’t  Know 


Graphic  User 
Interface 

Performance  Task 

Use  of  toolbar, 
menus,  and  dialog 
boxes 

Locate  and  use  features  on  a toolbar. 
Locate  and  select  items  on  menus. 
Locate  and  respond  to  dialog  boxes. 

K 

DK 

Menu  Shortcuts 

Identify  and  use  menu  shortcuts. 

K 

DK 

Selecting 

Select  tools  and  other  features  within  a GUI. 

K 

DK 

Click,  Double 
Click/Rieht  Click, 

Demonstrate  clicking,  double-clicking,  right  clicking  (on  PC),  command 
clicking  (On  MAC),  and  dragging. 

K 

DK 

1029 


BEST  COPY  AVAILABLE 
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Drag 

Window 

Identify  and  use  the  functions  of  a window. 

K 

DK 

Scrolline 

Scroll  up  and  down  a window. 

K 

DK 

Resizing 

Resize  a window. 

K 

DK 

Closing 

Close  a window. 

K 

DK 

Application 

Software 

Performance  Task 

Launching 

Launch  an  application. 

K 

DK 

Creating  files 

Create  a file. 

K 

DK 

Saving 

Save  a file. 

K 

DK 

Common  features 

Identify  the  features  that  most  applications  have  in  common. 

K 

DK 

Basic  four 

productivity 

applications 

Performance  Task 

:: — : 

Word  processing 

Demonstrate  the  use  of  a word  processing  application. 

K 

DK 

Spreadsheet 

Demonstrate  the  use  of  a spreadsheet  application. 

K 

DK 

Database 

Demonstrate  the  use  of  a database  application. 

K 

DK 

Graphics 

Use  a graphic  application. 

K 

DK 

File 

Management 

Performance  Task 

Saving  a file  on  a 
hard  drive 

Save  a file  on  a hard  drive. 

K 

DK 

Saving  a copy  of 
same  file  at 
different  location 

Save  a copy  of  same  file  at  different  location. 

K 

DK 

Creating  file 
folders 

Create  a new  file  folder. 
Save/Move  files  to  a folder. 

K 

DK 

Findinga  * 

directory  from  a 
pathname 

Find  a directory  from  a pathname. 

K 

DK 

Naming 

conventions 

Recognize  and  use  file  -naming  conventions  (as  they  may  appear  on  a MAC 
and  a PC). 

K 

DK 

Explain  the  importance  of  using  the  correct  naming  conventions. 

File  Recovery 

; Identify  and  demonstrate  the  methods  available  to  recover  files  and  lost  data. 

K 

DK 

Teaching,  Learning  and  Curriculum 

Element  of  Assessment:  Lesson  Plan  and  Teaching  Practice  Using  Technology 
Performance  Task:  Design  and  Teach  Technology  Enriched  Learning  Activities  That  Maximize 
Student  Learning  With  Diverse  Needs. 

Name  of  Rubric:  Checklist  for  Lesson  Observation  and  Assessment  Using  Interactive  and 
Presentation  Technologies 


Last  Name: Initials: 

Student  Id: 

Program  of  Study: 

Subject: Topic: 


Grade: Date: Time: 

Signature  of  Evaluator/Observer; Marks:  5x2=  10  (Each  element 

carries  2 marks) 


A,  LESSON  PLAN  PREPARATION 


DESCRIPTORS 


COMMENTS 
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BEST  COPY  AVAILABLE 


Appropriate  technological  tools  were  chosen. 

Technology  relevant  to  the  topic  and  logically  integrated  in  the  lesson. 


B.  LESSON  PRESENTATION 


DESCRIPTORS 

COMMENTS 

Effective  and  skillful  use  of  technological  tools 

Technology  chosen  is  purposeful  and  gained  immediate  attention  from  the 
class. 

Achieved  appropriate  content  of  the  lesson. 

C.  HANDLING  OF  TECHNOLOGICAL  EQUIPMENT 


DESCRIPTORS 

COMMENTS 

Knew  all  the  operations  of  the  technological  tools 

D.  SENSITIVITY  TO  DIVERSE  NEEDS 


DESCRIPTORS 

COMMENTS 

Technology  used  was  sensitive  to  the  special  needs  of  individual  identities. 
Technological  tool  used  communicated  to  different  sectors  of  the  class. 

E.  GENERAL  IMPRESSION  OF  THE  STUDENT  TEACHER 


DESCRIPTORS 

COMMENTS 

Used  a variety  of  technological  tools  and  looked  confident,  enthusiastic  and 
enjoys  teaching  with  technology. 

AECT  Presentation  Conclusion 

The  presentations  were  based  on  the  International  Society  for  Technology  in  Education  (ISTE)  National 
Educational  Technology  Standards  (NETS)  for  Teachers. 

The  class  included  students  from  varying  professional  backgrounds.  We  believe  that  all  students, 
regardless  of  professional  and  academic  background,  found  the  assessment  of  these  standards  important  to  the 
success  of  their  implementation.  In  evaluating  these  standards,  most  students  found  that  creating  an  assessment 
process  was  difficult  because  it  involved  a great  deal  of  compromise  as  well  as  huge  amounts  of  time  to  complete 
the  assessment  of  a standard. 

Despite  the  difficulties  it  was  an  extremely  rewarding  process  because  we  were  given  experience  of  the 
difficulties  and  challenges  inherent  in  any  educational  assessment. 
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Abstract 

Because  the  complex  link  and  node  structure  awaiting  users  can  lead  them  into  becoming  lost  in 
hyperspace  and  cause  them  cognitive  overload,  navigating  the  hypertext  system  is  often  not  an  easy  task, 
especially  for  novices.  They  have  a difficulty  perceiving  a structure  a structure  of  entire  system,  locating 
specifying  information,  using  navigational  aid.  This  phenomenon  can  be  more  expected  in  the  highly 
structured  EPSS.  In  that  point,  content  providers  or  designers  for  web-based  integrated  EPSS  should  know 
how  to  design  interface  and  information  structure  based  on  its  content  and  purpose. 

This  study  analyzed  four  interface  design  methods  (simple  selection  menu  style,  menu  with  global 
navigation,  menu  with  global  and  local  navigation,  and  pull-down  menu)  and  four  information  structures 
(linear  structure,  grid,  hierarchy,  and  network)  in  terms  of  the  complexity,  flexibility,  navigation,  domain 
knowledge,  and  cognitive  load.  Based  on  this  analysis,  guidelines  for  building  the  integrated  web-based 
EPSS  effectively  are  provided. 

Introduction 

Building  an  effective  EPSS  has  been  a major  issue  not  only  in  business  but  also  in  education  and 
government.  As  more  people  come  to  depend  on  the  use  of  computers  and  networking  to  perform  their  jobs,  and  as 
hardware  and  software  technologies  continue  to  advance,  the  potential  for  EPSS  appears  tremendous  (Collis  & 
Verwijs,  1995;  Malcolm, 1998;  Ockerman  etal.,  1997). 

Hypertext  and  hypermedia  system  enabled  designers  and  programmers  to  tile  all  performance  support 
system  elements  together  in  a way  that  allowed  users  to  follow  their  own  streams  of  thought  in  searching  for 
information  (Gery,  1991).  Hypertext  is  such  a technology  that  provides  a powerful  new  way  of  organizing, 
displaying,  and  accessing  information  that  could  affect  all  forms  of  systems  (Shneiderman  and  Kearsley  1989). 
Hypertext  consists  of  the  associative  links  between  multiple  nodes,  which  are  one  or  more  parts  of  information, 
forming  an  interconnected  networked  (Nelson  1974;  Conklin  1987).  The  linking  system  in  the  hypertext  system 
allows  users  to  browse  through  the  system  utilizing  navigational  tools  (Eklund  1995).  However,  navigating  the 
hypertext  system  is  often  not  an  easy  task,  especially  for  novices  (King,  1996).  The  potentially  complex  link  and 
node  structure  awaiting  users  can  lead  them  into  becoming  lost  in  hyperspace  (Nielsen  1990)  and  cause  them 
cognitive  overhead  (Conklin,  1987).  Users  have  a difficulty  perceiving  a structure  of  entire  system,  locating 
specifying  information,  using  navigational  aid.  This  phenomenon  can  be  more  expected  in  the  highly  structured 
EPSS. 

Such  problems  have  prompted  research  on  the  manner  in  which  users  interact  with  hypertext  system. 

Usable  design  guidelines  and  principles  for  navigation  can  maximize  coherence  of  integrated  EPSS  and  minimize 
users'  cognitive  overhead  and  disorientation. 

Interface  design 

When  designing  an  integrated  EPSS,  two  important  issues  are  involved:  1)  interface  design  - how  to  present 
link  system  that  provide  access  to  the  structure,  and  2)  information  structure  - how  to  incorporate  the  original 
structure  of  the  content  into  the  structure  of  an  EPSS. 

Interface  design  is  basically  concerned  with  the  presentation  of  text,  graphics,  and  linking  system  on  the 
screen.  It  provides  a contextual  or  structural  model  for  the  specification  of  the  logical  and  functional  organization  of 
the  user  interface  component,  as  well  as  a communication  and  means  between  users  and  system  (Laverson,  Norman 


543 

1032 


et  al.  1987;  Norman  and  Chin  1988;  Lai  and  Waugh  1994;  Oliveira,  Goncalves  et  al.  1999).  Many  researchers  has 
been  agreed  that  interface  should  be  designed  to  provide  users  efficient  and  effective  organizational  model  that  can 
help  users  understand  the  entire  system  and  navigate  system  to  find  information  without  getting  lost  or  cognitive 
overload  (Lai  and  Waugh  1994;  Dieberger  1997;  Shneiderman  1997;  Schenkman  and  Jonsson  2000). 

Interface  on  an  EPSS  provides  three  major  roles  for  navigation;  1)  presenting  links,  2)  supporting  structural 
cues,  and  3)  providing  path  mechanism.  The  interface  is  the  most  immediately  visible  part  when  a system  was  user- 
centered  (Nielsen  1990).  The  most  fundamental  function  of  interface  is  to  display  links  one  the  screen  so  that  a user 
could  navigate  through  a system  (Shneiderman  1998).  Selecting  the  links  is  frequently  difficult  to  navigate  in  spite 
of  the  fact  that  a graphical  browser  provides  the  easy  to  navigation  method,  "point  and  click."  One  factor  of 
difficulty  in  selecting  paths  for  navigation  is  the  presentation  of  link  system.  The  user  navigation  performance  can 
be  influenced  by  the  design  and  placement  of  links  (Carlson  and  Kacmar  1999). 

The  second  role  is  to  support  the  user  perception  of  structural  cues  for  an  effective  navigation.  If  the 
interface  does  not  provide  appropriate  structural  information,  users  cannot  perceive  where  they  are  and  cannot 
decide  where  to  go.  Users  also  could  experience  "lost  in  space"  (Dieberger,  1997). 

Information  is  structured  by  links  and  users  follow  paths  developed  by  designer.  As  a result,  if  interface 
design  provides  the  flexible  path  mechanism  that  can  allow  users  to  jump  to  the  information  directly  without  passing 
pages  that  are  not  necessary  to  get  the  information,  users  could  find  information  fast.  However,  there  is  some 
possibility  to  increase  user's  cognitive  load  without  providing  structural  and  navigational  cues. 

There  are  four  generally  used  interfaces  for  navigating  the  web  site:  1)  simple  selection  style,  2)  global 
navigation  aid,  3)  global  and  local  navigation,  and  4)  pull-down  menu  styles. 

1)  Simple  Selection  Menu  Style 

One  type  of  menu  style  is  simple  selection  menu  style,  which  is  much  similar  to  a table  of  contents  for  a 
print  book  (Chimera  and  Shneiderman  1994).  In  this  menu  style,  a user  can  go  to  deeper  levels  of  the  web  site  by 
selecting  links  presented  in  the  current  web  page.  The  major  drawback  of  this  menu  style  is  that  a user  has  a 
difficulty  in  perceiving  the  entire  structure  of  the  web  site  (Chimera,  1994),  since  when  a user  moves  into  lower 
levels,  previous  menu  is  replaced  with  new  level.  Another  drawback  of  this  menu  style  is  that  a user  is  unable  to 
traverse  to  beyond  below  level  because  a user  has  to  wait  for  anew  below  level  select  link  again  before  moving 
lower  level  and  select  the  link  below  level  again.  An  experienced  user  is  unable  to  navigate  faster  than  that  that  of 
novice  and  this  is  inefficient  and  frustrating  for  experienced  users  (Laverson,  1987). 

2)  Global  Navigation  Aid  Menu  Style 

The  second  menu  style  is  persistent  menu  that  has  two  split  parts.  The  links  on  the  top  level  remain  in  the 
similar  area  on  the  left  all  the  time.  The  content  is  located  in  the  area  on  the  right  and  is  replaced  by  its  subsequent 
menu  when  users  move  other  pages.  The  advantage  of  this  menu  style  is  that  a user  can  go  to  each  page  of  the  top 
level  by  clicking  top  level  links  on  the  left  side  from  any  page  in  the  web  sit  and  that  it  provides  the  global  structural 
cue  to  users  (Nanard  and  Nanard  1991). 

3)  Global  and  Local  Navigation  Menu  Style 

The  third  menu  style  has  two  parts  of  navigation  link  for  global  level  and  local  level  (Nanard  & Nanard, 
1991).  The  navigation  links  of  top-level  pages  on  the  web  site  usually  place  on  the  top  of  web  page  and  the 
navigation  links  of  current  level  pages  on  the  web  site  usually  place  on  left  side.  The  advantage  of  this  menu  style  is 
to  provide  not  only  global  structure  cue  with  the  top-level  navigation  links  but  also  local  structure  cue  with  current 
level  navigation  links  (Furnas  1997).  A user  can  skip  several  levels  in  the  web  site  and  it  can  more  efficient  than 
other  menu  design  because  a "jump -ahead"  capability  can  reduce  the  time  to  navigate  and  find  information 
(Laverson,  1987). 

4)  Pull-Down  Menu  Style 

Pull-down  menu  style  appears  over  objects  in  the  interface  instead  of  in  static  menu  area.  Pull-down  menu 
allow  users  with  a mouse  to  access  the  Web  page  they  want  directly.  The  advantage  of  the  menu  style  is  jump  to  the 
any  page  with  mouse  move  and  click.  However,  users  can  be  disoriented  or  get  lost  since  this  menu  can't  provide 
structural  cues.  This  menu  style  would  be  useful  for  experienced  users. 
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Table  1.  Summary  of  Menu  Style 


Use 

Complexity 

Flexibility 

Access  Speed 

Disorientation 

Simple  Selection 

Easy 

Low 

Low 

Slow 

Low 

Global  Navigation 

Easy 

Middle 

Middle 

Middle 

Middle 

Global  & Local  Navigation 

Hard 

High 

High 

Fast 

High 

Pull-Down 

Middle 

Very  High 

Verv  High 

Verv  High 

Very  High 

Information  Structure 

The  structure  of  a hypertext  system  can  take  many  different  forms.  Four  basic  structures  are  linear,  grid, 
hierarchical,  and  network  structure.  However,  a web-based  integrated  EPSS  can  be  designed  with  more  than  one 
structure. 


1)  Linear  Structure 

The  simplest  way  to  organize  information  is  in  a sequence  based  on  chronological  or  logical  orders. 
Typically  it  is  useful  structure  to  retain  the  original  documentation.  Linear  structure  can  be  used  for  guided  tour,  job 
aids,  tutorials,  and  demonstration  of  procedure  in  Web-based  EPSS. 


2)  Grid 

Grid  structure  organizes  multi-dimensional  concepts  or  categorizations.  A serous  of  procedural  manuals 
and  list  of  training  courses  and  materials  can  be  best  organized.  Grid  structure  can  be  organized  with  one  concept  or 
categorization  in  horizontal  axis  by  vertical  axis  with  other  concept  or  categorization.  Unfortunately,  grid  structure 
can  be  difficult  not  only  to  organize  but  also  to  understand  unless  the  designer  or  user  recognizes  the 
interrelationships  between  concepts  and  categorizations  of  whole  information.  Therefore,  it  is  best  for  who  already 
have  knowledge  on  topics  and  its  organization  (Lynch  1999). 


3)  Hierarchy 

A hierarchical  structure  bas  been  used  widely  in  the  web  site  (Lynch,  1999),  on-line  documentation  (Gloor 

1997) ,  information  retrieval  system  (Rosenfeld  and  Morville  1998),  and  computer-based  instructional  programs  and 
training  (Lai,  1994;  Jonassen,  1986).  Organization  of  hierarchical  structure  starts  with  general  concept  or  topic  into 
specific  ones,  which  are  in  turn  divided  into  more  specific  to  individual  based  on  precedence  and  significance 
(Lynch,  1999;  Norman,  1988;  Sano,  1996).  Users  can  move  from  general  to  specific  and  back  to  general  through  the 
linking  system  presented  by  menu  design.  The  advantage  of  hierarchical  structure  is  that  familiar  to  most  people 
since  it  reflects  the  structure  of  printed  materials  (Shneiderman  and  Kearsley  1989),  it  is  ubiquitous  in  everyday  life 
(Sand  1996),  and  it  is  the  most  natural  structures  for  organizing  levels  of  abstraction  (Gloor,  1997).  Because  of  its 
familiarity  and  pervasiveness,  users  can  easily  and  quickly  understand  web  sites  and  hypertext  system  (Rosenfeld, 

1998)  without  heavy  cognitive  overload.  In  organized  hierarchical  information  structure,  the  disorientation  problem 
can  be  minimized  and  users  can  easily  navigate  among  information  nodes  by  following  the  linking  system  (Conklin, 
1987).  They  are  able  to  develop  a mental  model  of  the  site's  structure  and  their  location  within  that  structure 
(Rosenfeld,  1998). 
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4)  Network 

Network  structure  is  composed  of  associative  links  that  network  related  concepts  and  information  together. 
While  this  structure  provides  relatively  effective  navigation  mechanism,  users  can  easily  get  lost  or  disoriented 
because  it  is  so  hard  for  users  to  understand  and  predict  entire.  This  structure  can  apply  on  small  size  of  Web  site  for 
high  level  of  training,  such  as  simulation  game,  strategy  training.  It  would  be  difficult  to  manage  linking  system  if 
size  is  getting  bigger  (Lynch,  1999). 


Most  complex  web  sites  share  aspects  of  all  four  types  of  information  structures.  Except  in  sites  that 
rigorously  enforce  a sequence  of  pages,  users  are  likely  to  use  any  web  site  in  a free-form  "web-like"  manner,  just  as 
most  non-fiction  or  reference  books  are  used.  But  the  nonlinear  usage  patterns  typical  of  web  surfers  do  not  absolve 
developers  of  the  need  to  organize  their  thinking  and  present  it  within  a clear,  consistent  structure  that  comple  ments 
design  goals  for  the  site.  Table  2 shows  the  characteristics  of  four  basic  organization  patterns  explained  above  in 
terms  of  the  complexity,  flexibility,  easy-to-navigation,  domain  knowledge,  and  cognitive  load. 

Table  2.  Summary  of  Information  Structure 


Complexity 

Flexibility 

Navigation 

Domain 

Knowledge 

Cognitive 

Load 

Linear 

Low 

Low 

Very  Easy 

Low 

Low 

Grid 

High 

High 

Middle 

High 

High 

Hierarchy 

Middle 

Middle 

Easy 

Middle 

Middle 

Network 

Very  High 

Very  High 

Hard 

Very  High 

Very  High 

Depth  and  Breadth 

The  most  discussed  issues  in  information  structure  are  branching  factors,  depth  (the  number  of  vertical 
levels  or  links)  and  breadth  (the  number  of  horizontal  documents  or  links)  (Shneiderman  1997).  Much  has  been 
known  about  tradeoff  of  "depth  and  breadth  ",  and  "complexity  and  flexibility  issues  (Larson,  1998;  Shneiderman, 
1997).  The  previous  results  generally  support  that  deeper  hierarchical  structure,  the  less  helpful  and  less  favorable 
for  navigation.  Users  have  to  move  through  more  levels  to  located  information  in  the  deeper  structure  and  should 
need  more  not  only  cognitive  effort  but  also  physical  effort  to  interact  with  more  displays  and  links  for  information 
searching  (Shneiderman,  1989). 

Guidelines  for  building  an  integrated  EPSS 

Design  of  integrated  EPSS  can  be  affected  by  many  factors.  These  factors  include  as  follows: 

1)  Analyzing  the  nature  of  information  or  knowledge 
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In  building  an  effective  web-based  integrated  EPSS,  one  of  the  most  important  things  developers  have  to 
keep  in  mind  is  to  analyze  the  nature  of  information  or  knowledge  to  be  supported  by  the  site  as  exactly  as  possible. 
Depending  on  the  professional  medical  knowledge  with  a complex  information  structure  or  the  general  knowledge 
with  a simple  information  structure  for  its  delivery,  it  might  be  totally  different  in  terms  of  the  interface  design, 
information  structure,  and  the  depth  and  breadth  of  the  information. 

2)  Analyzing  user  characteristics 

In  order  to  build  an  effective  web-based  integrated  EPSS,  developers  should  clearly  analyze  the 
characteristics  of  the  target  users  of  the  site.  That  is,  the  developers  need  to  make  a more  detailed  analysis  of  the 
target  audiences,  such  as  teenagers,  professionals  of  some  fields,  and  so  forth.  Based  on  that  analysis,  the  developers 
can  grasp  the  major  concerns  of  the  audiences  and  predict  their  preferences.  This  will  lead  to  providing  appropriate 
functions  just  in  time  when  such  needs  arise.  If  the  EPSS  is  developed  without  considering  the  users’  characteristics 
including  their  preferences,  the  site,  no  matter  how  well  organized  or  useful,  will  not  appeal  to  the  some  portion  of 
the  users  any  more. 

3)  Localizing  contents  and  categories 

Due  to  the  rapid  development  and  spread  of  communication  technology,  traditional  borders  disappeared  in 
the  terms  of  information  sharing.  In  other  words,  we  can  share  any  content  as  soon  as  it  is  developed.  Considering 
the  importance  of  sharing  information,  this  is  very  desirable.  The  problem  arises  for  the  fact  that  every  country  has 
unique  cultural  and  social  context  as  well  as  different  language.  Even  in  the  same  country,  there  can  be  various 
cultural  groups.  Certain  problems  are  expected  in  the  use  of  certain  contents  in  different  countries  or  cultural  groups. 
For  example,  item  T,  which  is  categorized  as  ’A’  in  one  country,  can  be  categorized  as  ’B’  in  another  country.  There 
is  a need  to  readjust  contents  or  categories  in  terms  of  cultural  and  social  contexts.  The  easiest  solution  is  to  prepare 
different  versions  for  different  groups.  Here,  the  developers  may  most  likely  face  the  effect  of  tradeoff  due  to 
additional  cost. 

4)  Analyzing  client  system  capability 

The  developers  need  to  be  cognizant  of  the  system  environment  under  which  the  users  are  learning.  This  is 
closely  associated  with  analyzing  users  characteristics.  While  the  latter  is  more  concerned  with  physical  and 
affective  characteristics  of  the  users,  the  former  is  more  focused  on  their  learning  milieu.  With  the  rapid 
development  of  technologies,  we  have  recently  seen  many  products  with  newly -added  features.  Even  the  products 
marketed  in  the  same  year  maybe  different  from  one  another  in  terms  of  system  capability.  In  this  vein,  the 
developers  need  to  continue  their  analysis  on  the  learning  environment  of  the  users. 

5)  Analyzing  information  accessibility 

We  cannot  overemphasize  the  importance  of  information  accessibility  for  building  an  effective  web-based 
integrated  EPSS  because  the  EPSS  is  delivered  via  electronic  telecommunication  technology  like  Internet.  While 
most  users  with  intent  to  access  to  Internet  use  LAN  in  their  office,  they  rely  on  modem  at  home.  So,  their  major 
concern  is  how  expeditiously  and  conveniently  they  can  access  the  needed  information.  When  they  cannot  obtain  the 
information  they  need  at  their  convenience,  they  will  experience  inconveniences  arising  from  the  use  of  the  site. 

This  will  eventually  turn  the  users  away  from  the  site.  Therefore,  high  level  of  information  accessibility  is  very 
important  factor  of  a successful  EPSS. 

6)  Planning  information  updatability 

Information  updatability  is  another  critical  issue.  In  EPSS,  this  factor  is  more  important  than  any  other 
fields,  because  the  users  want  information,  which  is  most  recently  updated  at  their  convenience.  In  this  regard,  two 
most  important  issues  arise  as  to  how  promptly  the  information  can  be  updated  and  who  will  assume  the  role  of 
updating  the  information. 

7)  Harmonizing  internal  relationships  between  other  information  sources 

Information  in  EPSS  is  sophisticated  rather  than  fragmentary.  As  seen  in  Figure  1,  even  the  same  types  of 
information  can  be  restored  and  delivered  in  various  ways.  Some  information  can  be  delivered  by  visual  formats 
such  as  text  or  graphics  or  by  audible  formats  such  as  audio.  In  this  sense,  harmonizing  internal  relationship  between 
other  information  sources  is  very  important.  If  the  internal  relationship  is  not  well  organized,  the  EPSS  is  just  an 
archive  of  unrelated,  individual  information. 
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Figure  1.  Overall  Structure  of  Web-based  Integrated  EPSS 

8)  Integrating  external  relationships  between  other  EPSS 

Finally,  integrating  external  relationships  between  other  EPSS  is  another  important  issue.  The  developers 
should  consider  that  the  EPSS  has  to  be  related  to  external  sources  at  some  degree  in  order  to  maximize  its 
effectiveness.  Just  as  harmonizing  internal  relationship  between  other  information  sources  was  important, 
integrating  external  relationship  between  other  EPSS  in  a harmonious  way  is  a very  important  factor  of  successful 
EPSS. 

All  of  these  factors  impact  the  way  to  present  and  structure  the  web-based  integrated  EPSS.  In  order  to 
maximize  the  usability  of  web-based  integrated  EPSS,  therefore,  we  need  to  identify  and  select  appropriate  menu 
systems  and  information  structures. 
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Abstract 

The  purpose  of  this  presentation  is  to  discuss  how  general  problem-based  learning  model  and  social- 
constructivist  perspectives  are  applied  to  design  and  develop  a web-based  science  program,  which 
emphasizes  inquiry-based  learning  environment  for  5th  grade  students.  The  presentation  also  deals  with 
general  features  and  the  learning  process  of  web-based  science  program,  including  teacher's  action,  the 
student's  learning  activity,  and  cognitive  tools  to  scaffold  the  student's  inquiry.  A major  goal  of  PBL  in 
web-based  science  program  is  to  help  students'  construct  scientific  thinking  and  problem  solving  skills 
through  a set  of  interaction  including  teacher-to -student,  student  to-student,  group-to-group,  student- 
scientist,  and  student-cognitive  tools.  The  main  stages  of  PBL  in  web-based  science  program  as  a learning 
cycle  are  1)  '‘Your  Challenge  " which  students  engage  in  an  authentic  problem  situation,  2)  “Plan  Inquiry'' 
which  students  identify  what  they  know,  what  they  need  to  know  to  solve  a problem,  and  how  they  go  about 
finding  out,  3)  “Explore  Resources''  which  students  gather  information,  learn  primary  concepts  and 
principles  necessary  to  solve  the  problem,  and  contact  with  scientist  to  acquire  scientific  thinking  process 
for  solving  problem  and  their  perspective  about  the  problem,  4)  “Generate  Alternative  Solutions''  which 
students  come  up  with  alternative  solutions  to  the  problem  after  analyzing  information  and  data,  5) 
“Reflection  &,  Presentation " which  focus  on  feedback  from  teacher,  students,  and  scientist  about 
alternative  solutions.  After  revising  their  solutions,  students  present  their  best  solutions.  In  this  learning 
cycle,  scaffolding  or  cognitive  tools  support  students'  activity  to  communicate  scientific  arguments  and  to 
solve  the  problem. 

Why  PBL? 

Problem-Based  Learning  provides  minds-on  and  hands-on  experience  for  students  to  engage  in  real  world 
problem  (Torp  & Sage,  1998).  During  the  Problem-based  Learning,  students  (1)  play  the  role  of  stakeholders  in  the 
problem  scenario,  (2)  engage  in  an  ill-structured  problematic  situation,  (3)  identify  what  they  know  and  need  to 
know,  (4)  define  the  problem  to  focus  further  investigation,  (5)  gather  and  share  information  related  to  the  problem 
situation,  and  (6)  generate  several  possible  solutions  and  identify  the  solution  of  best  fit  (Torp  & Sage,  1998;  Savery 
& Duffy,  1996;  Albanese,  A.  M.,  & Mitchell,  S.,  1993). 

Hewitt  and  Scardamalia  (1996)  addresses  the  why  PBL  works. 

• Inquiry  is  focused  upon  communal  problems  of  understanding  where  meaning  is  negotiated 
through  questioning,  theory  refinement,  and  dialogue. 

• Students’  ideas  about  what  they  need  to  know  become  the  focus  of  inquiry. 

• Knowledge  is  shared  and  held  collectively.  New  information  that  is  shared  has  the  potential  of 
shaping  subsequent  investigations  by  others. 

• The  artifacts  of  student  inquiry  are  made  public  and  used  in  knowledge  production.  These  include 
problem  maps  that  integrate  information  and  highlight  connections,  graphic  organizers  that  help 
visualize  patterns  and  relationships,  and  loop  writing  that  provides  opportunities  for  students  to  , 
respond  to  the  thinking  of  their  peers. 

• Responsibility  for  planning,  organizing,  questioning,  and  summarizing  is  shared  among  the 
students  and  facilitated  by  the  teacher. 

Problem-based  learning  helps  students  develop  and  practice  ways  to  solve  problems  that  are  in  some  way 
relevant  to  issues  they  can  relate  to.  By  allowing  students  to  work  through  problems,  they  develop  reasoning  skills 
and  are  able  to  think  through  real  problems  more  critically.  Students  are  generating  their  knowledge  by  themselves 
with  experiences  and  prior  knowledge. 
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Providing  teachers*  space  about  how  to  coach  will  give  teachers,  who  are  using  PBL  in  their  teaching,  idea 
how  to  scaffold  and  facilitate  in  PBL  environments. 


Why  Web-Based? 

In  this  learning  environment,  students  should  be  provided  feedback  to  responses  made  during  the  learning 
process.  It  may  be  necessary  for  teacher-to-student,  student  to-student,  and  group -to -group,  to  communicate  in  the 
same  time  frame  and  have  coordinated  access  to  the  same  World-Wide  Web  pages.  In  order  to  provide  responsive 
feedback,  web-based  learning  environment  with  interactive  media  (BBS  or  Email)  is  need  (Dick  & Carey,  1996). 

This  is  designed  for  classroom  study  with  computers  connected  with  the  Internet.  Internet  will  give  students 
have  full  of  chance  to  search  resources.  Another  reason  for  choosing  web-based  learning  environment  is  for  any 
teachers,  who  have  the  Internet  connection  in  their  classroom,  and  who  want  to  use  this  learning  environment. 

The  purpose  of  this  learning  environment  is  to  discuss  how  general  problem-based  learning  model  and 
social-constructivist  perspectives  are  applied  to  design  and  develop  a web-based  science  program,  (comma)  which 
emphasizes  inquiry -based  learning  environment  for  5th  grade  students.  This  project  also  deals  with  general  features 
and  the  learning  process  of  web-based  science  program,  including  teacher’s  action,  the  student’s  learning  activity, 
and  cognitive  tools  to  scaffold  the  student’s  inquiry. 

A major  goal  of  PBL  in  web-based  science  program  is  to  help  student  construct  scientific  thinking  and 
problem  solving  skills  through  a set  of  interaction  including  teacher-to-student,  student  to-student,  group-to -group, 
student-ficientist,  and  student -cognitive  tools.  In  this  learning  environment,  teachers  are  active  coaches,  and  students 
are  active  students. 


Learning  Stages  of  PBL 


PBL  is  an  instructional  method  characterized  by  the  use  of  "real  world"  problems  [Figure,  1]  ‘‘Problem”  as 
a context  for  students  to  learn  critical  thinking  and  problem  solving  skills,  and  acquire  knowledge  of  the  essential 
concepts  of  the  course.  Using  PBL,  students  acquire  life  long  learning  skills,  which  include  the  ability  to  find  and 
use  appropriate  learning  resources.  These  problems  are  used  to  engage  students'  curiosity  and  initiate  learning  the 
subject  matter.  PBL  prepares  students  to  think  critically  and  analytically,  and  to  find  and  use  appropriate  learning 
resources. 
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Figure  4 Problem  (Screenshot) 


After  meeting  the  real  world  problem  scenario,  students  will  be  engaged  in  the  five  stages 
to  solve  the  problem.  The  main  stages  of  PBL  in  web-based  science  program  for  studerts  are  (1) 
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“Your  Challenge”,  (2)  “Plan  Inquiry”,  (3)  “Explore  Resources”,  (4)  “Generate  Alternative 
Solutions”,  and  (5)  “Reflection  & Presentation 

Your  Challenge 


Students  engage  in  an  authentic  problem  situation.  In  “Your  challenge”  stage,  students  assign  the  role  based 
on  problem  scenario  and  situation  by  discussions  and  negotiations  with  other  members  of  group  [Figure  2].  Groups 
were  made  up  of  three  to  four  students. 


Figure  2 Your  Challenge  Stage  (Screenshot) 

Plan  Inquiry 


Students  identify  what  they  know,  what  they  need  to  know  to  solve  a problem,  and  how  they  go  about 
finding  out  [Figure  3].  ”I  Know  This"  which  students  will  have  to  prove  what  they  know  from  meeting  the  problem 
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Figure  3 Plan  Inquiry  Stage  (screenshot) 
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as  well  as  from  their  experiences,  and  document  the  information  on  a "Know"  chart.  "I  need  to  know  this",  which 
document  what  critical  issues  are  to  finding  out  more  about  the  problem.  Determine  students’  initial  information 
gathering  effort.  "How  do  1 get  this"  which  determine  how  to  get  information,  and  how  to  hunches  students  have 
about  what  may  be  causing  the  problem  or  what  may  be  part  of  the  solution.  Under  the  guidance  of  a teacher,  the 
students  must  take  responsibility  for  their  own  learning,  identifying  what  they  need  to  know  to  better  understand  and 
manage  the  problem  on  which  they  are  working  and  determining  where  they  will  get  that  information.  This  allows 
each  student  to  personalize  learning  so  as  to  concentrate  on  areas  of  limited  knowledge  or  understanding,  and  to 
pursue  areas  of  interest. 


Explore  Resources 

Students  gather  information,  learn  primary  concepts  and  principles  necessary  to  solve  the  problem,  and 
contact  with  scientist  to  acquire  scientific  thinking  process  for  solving  problem  and  their  perspective  about  the 
problem., Resources  will  be  different  by  student’  role,  for  example,  ecologist  will  have  and  gather  information  that 
appropriate  and  relate  information  to  him/her,  so  that  student  will  became  an  ecologist. 


Generate  Alternative  Solutions 

Students  come  up  with  alternative  solutions  to  the  problem  after  analyzing  information  and  data,  and 
recommend  solutions  based  on  the  information  students  have  gathered  using  the  Decision-Making  Matrix  [Figure  4]. 
From  the  information  students  have  gathered,  students  will  have  to  come  up  good  judgments  supported  by  criteria, 
context,  self-correction,  and  explicit  reasons  for  drawing  a conclusion,  note  the  strategy,  pros,  cons,  and 
consequences  which  are  discussed  with  your  group  member  in  the  box,  and  assign  a score  to  each  pro  or  con,  from  1 
(weak)  to  5 (strong)  using  the  Decision-Making  Matrix  for  the  their  final  solution. 


Figure  4 Possible  Solution  Stage  (screenshot) 

Reflection  & Presentation 

Focus  on  feedback  from  teacher,  students,  and  scientist  about  alternative  solutions  [Figure  5].  After 
revising  their  solutions,  students  present  their  best  solutions.  After  revising  their  solutions,  students  present  their  best 
solutions.  When  students  finish  each  "Plan  Inquiry",  and  "Solutions"  they  will  show  this  stage.  By  clicking  button, 
which  is  under  "Plan  Inquiry",  and  "Possible  Solutions",  each  student  can  review  other  student's  work.  And  send  an 
e-mail  to  give  feedback  or  comments. 
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Figure  5 Reflection  and  Presentation  Stage  (screenshot) 


Ask  Teacher 

In  the  stages  of  Plan  Inquiry  and  Possible  Solution,  students  will  have  “Ask  Teacher”  that  works  as  a model 
for  supporting  students’  work  through  the  process  of  problem  solving. 

Teachers’  guidelines 

In  each  stages,  teachers  will  have  guidelines  about  “what  is  coaching”  and  “how  and  what  do  I 
coach?”  [Figure  6].  The  PBL  teachers  are  facilitator  or  activators  of  the  students’  learning  initiatives.  Working  to 
guide,  motivate,  and  probe  the  students’  reasoning  process  as  they  journey  through  the  problems  rather  than  to  direct 
it  is  often  a less  comfortable  role,  and  requires  a blend  of  creativity,  ingenuity,  and  flexibility  in  its  implementation 
Designing  problem-based  learning  in  web-based  environment  is  more  beneficial  to  provide  and 
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Figure  6 Teachers*  Guideline  (screenshot) 

support  scaffolding.  Most  web-based  instruction  just  ignores  the  teacher’s  role  and  interface.  We  can  hardly  capture 
the  teacher’s  work  place  in  the  web-based  learning.  Teachers  just  give  students  feedback.  However,  teachers  only 
guide  students  through  the  process  of  problem  solving,  and  they  provide  no  direct  answers  for  the  questions. 
Teachers  using  PBL  face  the  difficult  task  of  guiding  without  leading,  assisting  and  directing.  Teachers’  work  in 
PBL  involves  guiding  students  through  the  process  of  developing  possible  solutions,  and  determining  the  best 
solutions  with  the  justification. 

Teachers’  role  should  be  in  form  of  questioning,  cueing,  prompting,  coaching,  modeling  ideal  performance, 
mentoring,  telling  or  discussing.  Teachers  can  maintain  joint  attention  on  a goal  by  adopting  PBL  process,  using  role 
and  drama,  managing  group  work,  and  monitoring  student  engagement. 

If  student  provides  an  accurate  but  incomplete  explanation,  teachers  are  likely  to  provide  a recast  or 
expansion  of  the  student’s  explanation.  Teachers  deliberately  plan  their  presentation  of  problems  to  facilitate  the 
asking  of  though-provoking  questions  that  involve  the  comparison  of  different  problems  or  problem-solving 
techniques. 
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“Discussion**  according  to  The  American  Heritage  College  dic*tion*ary  (1997)  is  "a  formal  discourse  on  a 
topic.*’  Likewise  online  discussion  is  a formal  discourse  on  a topic;  however,  it  is  done  electronically  either 
synchronously  or  asynchronously. 

Online/web-based  course  instructors  often  find  that  they  have  more  discussion  with  their  online  students 
than  their  classroom  students  (Ekhaml,  2000).  However,  there  is  still  a perception  that  online/web-based  learning  is 
too  textual,  requires  the  student  to  learn  on  his/her  own,  and  alienates  students  from  the  instructor  and  other  students 
in  the  class  (Hutton,  1999).  For  these  reasons,  discussion  is  important  to  an  online/web-based  course.  Discussion 
helps  reduce  feelings  of  isolation,  facilitates  a spirit  of  community,  and  supports  achievement  of  course  goals.  Being 
able  to  engage  students  in  discussion  that  relates  to  the  course  when  teaching  an  online  course  can  sometimes  be  a 
challenge.  Keeping  the  discussion  to  the  topic  can  be  an  even  bigger  challenge. 

As  with  traditional  face-to-face  classes,  in  electronic/online  classes  there  is  instructor-to-class  discussion, 
student-to -instructor  discussion  (and  vice  versus)  and  student-to-student  discussion.  Fortunately,  technology 
provides  several  media  that  supports  efforts  to  engage  students  in  discussion.  Discussions  can  be  conducted  with 
email,  threaded  discussion,  synchronous  chat,  list  serves/bulletins,  and  planned  face-to-face  meetings. 

Ways  in  which  online  discussions  can  occur 
Email 

Email  enables  the  instructor  to  communicate  with  one  or  many  students  asynchronously.  In  addition,  to 
giving  specific  feedback  to  queries  from  students,  the  instructor  can  email  announcements  or  content-related 
information  to  the  students.  Email  can  also  be  used  for  office  hours  - students  can  email  the  instructor  during  the 
specified  office  hours  and  know  that  a response  is  forthcoming  within  the  confines  of  the  office  hours. 

Often  students  will  communicate  more  frequently  with  an  instructor  via  email  than  they  do  face-to-face 
when  in  a traditional  classroom.  Additionally,  students  can  interact  with  one  another  via  email.  Students  can  submit 
individual  assignments,  work  collaboratively  on  assignments,  request  technical  assistance  or  query  the  instructor 
about  content-related  issues  via  email.  Instructors  can  return  assessments  to  assignments,  provide  support  for 
technical  and/or  content-related  issues,  and  make  announcements  via  email. 

A consideration  to  communicating  via  email  is  that  students  tend  to  want  the  responses  to  be  immediate; 
i.e.,  within  hours  or  the  same  day.  The  instructor  should  make  clear  how  often  email  is  retrieved  and  provide  a 
timeframe  in  which  students  can  expect  a response.  Another  consideration  to  using  email  is  the  use  of  language. 
Students  tend  to  communicate  differently  when  communicating  via  email.  For  example,  their  frustration  may  read 
like  anger,  statements  originally  meant  to  be  humorous  may  be  offensive,  and  persistence  may  appear  to  be  coercive. 
To  minimize  frustration  and  misunderstanding  the  instructor  should  identify  expectations  for  communicating  via 
email.  The  instructor  should  be  sure  to  tell  students  the  timeframe  that  responses  will  be  given  to  their  emails  and 
whether  responses  will  be  given  over  the  weekends  and  during  semester  breaks.  Also,  a few  rules  of  Netiquette  with 
the  students  are  helpful.  A few  rules  of  Netiquette  regarding  email  include 


• Never  say  in  email  what  you  would  not  say  to  your  reader's  face 

• Be  ethical 

• Proofread  before  clicking  the  send  button 

• Do  not  type  in  all  capital  letters;  it  reads  as  though  you  are  shouting 

• Be  polite 

• Be  concise 

• Fill  in  the  subject  line 

• Avoid  spamming 

• Include  a signature  line  that  contains  your  full  name,  telephone  number,  office  address,  and  email 
(The  Netiquette  Home  Page  http://www.albion.com/netiquette  - available  1 1/4/Oland  Dark 
Mountain's  Netiquette  Guide  http://www.darkmountain.com/netiquette  - available  1 1/4/01) 
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The  use  of  emoticons  and  acronyms  in  email  are  also  useful.  They  enable  the  writer  to  say  something  or 
portray  an  emotion  concisely;  for  example,  AFAIK  means  ’as  far  as  I know’,  NRN  means  ’no  reply  necessary’  and 
POA  means  ’point  of  advice’.  (Netiquette  Home  Page-A  Service  ofAlbion.com;  available  at 
http://www.albion.com/netiquette  - 1 1/4/01,  Dark  Mountain’s  Netiquette  Guide;  available  at 
http://www.darkmountain.com/netiQuette  - available  1 1/4/01) 

Listserves/Bu  lie  tins 

Listserves  are  similar  to  email  in  that  they  are  asynchronous.  Unlike  email  everyone  that  is  a member  of  the 
listserve  receives  the  submissions  to  the  listserve.  List  serves/Bulletins  support  structured  or  unstructured  discussion. 

Threaded  Discussions 

As  with  email,  threaded  discussions  allow  the  student  to  have  discussion  with  the  instructor  as  well  as  with 
classmates.  Because  threaded  discussions  are  asynchronous,  students  can  give  careful  consideration  to  their 
responses  and  post  them  at  their  convenience.  When  focused  on  a specific  topic,  the  discussion  can  be  quite 
engaging.  To  keep  the  discussion  on  track,  online  instructors  can  design  questions  that  pertain  to  a particular  article 
or  class  reading  thus  requiring  students  to  focus  on  the  course  content  for  the  discussion.  Narrowing  the  topic  to 
address  specific  questions  related  to  the  course  content  helps  to  keep  the  discussion  from  becoming  fragmented  (Ko 
& Rossen,  2001).  Beaudin  (1999)  suggests  that  online  instructors  write  questions  that  specifically  elicit  on-topic 
discussion,  provide  guidelines  for  on-topic  responses,  reword  the  original  question  when  responses  are  no  longer  on 
topic,  and  provide  a discussion  summary.  Another  means  of  keeping  the  discussion  on  track  is  for  the  online 
instructor  to  respond  to  the  students’  responses.  Students  look  to  see  if  the  instructor  is  engaged  in  the  discussion  and 
pattern  themselves  accordingly.  According  to  Ko  and  Rossen  (2001),  "when  students  see  that  an  instructor  only 
rarely  engages  with  them,  they  are  discouraged  from  posing  questions  and  comments  aimed  even  indirectly  at  the 
instructor,  and  they  may  conclude  that  the  instructor  will  be  unaware  of  what  is  going  on  in  the  classroom"  (p.227). 

Regarding  student  responses,  the  instructor  can  provide  guidelines  for  this  as  well.  Guidelines  might 
include  having  students: 

• Respond  in  complete  sentences  with  little  no  errors  in  spelling,  grammar  and  punctuation 

• Indicate  support  for  their  response  from  the  assigned  reading  or  other  literature 

• Respond  a minimum  number  of  times  and  with  more  than  a mere  ’I  agree  with  ...’ 

• Respond  within  a certain  timeframe  (e.g.,  one  week)  to  ensure  that  the  discussion  does  not  linger 
and  loose  its  momentum  all  together. 

In  some  instances,  the  online  instructor  may  feel  it  necessary  to  assign  points  to  contributions  to  threaded 
discussions.  Specific  criteria  regarding  participation  should  be  detailed  in  the  course  syllabus.  The  grading  criteria 
can  be  based  on  the  quantity  as  well  as  the  quality  of  participation.  Regarding  quantity,  the  instructor  might  require 
that  students  post  a comment  or  question  to  the  discussion  within  a specified  timeframe  (Ko  & Rossen,  2001).  The 
quality  of  the  responses  might  be  assessed  on  how  well  students  address  the  question  by  paraphrasing  information 
from  the  readings  and  citing/quoting  supporting  literature. 

Activities  that  are  supported  by  threaded  discussion  include  online  debates,  case  studies,  article  discussions, 
collaborative  projects,  guest  speakers,  and  open  discussion  (Ekhaml,  2000).  Each  of  these  activities  with  the 
exception  of  the  open  discussion  requires  that  there  be  a topic  or  question  that  drives  the  discussion;  thus,  keeping 
the  discussion  on  track. 

Synchronous  chats 

Synchronous  chats  (chats)  allow  the  students  to  feel  a since  of  community  with  one  another  and  the  teacher. 
Chats  facilitate  collaborative  projects,  enable  students  to  get  clarity  about  assignments,  and  supports  discussion 
whether  it  is  structured  (i.e.,  focused  on  a specific  topic)  or  unstructured  (i.e.,  open  forum  with  most  topics  being 
appropriate).  Chats  also  require  that  everyone  be  online  at  the  same  time  and  can  be  text  -based  or  voice  or  video 
enhanced.  Because  scheduling  can  be  a problem  for  some  students,  it  is  best  to  schedule  more  than  one  time  in 
which  students  can  participate.  As  with  threaded  discussions,  participation  expectations  (e.g.,  minimum  number  of 
postings,  quality  of  postings,  points  associated  with  postings,  identify  person  one  is  responding  to)  should  be 
specified  prior  to  the  start  of  the  chat  and  the  chat  should  have  a focus. 

Debates/collaborative  discussions,  article  discussions,  expert  speaker,  and  software  reviews  are  a few 
activities  that  are  suitable  for  chats.  Each  of  these  activities  has  a purpose,  can  be  facilitated  by  either  instructor  or 
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student,  and  can  have  a specified  time  frame.  Kirby  (1999)  found  that  students  were  more  comfortable  with  debates 
via  chats  after  the  second  occurrence.  When  students  submit  their  points  and  counterpoints,  it  can  be  confusing  and 
discussion  can  appear  chaotic  since  someone  may  be  responding  to  a person’s  comment  two  or  three  comments 
above.  Having  several  small  group  chats  might  help  alleviate  the  chaotic  nature  of  chats  since  fewer  people  are 
“chatting”. 

Planned  face-to-face  meetings 

When  logistics  permit,  planned  face-to-face  meetings  can  support  and  prompt  more  online  discussions. 
During  the  meetings,  everyone  is  able  to  put  a face  to  the  names  of  the  individuals  they  will/have  been 
communicating  with  online.  Such  name-to-face  recognition  nurtures  community  among  the  students.  Cooper  (2000) 
particularly  encourages  having  an  initial  class  meeting  because  it  lets  students  meet  the  instructor  and  each  other, 
ask  questions,  and  learn  about  the  course  assignments,  assessments,  required  materials,  office  hours  and  the  online 
software.  Subsequent  face-to-face  meetings  can  support  the  students  in  their  efforts  to  complete  assignments  and/or 
get  clarity  about  assignments  as  well  as  nurture  the  affective  needs  of  the  online  students. 

Lessons  learned 

Teaching  online  was  a learning  experience  for  me  as  well  as  for  my  students.  I had  assumed  that  my 
students  had  certain  technology  skills  and  had  taken  the  online  course  orientation,  which  would  familiarize  them 
with  the  online  software  features.  This  was  not  an  accurate  assumption,  and  I learned  that  one  should  never  assume 
anything  when  teaching  online.  Instead  I should  have  had  a face-to-face  meeting  to  meet  my  students  in  person, 
demonstrate  use  of  the  online  course,  orient  them  to  the  course  expectations,  and  as  a consequence  have  a positive 
impact  on  our  learning  community.  Fortunately,  I did  have  a face-to-face  meeting  about  a month  into  the  semester 
and  it  created  an  atmosphere  that  was  supportive  and  friendly  for  all  of  us.  The  new  atmosphere  was  evident  in  the 
email  communications  that  I received  from  the  students.  Prior  to  the  face-to-face  meeting  some  emails  were  cryptic 
and  terse  as  well  as  offensive;  after  the  face-to-face  meeting  the  emails  were  clear,  detailed  and  in  most  cases  non- 
offensive. Having  variety  in  the  ways  in  which  students  could  communicate  was  a definite  plus  as  indicated  by  my 
positive  course  evaluations  regarding  online  communication  strategies.  We  used  email,  message  boards  for 
announcements,  document  sharing  to  peer  review,  focused  threaded  discussions,  focused  chats,  and  face-to-face 
meetings.  In  an  online  course  it  is  important  to  communicate/discuss  issues  frequently.  One  means  of 
communicating  that  1 did  not  utilize  in  my  class  was  group  projects;  however,  in  subsequent  online  courses  I will. 
Collaborative  projects  increase  students’  motivation,  support  their  need  for  familiarity  with  others,  and  support  the 
learning  community  (Ekhaml,  2000). 

Knowlton,  Knowlton,  & Davis  (2000)  suggest  that  to  facilitate  discussions  for  maximum  learning  the 
instructor  should  make  students  responsible  for  participation,  model  appropriate  participation,  synthesize  students 
comments,  and  ask  questions  to  encourage  elaboration  and  clarification.  I have  learned  that  following  this  model 
will  help  me  create  an  online  learning  environment  that  is  friendly  and  supports  students  in  achieving  course  goals. 
This  is  the  type  of  online  learning  environment  that  I as  an  instructor  want  to  create  and  maintain. 
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Abstract 

The  concept  of  EPSS  (Electronic  Performance  Support  System)  originated  in  business  settings.  Recently, 
there  have  been  many  attempts  to  apply  the  concept  to  schools:  educational  EPSSs  (E-EPSSs)  have  become 
available  on  the  Web,  However,  there  is  little  evaluation  research  and  few  evaluation  frameworks  for  these 
emergent  E-EPSSs,  The  primary  purpose  of  this  article  is  to  provide  our  design  recommendations  for  how 
to  improve  the  quality  of  E-EPSSs  in  general,  based  upon  the  evaluation  of  one  specific  E-EPSS,  called 
STEPS  (Support  for  Teacher  Enhancing  Performance  in  Schools),  To  achieve  this  purpose,  the  article  first 
reviews  E-EPSSs  in  terms  of  teachers’  professional  development  and  discusses  their  encompassing  trends, 
needs,  and  definitions.  Secondly,  it  presents  an  evaluation  case  of  STEPS.  An  evaluation  perspective  called 
’’perception -oriented  usability  evaluation”  drives  the  evaluation.  Lastly,  it  lists  recommendations  for 
improving  STEPS  as  well  as  E-EPSSs  in  general  based  on  our  STEPS  evaluation  results  and  literature 
review. 


Introduction 

An  exponential  increase  in  information  requires  teachers  to  continuously  develop  their  professional  skills. 
As  a response  to  this  requirement,  many  researchers  have  proposed  creating  an  EPSS  (Electronic  Performance 
Support  System)  to  support  instructional  design  activities,  which  is  one  of  the  main  tasks  of  the  teachers,  through 
job-embedded  learning  (Reigeluth,  1999;  Gustafson,  1993),  and  to  promote  training  in  education  (Scales,  1994). 
EPSS  has  also  been  acknowledged  as  a system  that  can  assist  the  school  reform  movement  rather  than  a mere  tool 
that  may  exert  its  influence  in  only  a piecemeal  way  (Northrup  & Pilcher,  1998;  Scale,  1994).  Applying  an  EPSS 
systemically  not  only  can  alleviate  instructional  and  administrative  burdens  by  supporting  teacher  performance,  but 
also  can  provide  teachers  with  job-embedded  training  opportunities.  Using  an  EPSS,  teachers  can  receive  training 
within  their  teaching  context.  They  don’t  necessarily  need  to  leave  their  classrooms  and  school  environment  to 
improve  their  professional  skills. 

The  concept  of  EPSS  originated  in  business  settings;  however,  recently,  there  have  been  many  attempts  to 
apply  the  concept  to  schools  (Barker  & Benerji,  1995;  Northrup  & Pilcher,  1998),  and  some  educational  EPSSs  (E- 
EPSSs)  have  become  available  on  the  Web.  However,  there  is  little  evaluation  research  and  correspondingly  few 
evaluation  frameworks  for  these  emergent  E-EPSSs.  In  response  to  this  scarcity  of  evaluation  research,  this  article 
describes  an  evaluation  study  on  one  E-EPSS,  called  STEPS  (Support  for  Teacher  Enhancing  Performance  in 
Schools.)  The  article  is  composed  of  the  following  sections: 

• Overview  of  the  general  features  of  a selected  E-EPSS,  STEPS. 

• Review  of  E-EPSS  as  embedded  in  teacher  professional  development  and  its  encompassing  trends, 
needs,  and  definitions. 

• Description  of  evaluation  methodology. 

• Results  of  the  STEPS  evaluation. 

• Recommendations  for  the  improvement  of  STEPS  and  the  design  of  E-EPSS  in  general. 

STEPS  as  an  Evaluand 

STEPS  (Support  for  Teacher  Enhancing  Performance  in  Schools)  is  an  EPSS  designed  specifically  to  help 
pre -service  and  practicing  pre-K-12  teachers  develop  instructional  lessons,  units,  and  curricula  aligned  to  Florida's 
Sunshine  State  Standards.  According  to  Northrup  and  Pilcher  (1998),  the  purpose  of  STEPS  is  to  support  school 
reform  and  sustain  accountability  of  the  integrated  curriculum  that  utilizes  technology,  alternative  assessment,  and 
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diverse  learning  environments  and  its  conceptual  frameworks  include  flexibility,  learning  by  doing,  and  a user- 
designed  structure. 


Figure  1.  The  screen  shots  of  STEPS 
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STEPS  home  page 


Main  page  for  the  grades  3-5  teachers 


STEPS  is  available  through  the  World  Wide  Web  or  as  a standalone  CD-ROM.  The  web  version  of  the 
EPSS  was  selected  for  this  study.  The  first  picture  above  is  the  screen  shot  of  the  home  page  of  STEPS.  It  consists 
of  four  menus  broken  down  by  the  grade  level:  PK-2,  3-5,  6*8,  and  9-12.  If  the  user  clicks  the  signpost  marked  3-5, 
then  the  main  page  for  the  grades  3-5  teachers,  which  is  depicted  in  the  second  picture,  opens.  The  main  page  of 
grades  3-5  consists  of  two  groups  of  menus:  the  left-hand-side  frame  contains  links  to  the  Main  Menu,  which 
includes  Lesson  Architect,  Tutorial  Library,  Best  Practices,  Sample  Unit,  Web  Links,  State  Standards,  Florida 
Information  Resources  Network  (FIRM),  and  Florida  Department  of  Education  (FDE),  and  the  right-hand-side  frame 
includes  links  to  the  same  menus  in  the  left-hand  frame.  The  main  page  for  each  grade  group  has  the  same  menu 
structure. 

The  main  six  components  of  STEPS  are  Lesson  Architect,  Tutorial  Library,  Best  Practices,  Sample  Unit, 
Web  Links,  and  Coach.  Lesson  Architect  is  the  main  component  of  STEPS.  It  guides  teachers  through  the  processes 
of  instructional  design  and  curriculum  planning.  It  uses  Gagne’s  Events  of  Instruction  (1992)  and  Dick  and  Carey’s 
Systematic  Instructional  Design  Models  (1996)  as  theoretical  foundations  of  instructional  design,  and  various 
curriculum  approaches  such  as  webbing  and  threaded  curriculum  (Northrup  & Pilcher,  1998). 

Tutorial  Library  is  a collection  of  about  forty  instructional  tutorials.  The  tutorials  follow  the  four  premises 
of  the  STEPS  curriculum:  integrated  curriculum,  integrated  technology,  alternative  assessment,  and  diverse  learning 
environment  (Northrup  & Pilcher,  1998).  Some  tutorials  are  linked  from  the  Lesson  Architect  so  that  users  can  get 
tutorials  while  they  are  planning  lessons  using  the  Lesson  Architect.  This  function  demonstrates  one  of  the  main 
advantages  of  EPSS-just-in-time  support  to  the  users. 

Best  Practices  provide  a forum  for  sharing  successful  classroom  activities  that  were  developed  and  tested 
by  teachers  in  their  real  classrooms.  In  addition  to  keyword  searches,  users  can  search  activities  by  Sunshine  State 
Standard  item  or  by  theme.  They  can  also  browse  by  subject  areas.  This  provides  instructional  strategies  and  tactics 
for  designing  classroom  activities. 

Sample  Units  provide  sample  curriculum  units  created  by  teachers  of  the  same  grade  level.  Grades  3-5 
sample  units,  for  example,  center  around  an  archaeological  theme  and  provide  10-day  model  curriculum  units.  The 
model  units  utilize  teachers’  cross-curricular  connections  in  Math,  Science,  Social  Studies,  and  Language  Arts  and 
follow  the  benchmarks  described  in  Florida’s  Sunshine  State  Standards.  The  teachers  used  Lesson  Architect  to 
create  the  units. 

Web  Links  include  over  400  web  sites  relevant  to  Math,  Science,  Social  Studies,  and  Language  Art 
identified  in  Florida's  Sunshine  State  Standards.  Each  web  site  is  linked  to  a brief  description  of  the  website  and  a 
list  of  applicable  benchmarks. 

Coach  offers  three  levels  of  scaffolding:  the  "big  picture"  level,  the  "what  do  I do"  level,  and  the  "how  do  I 
do"  level.  It  models  the  Knowledge  Integration  Environment  created  by  the  University  of  California,  Berkeley 
(Bell,  Davis,  & Lynn,  cited  in  Northrup  & Pilcher,  1998).  If  users  click  “Ask  Coach’’  or  a corresponding  graphic 
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icon,  a pop-up  window  appears  and  provides  screen-  and  field -sensitive  help  such  as  descriptions  of  what  the  Lesson 
Architect  is  or  how  users  can  search  information  on  relevant  screens.  Gery  (1991)  calls  it  an  extrinsic  support  that 
is  integrated  with  the  software  but  not  a primary  function. 

Literature  Review 

E-EPSS  as  a tool  for  teacher  professional  development 

Traditional  professional  development  programs  for  teachers  have  stressed  knowledge  acquisition  through 
workshops  and  courses.  These  programs  have  had  difficulties  in  providing  sustained  support  needed  for  teachers  to 
apply  what  they  have  learned  in  their  classrooms . The  performance-centered  approach,  however,  provides  such  a 
sustained  support  by  employing  more  practical  knowledge  dissemination.  Teachers  engage  in  performing  processes 
through  which  they  find  problems,  organize  information,  and  infer  a series  of  proper  decision-making  activities 
needed  to  solve  their  classroom  problems.  Specifically,  the  new  professional  development  approach  engages 
teachers  in  a series  of  concrete  tasks  of  teaching,  assessment,  observation,  and  reflection.  It  is  grounded  in 
participant-driven  inquiry,  reflection,  and  experimentation.  It  supports  collaboration  and  knowledge  sharing  among 
teachers,  and  focuses  on  communities  of  teaching  practice  rather  than  on  individual  teachers.  It  is  intensive  and 
sustained  and  supported  by  modeling,  coaching,  and  collective  problem-solving.  Finally,  it  attempts  to  relate  itself 
to  other  aspects  of  organizational  change. 

As  a tool  for  the  performance-centered  approach  and  one  of  the  viable  alternatives  for  traditional 
professional  development,  an  EPSS  has  been  proposed  for  supporting  the  most  critical  activities  of  teachers, 
instructional  design  activities,  i.e.,  curriculum  development  and  lesson  design.  EPSS  has  been  defined  in  various 
ways;  however,  there  exist  general  agreements  on  the  major  goals  of  an  EPSS.  The  goals  are  to : (1)  provide 
"whatever  is  necessary  to  generate  performance  and  learning  at  the  moment  of  need,"  referred  to  as  "just-in-time 
training  systems  " (Geber,  1991,  p.34);  (2)  enable  "day-one  performance,"  the  idea  that  novice  users  should  be 
productive  on  the  very  first  day  that  they  start  using  a system  (Gery,  1995);  and  (3)  support  higher  levels  of 
performance  for  the  work  being  done  today,  while  helping  to  build  the  knowledge  infrastructure  for  work  to  be  done 
in  the  future  (Winslow  & Bramer,  1 994). 

In  the  education  field,  there  have  been  many  electronic  performance  applications  that  meet  or  closely  meet 
these  common  goals  of  EPSS  although  they  are  not  always  given  the  label  “EPSS”  (Collis  & Verwijs,  1995).  The 
applications  include  instructional  material  development  tools,  grade  books,  and  behavioral  management  support 
systems.  What  are  the  differences  between  these  electronic  performance  applications  for  teachers  and  E-EPSS?  E- 
EPSS  is  an  “integrated”  support  system  that  includes  tools,  expert  systems,  instructional  activities,  and  databases 
that  assist  teachers  “at  the  time  and  place  they  need  the  assistance.”  By  supporting  teacher  practices  at  the  moment 
of  need,  it  helps  teachers  to  improve  their  professional  skills.  The  electronic  performance  applications,  on  the  other 
hand,  are  single  tools  that  support  teachers  in  performing  specific  tasks.  They  are  developed  primarily  as 
supplementary  instructors,  rather  than  as  teacher  professional  development  tools,  that  usually  assist  students’ 
learning  activities.  In  addition,  they  are  based  on  traditional  classroom  practices  that  assume  education  is  a delivery 
or  transfer  of  knowledge  (Chiero,  1996;  Collis  & Verwijs,  1995). 

The  recent  advance  of  the  Internet  and  other  computer  technology  has  created  opportunities  for  teachers  to 
use  various  Internet-based  communication  functions  such  as  emails,  distribution  lists,  and  forums.  Using  these 
tools,  teachers  can  now  consult  with  experts  in  the  areas  where  they  have  questions  and  share  their  knowledge  and 
experiences  with  other  teachers  effectively  and  efficiently.  Several  examples  of  EPSSs  that  utilize  such 
communication  functions  are  currently  available  on  the  web.  The  STEPS  web  site  available  at 
http://143.88.86.98/pacee/steps/welcome.cfm  (or  at  http ://www. ibinder.uwf.edu/steps/welcom.cfmI  is  an  EPSS 
designed  specifically  to  help  K-12  teachers  to  develop  instructional  lessons,  units,  and  curricula.  Another  example. 
Pathways  to  School  Improvement,  available  at  http://www.ncrel.org/sdrs/pathwavg.htm  is  designed  primarily  to 
assist  high  school  teachers  with  their  curriculum  development. 

E-EPSS  Design 

Many  design  guidelines  and  strategies  for  general  EPSS  can  be  applied  to  the  design  of  E-EPSS.  However, 
in  designing  E-EPSS,  special  attention  must  be  paid  to  reflecting  teachers’  unique  professional  characteristics  onto 
the  system.  This  section  discusses  design  suggestions  adopted  from  the  general  EPSS  design  literature. 

Components  of  E>EPSS. 
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Leighton  (1997)  synthesizes  the  ideas  of  Geiy  (1991,  1995)  and  Raybould  (1990),  and  contends  that  an 
EPSS  has  four  typical  components:  tools,  an  information  base,  an  advisor,  and  learning  experiences.  Similarly,  Carr 
(1992)  explains  that  an  EPSS  can  play  the  roles  of  a librarian,  an  advisor,  and  an  instructor.  Even  though  researchers 
name  components  of  E-EPSS  differently,  their  classifications  have  commonalties.  Typically,  an  B-EPSS  is 
composed  of  tools,  a database,  an  expert  system,  and  instructions  (Gery,  1991;  Raybould,  1990).  The  following  list 
summarizes  functions  and  examples  of  each  component: 

• Tools:  Tools  usually  embody  recommended  procedures  or  best  practices  that  should  be  employed 
by  the  user  (Reeves,  1995).  Examples  of  tools  include  word  processing,  spreadsheets , templates, 
and  forms . 

® Database:  A dynamic  EPSS  includes  an  infobase  and  users  supply  much  of  its  content.  The  shared 
base  of  user  experiences  grows  over  time,  making  the  infobase  increasingly  valuable  to 
organizations  (Laffey,  1995).  An  infobase  may  contain  on-line  documents,  reference  material, 
case  history  data,  etc. 

• Expert  system:  An  expert  system  usually  provides  two  distinct  types  of  support:  proactive  support 
and  reactive  support.  Proactive  support  is  usually  delivered  through  a coach  that  provides 
assistance  in  setting  goals  and  monitoring  task  completion.  Reactive  support  is  delivered  through 
context-sensitive  on-line  help  that  assists  users  when  they  have  reached  an  impasse  and  cannot 
proceed  without  overcoming  a problem  in  using  the  software. 

• Instructions:  Instructions  typically  include  Computer-Based  Training  (CBT),  but  not  in  a 
traditional  form  of  CBT.  Traditional  CBT  might  employ  a sequential  approach  in  providing  a 
four-hour  course  on  instructional  design  process.  Learning  experiences  within  an  EPSS,  however, 
must  be  organized  into  capsules  that  contain  five  to  fifteen  minutes  of  instruction.  The  capsules 
typically  deal  with  specific  topics  that  can  be  accessed  while  tasks  are  being  performed.  Examples 
of  instructions  include  multimedia,  CBT,  tutorials,  simulations,  and  scenarios. 

Depending  on  the  scope  and  nature  of  an  EPSS  and  technological  platform,  an  EPSS  is  made  up  of  a 
combination  of  at  least  one  or  more  of  these  four  components. 

Interface  Design  of  E>EPSS. 

User  interface  is  the  single  most  important  element  of  a successful  electronic  performance  support  system 
(Gery,  1995;  Cole,  Fisher  & Saltzman,  1997).  It  is  important  to  design  the  interface  of  an  E-EPSS  in  a way  that  will 
support  teachers'  performance  (Law,  Okey,  & Carter,  1995).  To  do  so,  designers  need  to  consider  teachers’  mental 
models  about  teaching  (i.e.,  what  teachers  think  about  instruction),  workflow,  and  daily  activities.  The  interface  that 
follows  teachers’  natural  workflow  using  screen  metaphors  that  are  familiar  to  them,  facilitates  understanding  of  the 
functions  of  an  EPSS  and  accordingly  reduces  time  needed  for  training.  Hansen  and  Perry  (1993)  argue  that  long- 
term success  of  a system  depends  on  teachers’  degree  of  comfort  and  confidence  in  using  a system. 

As  ways  to  capture  teachers’  mental  models.  Law  et  al.  (1995)  suggest  the  case-based  reasoning  approach 
through  which  one  can  analyze  complex  problems.  They  also  recommend  conducting  a task  analysis  to  determine 
what  task  components  a product  should  include  and  how  each  component  contributes  to  the  overall  product.  To 
identify  necessary  components  to  be  included,  it  is  necessary  to  analyze  the  actual  daily-based  tasks,  the 
performance  of  these  tasks,  and  the  elements  that  can  alter  each  task  (Moore,  1998).  The  analysis  techniques  of 
EPSS  task  analysis  are  similar  to  those  of  ISD  (Witt  & Wager,  1994).  They  include  interviews,  observations, 
questionnaires,  and  small -group  discussions.  EPSS  task  analysis,  however,  requires  collecting  data  from  both 
experts  and  novices,  especially  the  data  on  their  cognitive  work  processes,  unique  professional  demands,  and  job- 
specific  situations  (Villachica  & Stone,  1999;  Stevens  & Stevens,  1995).  In  other  words,  the  components  and 
content  of  an  EPSS  for  teachers  need  to  reflect  teachers’  unique  workload  in  different  types  of  schools - 
kindergarten,  elementary  school,  and  secondary  school.  Orey,  Moore,  Hardy,  and  Serrano  (1997)  report  that  middle 
school  teachers  spend  an  average  of  3 1 .6  minutes  per  day  preparing  resources  and  49.4  minutes  per  day  planning 
lessons.  Depending  on  types  of  school  categories  of  teachers’  workloads  and  proportion  of  time,  time  breakdown  is 
somewhat  different.  However,  they  are  similar  in  that  deskwork  (i.e.,  grading)  consumes  a large  portion  of  the 
teachers’  office  time. 

The  structure  of  an  EPSS  should  be  easy  to  use,  flexible,  and  tailored  for  end-users  having  different  needs 
and  different  expertise  (Hansen  & Perry,  1993;  Remmers,  1998).  A solution  to  accommodate  different  levels  of 
competency  of  teachers  is  not  to  make  all  information  directly  visible,  but  to  make  it  accessible,  for  instance  by 
search  tools  that  enable  end  users  to  find  precisely  the  information  they  need  (Sherry  & Wilson,  cited  in  Remmers, 


1998).  The  structure  of  an  EPSS  must  be  customizable  for  the  needs  of  different  districts,  schools,  and  teachers.  In 
addition,  cognitive  loads  and  relational  maps  can  be  considered  in  design. 

Raybould  (cited  in  Gery,  1991)  suggests  six  effective  screen  structures  for  different  information  maps: 
single  frame,  tree,  network,  linear  format,  rule,  and  animation.  The  Yale  Style  Manual  (1999)  introduces  structures 
of  Web-based  EPSS  that  include  sequence,  grid,  hierarchy,  web  structure,  and  empirical  structure.  Since  each 
structure  has  its  own  strengths  and  needs,  the  developer  can  choose  the  system  which  best  matches  the  prospective 
users'  characteristics.  For  example,  empirical  structure  is  an  effective  way  to  organize  a less-abstract  view  of  the 
content  when  the  users  are  novices  in  the  field  (Remmers,  1998). 

Carroll’s  (1998)  study  on  text  interface  led  to  the  design  of  minimal  manuals  that  drastically  cut  verbiage, 
encourage  active  involvement  with  hands  on  experiences  as  soon  as  possible,  and  promote  guided  exploration  of 
system  features.  According  to  Paivio  (1990,  1991),  clear  texts  or  images  only  are  better  sometimes  than  unclear  texts 
with  images  and  vice  versa.  To  employ  graphics  in  the  design  of  EPSS,  it  is  necessary  to  understand  how  graphics 
and  other  media  (e.g.,  texts,  audio,  and  videos)  are  cognitively  processed  and  affect  learning.  The  dual  coding 
theory  by  Paivio  (1990,  1991)  describes  how  graphics  become  associated  with  texts  in  space  and  time.  It  also 
describes  ways  to  organize  materials  according  to  the  students’  previous  experiences. 

Lastly,  an  action-oriented  interface  should  anchor  tools  in  the  corresponding  task  domains  and  support  error 
recognition  and  recovery  in  addition  to  the  users’  performance. 

Usability  Evaluation 

Amidst  the  rise  of  the  user-centered  design  principle,  many  system  design  institutions  have  realized  the 
importance  of  usability  evaluation  and  are  practicing  it  as  an  ongoing  system  design  and  development  process. 

The  formal  definition  of  usability  written  by  International  Standards  Organization  (ISO,  9241  -1 1)  is  "the 
effectiveness,  efficiency,  and  satisfaction  with  which  specified  users  achieve  specified  goals  in  a particular 
environment"  (Bevan  & Macleod,  1 994,  p.l35).  Bevan  and  Macleod  explain  that  this  definition  characterizes 
usability  as  a "quality  of  use  that  can  be  measured  as  the  outcome  of  interaction  in  a context"  (p.  1 35).  What  they 
mean  by  this  is  that  the  overall  system  context  is  composed  of  users,  their  goals  (or  tasks),  and  the  physical  and 
organizational  environment,  as  well  as  the  system  itself.  The  quality  of  use  is  indicated  by  the  degree  of 
effectiveness,  efficiency,  and  satisfaction  as  experienced  in  the  result  of  the  interaction  among  the  four  system 
components.  The  first  two  usability  indicators,  effectiveness  and  efficiency,  are  usually  assessed  by  collecting 
behavioral  performance  measures  in  regard  to  leamability,  efficiency,  productivity,  memorability,  and  number  of 
errors  (Nielsen,  1993).  Satisfaction,  on  the  other  hand,  is  usually  assessed  by  examining  users’  perceptions  about 
the  system. 

Perceptions  affect  every  aspect  of  the  system,  including  usability.  It  is  not  rare  to  hear  from  users  that  they 
think  a certain  system  is  usable  when  they  actually  failed  to  perform  tasks  using  the  system.  The  reverse  is  often 
heard,  too.  This  discrepancy  between  actual  performance  and  perceptions  has  not  been  often  researched  in  the  field 
of  usability  evaluation.  Traditionally,  usability  studies  tended  to  collect  only  performance  measures  and  determine  a 
system’s  overall  usability  based  on  them.  Even  worse,  they  tended  to  disregard  users’  perceptions  as  invalid  usability 
data. 

Studies  such  as  those  by  Bailey  (1995),  Tractinsky  (1997),  and  Morris  and  Dillon  (1997),  however,  began 
to  address  the  discrepancy  between  performance  and  perceptions  and  to  emphasize  the  importance  of  assessing 
perceptions.  Since  this  kind  of  study  is  relatively  new  to  the  usability  field,  there  is  no  standardized  term  to  refer  to 
the  concept  of  perceptions.  Words  such  as  “preference”  (Bailey  1 995),  “apparent  usability”  (Tractinsky,  1 997),  and 
“impression”  (Morris  & Dillon,  1997)  are  used  interchangeably  with  “perceptions.”  According  to  Tractinsky 
(1997),  people  formulate  preferences  for  one  system  over  the  other  on  the  basis  of  their  vague  beliefs  about  which 
interface  would  provide  the  fastest  performance  or  apparent  usability  that  they  perceive  from  the  aesthetics  of  the 
system.  The  preferred  system,  however,  does  not  always  result  in  better  user  performance  or  actual  usability.  In 
their  study  on  Netscape,  Morris  and  Dillon  (1997)  conclude  that  users'  initial  perceptions  of  Netscape's  usefulness 
and  ease  of  use  significantly  influence  their  attitude  toward  using  Netscape  as  well  as  their  intention  to  use  it.  The 
implication  of  these  studies  is  that  we  need  to  treat  performance  and  perceptions  separately.  They  also  tell  us  that 
we  should  examine  users’  perceptions  of  the  system’s  usability  as  well  as  their  behavioral  performance  while  using 
the  system,  to  measure  overall  usability  accurately. 

Evaluation  Methodology 
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Perception -Based  Usability  Evaluation 


In  this  study,  we  used  a perception-based  usability  evaluation  method.  An  EPSS  like  STEPS  was  a 
relatively  new  idea  to  the  education  community,  so  we  were  interested  in  investigating  whether  target  users  of 
STEPS  liked  the  system  and  perceived  it  to  be  usable.  In  addition,  STEPS  was  still  under  construction  at  the  time  of 
the  evaluation  so  collecting  performance  data  at  that  stage  did  not  seem  to  be  meaningful. 

This  evaluation  elicited  participants’  perceptions  of  three  aspects  of  the  STEPS  web  site:  usability  of  the  six 
main  menus,  existence  of  necessary  EPSS  components,  and  effectiveness  of  nine  main  menu  icons.  The  STEPS 
web  site  consisted  of  nine  main  menus  and  six  of  them  were  primary  components  of  the  site.  The  six  menus 
included  Main  Menu,  Lesson  Architect,  Best  Practice,  Sample  Unit,  Web  Links,  and  Tutorial  Library.  The  matter  of 
our  primary  concern  was  to  evaluate  whether  the  participants  perceived  the  six  menus  to  be  effective,  efficient,  and 
satisfactory  (i.e.,  usable.)  In  addition,  we  were  interested  in  assessing  whether  the  participants  perceived  that  STEPS 
contained  all  the  necessary  features  of  an  EPSS.  As  described  in  the  literature  review  section,  an  EPSS  typically 
contains  four  basic  components:  tools,  database,  expert  system,  and  instructions.  Lastly,  we  wanted  to  evaluate  the 
effectiveness  of  icons  associated  with  the  nine  main  menus.  On  the  main  page,  each  main  menu  was  represented 
through  the  combination  of  an  icon  and  a textual  link  underneath.  Our  concern  regarded  whether  or  not  the  icons 
matched  the  textual  links,  representing  the  content  of  their  corresponding  pages  well. 

Data  Collection  Methods 

Evaluation  instruments  included  a usability  questionnaire,  a components  questionnaire,  a matching 
worksheet,  observations,  and  structured  interviews.  The  usability  questionnaire  (See  Appendix  A)  aimed  to  measure 
the  participants’  perceptions  of  the  usability  of  the  six  main  menus.  It  consisted  of  yes  -no  answer  items  such  as, 

"Are  the  page  contents  useful  for  intended  users?"  "Are  the  navigation  icons  or  texts  consistent?"  and  "Do  you  like 
using  this  system?"  We  used  one  questionnaire  for  each  of  the  six  main  menus.  The  components  questionnaire 
aimed  to  measure  participants'  perceptions  of  whether  the  STEPS  website  contained  all  the  necessary  EPSS 
components  or  not  (See  Appendix  B).  It  consisted  of  questions  like,  "Does  this  EPSS  have  tools  (e.g.,  templates, 
forms,  word  processor,  spreadsheets)  for  facilitating  teacher  performance?"  The  matching  worksheet  (See  Appendix 
C)  assessed  whether  participants  could  match  the  main  menu  graphics  with  textual  links  correctly.  We  provided 
participants  with  a list  of  icons  and  texts  and  asked  them  to  match  corresponding  icon  and  text.  While  the 
participants  were  exploring  the  site  and  responding  to  the  questionnaires  and  the  matching  worksheet,  observations 
were  made  to  collect  participants’  reactions  such  as  expressions  of  their  frustration.  The  structured  interviews  (See 
Appendix  D)  at  the  end  of  the  evaluation  aimed  to  triangulate  data  from  the  questionnaires  and  observations. 

To  ensure  the  trustworthiness  of  the  evaluation,  we  employed  triangulation  and  member-checking 
procedures.  This  evaluation  used  three  kinds  of  triangulation  techniques:  (1)  data  triangulation  through  the  use  of 
multiple  data  sources  (e.g.,  questionnaires,  observations,  and  interviews),  (2)  participant  triangulation  by  asking 
individuals  with  diverse  backgrounds  to  evaluate  the  website  (e.g.,  elementary  school  teachers,  instructional 
designers,  and  professors),  and  (3)  method  triangulation  through  the  use  of  various  data  collection  methods  (e.g., 
questionnaires,  observations,  matching  worksheets,  and  interviews.  After  analyzing  observation  and  interview  data, 
we  summarized  the  participants’  opinions  and  then  asked  the  participants  to  review  the  summaries.  The  aim  was  to 
ensure  that  the  data  analysis  results  matched  their  original  opinion. 

Participants 

Five  participants  engaged  in  the  evaluation.  Virzi  (1992)  conducted  experiments  regarding  sample  size  for 
usability  studies  and  concluded  that  observing  four  or  five  participants  will  allow  a practitioner  to  discover  80%  of  a 
product's  usability  problems  and  observing  additional  participants  will  reveal  increasingly  fewer  new  usability 
problems.  In  addition,  it  is  well  known  in  the  qualitative  case  study  literature  that  there  is  a certain  point  after  which 
discovery  of  new  findings  reaches  saturation. 

The  target  users  of  the  STEPS  website  were  K- 1 2 teachers.  Therefore,  we  invited  people  who  had  teaching 
backgrounds  to  participate;  three  of  them  have  taught  in  K-12  settings  and  two  have  worked  as  teacher  educators. 

We  also  invited  people  who  had  expertise  in  design  and  development  of  computer-assisted  instructional  systems,  in 
addition  to  teaching  experience.  The  purpose  was  to  gather  expert  opinion  on  the  design  of  the  website.  Appendix 
E summarizes  profiles  of  the  participants. 

Evaluation  Procedures 

The  evaluation  took  place  in  the  most  typical  computing  environment  of  each  participant.  Some 
participants  did  computing  at  home  and  others  worked  at  the  computer  labs  provided  by  their  university.  All 
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participants  used  LAN  Internet  connection  provided  by  the  university.  They  used  an  IBM  compatible  PC  or  a 
notebook  that  had  24  RAM  or  more  memory,  Pentium  133  or  faster  processor,  and  12-  or  14-inch  monitors. 

At  the  beginning  of  the  evaluation,  the  participants  filled  out  a Demographic  Information  questionnaire 
(See  Appendix  F.)  Then  they  were  given  instructions  about  how  to  conduct  the  evaluation.  They  were  instructed  to 
e>q>ress  any  criticism  frankly,  use  a think-aloud  technique,  and  feel  free  to  ask  any  questions  about  the  evaluation 
procedures.  It  was  particularly  emphasized  that  if  they  were  having  difficulties  in  using  the  website  it  was  not  their 
fault  but  the  fault  of  the  website’s  ineffective  design.  When  the  participants  felt  comfortable  about  beginning  the 
evaluation,  we  opened  up  the  home  page  of  the  STEPS  website  and  introduced  the  site's  general  purpose.  Then,  the 
participants  were  given  six  usability  questionnaires,  a components  questionnaire,  and  a matching  worksheet.  They 
were  requested  to  complete  the  questionnaires  and  the  matching  worksheet  while  exploring  the  site.  Different 
participants  looked  at  different  levels  of  instruction  since  the  entire  STEPS  website  was  too  big  for  one  person  to 
explore  in  a given  period  of  time.  However,  the  evaluation  criteria  and  scope  were  the  same  for  all  participants. 
Participant  A explored  the  K3-5  level,  participant  B the  K6-8,  participant  C the  K9-12,  participant  D the  K3-5,  and 
participant  E the  K3-5.  The  participants  thought  out  loud  while  exploring  the  site  and  we  took  notes  of  their 
comments.  We  sometimes  asked  probing  questions  if  they  did  not  verbalize  problems  voluntarily.  When  they 
finished  all  the  questionnaires  and  the  worksheet,  a structured  interview  was  conducted. 

Evaluation  Results 

Results  were  derived  from  three  evaluations:  1)  evaluation  of  the  six  main  menus’  usability  using 
questionnaires,  observations,  and  interviews;  2)  evaluation  of  the  EPSS  conponents  using  a questionnaire;  and  3) 
evaluation  of  the  match  between  icons  and  texts  using  a matching  worksheet. 

Usability  of  six  main  menus 

The  usability  questionnaires  asked  for  participants’  perception  regarding  each  menu’s  effectiveness, 
efficiency,  and  appeal.  Table  1 summarizes  percentages  of  positive  (i.e.,  yes)  responses. 


Table  1.  Percentage  of  positive  responses  in  six  usability  questionnaires 


Best 

Sample 

Web 

Tutorial 

Lesson 

Main 

Practice 

Unit 

Links 

Library 

Architect 

Menu 

Effectiveness 

Are  the  page  contents  useful  for  intended 

100 

100 

80 

60 

20 

0 

users? 

Do  you  believe  this  site  can  facilitate  teacher 

100 

80 

60 

60 

40 

0 

performance? 

Mean 

100 

90 

70 

60 

30 

0 

Efficiency 

Are  the  navigation  icons  or  texts  consistent? 

80 

60 

0 

0 

20 

0 

Is  it  easy  to  navigate  back  and  forth? 

80 

80 

0 

100 

0 

0 

Is  the  screen  design  user-friendly?  (e.g.. 

80 

60 

20 

20 

20 

0 

letter  size,  color,  graphics,  etc.) 
Is  the  information  concise? 

80 

40 

40 

20 

20 

20 

Are  the  sentences  easy  to  understand? 

80 

100 

100 

100 

100 

20 

Mean 

80 

60 

35 

48 

32 

8 

Appeal 

Are  you  interested  in  this  EPSS? 

100 

60 

60 

60 

20 

0 

Do  you  like  using  this  system? 

100 

60 

60 

60 

40 

0 

Mean 

100 

60 

60 

60 

30 

0 

In  the  effectiveness  evaluation,  participants  responded  most  positively  to  Best  Practice.  Sample  Unit  was 
second  at  90%,  followed  by  Web  Links,  Tutorial  Library,  Lesson  Architect,  and  Main  Menu.  In  the  efficiency  test. 
Best  Practice  was  rated  to  be  the  most  positive  followed  by  Sample  Unit,  Tutorial  Library,  Web  Links,  and  Lesson 
Architect,  with  Main  Menu  receiving  the  least  positive  evaluation.  In  the  appeal  evaluation.  Best  Practice  received 
the  most  positive  responses,  followed  by  Sample  Unit,  Web  Links,  Tutorial,  Lesson  Architect,  and  Main  Menu. 
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Overall,  the  participants  responded  to  the  three  criteria  in  a similar  pattern.  They  favored  Best  Practice  the  most  and 
Main  Menu  the  least  for  all  three  criteria. 

The  results  from  the  interviews  were  congruent  with  those  of  the  questionnaires.  Most  participants  made 
positive  comments  on  Best  Practice,  including  “menus  organized  by  subject  makes  it  easy  to  grasp  the  content,”  “it 
provides  examples  of  specific  objectives,”  “table  structure  of  the  menu  makes  it  easy  to  navigate,”  “color  change 
gives  the  indications  of  where  I am,”  and  “it  provides  concise  information  with  graphics  and  animations.”  For 
Sample  Unit,  participants  made  both  positive  and  negative  comments.  On  the  other  hand,  participants  responded 
more  negatively  to  Main  Menu  and  Lesson  Architect.  For  Main  Menu,  in  particular,  five  participants  expressed 
similar  concerns  such  as  “the  location  of  buttons  is  inconsistent  so  navigation  is  not  easy,”  and  “after  clicking,  there 
is  no  consistency  in  screen  display.”  For  Lesson  Architect,  participants  responded  negatively,  making  comments 
such  as,  “it’s  hard  to  know  the  function  of  icon-only  buttons,”  “there  is  no  direction  after  clicking,”  “too  much 
scrolling  to  do,”  and  “data  are  not  organized  so  it’s  not  easy  to  find  needed  information.” 

EPSS  components 

Most  participants  identified  STEPS  as  a system  that  contained  database  and  tool  components  more  than 
instruction  or  expert  system  components.  All  participants  agreed  that  STEPS  contained  database,  80%  agreed  it 
contained  tools,  40%  instruction,  and  20%  expert  system.  They  pointed  out  that  Lesson  Architect  was  a template 
embedded  with  word  processing  functions.  They  commented  that,  using  Lesson  Architect,  a teacher  could  develop 
and  save  instructional  designs  and  curriculum  plans  with  the  help  of  instructional  theories  such  as  Gauge’s  Event  of 
Instruction.  Web  Links  included  links  to  numerous  web  sites.  Best  Practice  was  a database  of  effective  classroom 
activities. 

Although  participants  did  not  recognize  it  clearly,  STEPS  did  include  expert  system  and  instruction 
components.  Ask  Coach  and  Tutorial  Library  were  representations  of  the  components.  Ask  Coach  provided  field- 
sensitive  help  in  a pop-up  window;  however,  most  participants  stated  that  the  help  was  not  informative  or  useful  for 
actual  instructional  activities.  The  fact  that  Ask  Coach  was  still  under  construction  might  have  been  the  cause  of  this 
response.  Also,  most  users  thought  that  it  is  very  limited  in  that  it  only  provides  fixed  explanations  and  cannot  make 
a relational  query  within  the  Coach  or  support  customized  help  depending  on  users’  level  of  competence.  As  for  the 
Tutorial  Library,  participants  responded  that  the  information  was  not  useful  for  specific  design  activities. 

Match  between  icon  and  text 

None  of  the  participants  matched  more  than  four  of  the  nine  pairs.  Evaluator  A matched  3 pairs,  evaluator 
B matched  2,  Evaluator  C,  1,  Evaluator  D,  4,  and  Evaluator  E,  3.  The  difficulties  in  matching  icons  with  texts  were 
caused  mainly  by  the  inadequacy  of  icons  in  portraying  corresponding  textual  information.  In  addition,  the  way  line 
spacing  was  used  made  it  difficult  for  participants  to  know  which  icon  corresponded  with  which  text. 

Overall  comments 

The  participants  agreed  unanimously  that  an  EPSS  like  STEPS  was  a powerful  tool  for  effective 
instruction.  However,  they  said  that  STEPS  should  go  beyond  being  an  electronic  book.  They  commented  that  it 
should  facilitate  interaction  between  teachers  and  provide  guidelines,  not  just  information.  They  thought  teachers 
would  be  able  to  enhance  their  performance  with  the  help  of  such  interaction  and  guidelines. 

Recommendations  for  Designing  an  &EPSS  Effectively 

Based  upon  our  evaluation  results  and  literature  reviews  on  E-EPSS  design,  we  recommend  the  following 
heuristics  for  effective  E-EPSS  design. 

Design  an  interface  that  supports  teacher  performance 

Participants  did  not  think  that  Main  Menu  and  Lesson  Architect  facilitated  teacher  performance.  This 
response  is  critical  in  the  case  of  Lesson  Architect  especially,  because  Lesson  Architect  is  supposed  to  facilitate 
teacher  performance  by  helping  teachers  to  build  lesson  plans.  Laffey  (1995)  asserts  that  current  models  of  EPSSs 
provide  the  kinds  of  resources  needed  but  do  not  support  the  processes  by  which  these  resources  are  used  or 
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customized  for  the  work  environment.  Instead  of  seeing  an  EPSS  as  merely  a vehicle  for  delivering  information,  we 
need  to  see  it  as  a re  -conceptualization  of  the  work  environment. 

Provide  context  specific  information 

Participants  thought  STEPS  lacked  expert  system  functions  and  particularly  context  - or  subject-specific 
guidelines.  Evaluator  B,  for  example,  responded,  “STEPS  looks  like  a teacher  guidebook  but  its  information  is  too 
general,  not  subject-specific.”  She  recommended  that  the  Language  Arts  section  should  contain  more  specific 
information  to  be  useful  in  actual  instructional  design  activities. 

The  process  of  determining  the  content  of  an  E-EPSS  should  involve  a thorough  examination  of  related 
literature  and  consultation  with  subject  matter  experts,  teachers,  and  teacher  educators.  It  should  also  include 
identification  of  teachers’  daily  activities  such  as  lesson  planning  and  instruction  design.  Once  the  activities  are 
identified,  the  designer  needs  to  specify  possible  questions  teachers  could  ask  at  each  stage  of  the  activity,  and 
provide  necessary  supports  that  will  answer  the  questions.  This  kind  of  support  must  be  updated  and  expanded  when 
necessary. 

Provide  structured  navigational  schemes 

The  evaluation  results  show  that  people  like  structured  navigational  scheme  such  as  menus  organized  by 
subject  in  a table  format.  To  provide  a structured  navigational  scheme,  it  is  important  to  build  a “bird’s-eye  view” 
into  the  design.  Users  should  be  informed  about  where  they  are  now  and  where  they  should  go  next . In  STEPS,  the 
hierarchy  and  relationships  among  different  levels  (corporation  level,  unit  level,  and  lesson  level)  are  not  indicated 
clearly,  resulting  in  confusion  in  navigation. 

The  Object  Action  Interface  (OAI)  model  by  Shneiderman  (1998)  is  useful  for  designing'a  structured 
interface.  It  guides  system  designers  to  view  interfaces  in  terms  of  the  tasks  that  the  interfaces  will  carry  out.  By 
matching  interfaces  with  tasks,  designers  can  produce  task-oriented  and  structured  interface  designs. 

Consider  the  level  of  students  the  users  teach 

Participants  mentioned  that  content  in  STEPS  should  consider  grade  levels.  If  the  content  is  for  elementary 
teachers,  for  example,  it  should  reflect  perspectives  and  difficulty  levels  appropriate  for  elementary  students.  In  this 
way,  the  teachers  will  be  able  more  easily  to  relate  what  they  get  through  STEPS  to  their  students’  learning 
activities.  Rummers  (1998)  contends  that  the  structure  of  an  EPSS  should  be  easy  to  use,  flexible,  and  tailored  for 
end-users  with  different  needs  and  expertise. 

Provide  interactive  screen  designs 

Participants  made  comments  such  as,  “it’s  more  like  a technical  report,”  “texts  are  not  easy  to  read,”  “the 
content  seems  like  texts  from  a book,”  and  “I  need  to  do  too  much  scrolling  and  get  lost  often.”  These  comments 
tell  us  that  STEPS  could  be  perceived  as  an  electronic  page-turner.  To  prevent  this  perception,  STEPS  needs  to 
incorporate  action-oriented  design  approaches  such  as  anchoring  tools  in  the  task  domain  and  supporting  error 
recognition  and  recovery,  and  user  performance. 

Limitation  of  study 

Although  we  referred  to  a published  article  by  the  STEPS  developers,  Northrup  and  Pilcher  (1998),  we  did 
not  have  access  to  enough  of  the  developers’  opinions  regarding  their  intentions  about  STEPS  development  and  the 
purpose  of  the  program.  In  addition,  we  did  not  have  any  information  about  their  development  processes  such  as 
difficulties  in  the  processes  and  timelines  of  their  development  and  implementation. 

The  evaluation  materials  that  we  developed  for  this  study  are  the  results  of  our  research  on  usability  and 
educational  evaluation  studies.  Although  we  hope  the  materials  helped  us  measure  participants’  perceptions 
accurately,  we  think  the  materials  need  to  be  subjected  to  quality  examination  such  as  reliability  and  validity  tests. 

It  should  be  noted  that  the  questionnaires  measured  participants’  subjective  perceptions  of  the  program. 
Objective  performance  measures  such  as  time  taken  to  finish  certain  tasks  or  error  rate  were  not  collected  in  this 
study.  A future  evaluation  study  that  collects  performance  data  will  provide  us  with  a richer  picture  of  the 
program’s  usability  and  educational  value. 
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Integrating  Educational  Technology  into  Field  Experiences  and  Teacher  Education 

Curriculum-A  Systemic  Approach 


Abstract 


Jyh-Mei  Liu 
Ashland  University 


Field  experience  is  a crucial  stage  for  prospective  teachers.  However,  evidence  shows  that  there  are  only 
twenty  percent  of  teachers  in  the  nation  feel  well  prepared  to  integrate  educational  technology  into 
classroom  instruction  (1999,  Year  2 StaR  Report).  As  a result,  the  prospective  teachers  may  not  observe 
classroom  teachers  integrating  educational  technology  during  field  experiences.  This  presents  a severe 
deficiency  in  the  teacher  preparation  process.  This  project  is  about  systemically  integrating  educational 
technology  into  field  experience  and  the  Teacher  Preparation  curriculum.  Through  consultation  and 
training,  cooperating  teachers  work  with  prospective  teachers  to  implement  educational  technology  into 
field  experiences,  while  faculty  members  in  the  Teacher  Preparation  Program  and  the  Arts  and  Sciences 
Departments  set  role  models  in  the  methods  courses  and  regular  curriculum.  Systemic  growth  and 
collaboration  will  sustain  in  the  participating  schools  and  the  teacher  education  institution. 

Need  for  the  Project 

Field  experience  and  student  teaching  are  two  crucial  stages  for  prospective  teachers  before  they  become 
regular  classroom  teachers.  They  take  methods  courses  in  the  teacher  education  program  that  prepare  them  in 
concepts  regarding  how  a classroom  is  constructed  and  how  learning  and  teaching  are  developed  in  the  classroom 
context.  It  is  not  until  the  field  experience  that  the  prospective  teachers  often  realize  the  real  work  of  teaching  from  a 
teacher’s  point  of  view.  They  observe  and  depend  on  the  cooperating  teachers  to  provide  the  models  and  to 
demonstrate  how  learning  and  teaching  can  take  place  effectively  in  the  classroom.  Based  on  CEO  Forum  Reports 
(1999,  Year  2 StaR  Report),  only  20  percent  of  teachers  in  the  nation  feel  well  prepared  to  integrate  educational 
technology  into  classroom  instruction.  This  means  that,  when  prospective  teachers  go  into  the  field,  the  chances  are 
that  they  may  not  observe  classroom  teachers  integrating  educational  technology,  depending  on  whether  or  not  their 
cooperating  teachers  are  among  that  20  percent  of  teachers.  Given  the  national  awareness  of  how  technology  should 
be  integrated  as  a powerful  learning  and  instructional  tool  in  the  classroom  (NCATE,  ISTE  Standards),  this  statistic 
reminds  us  that  there  is  a severe  deficiency  in  teacher  preparation  programs.  No  matter  what  kind  of  technology 
training  and  method  courses  we  equip  our  prospective  teachers  with  in  the  teacher  preparation  curriculum,  they  may 
or  may  not  be  able  to  see  how  educational  technology  is  being  used  in  real  classrooms  during  field  experience. 

The  Teacher  Education  Program  at  a reputable  private  university  located  in  north  central  Ohio  requires 
perspective  teacher  to  take  at  least  two  courses  in  educational  technology.  The  course  content  includes  skills  in  using 
different  traditional  and  non-traditional  technology,  theories  about  why  and  how  educational  technology  should  be 
integrated  to  enhance  learning  and  instruction,  and  examples  of  integration.  However,  if  the  field  experience  does 
not  ensure  the  perspective  teacher  an  appropriate  context,  the  training  and  knowledge  the  perspective  teacher 
received  from  the  program  cannot  be  validated  and  reinforced  in  real  classrooms. 

The  Teacher  Education  Program  and  the  College  of  Education  at  this  university  are  among  the  largest  in  the 
state  of  Ohio.  The  university  sends  out  over  500  students  for  field  experiences  every  semester  and  works  with  23 
school  districts  (103  school  buildings)  including  over  350  cooperating  teachers.  Based  on  the  survey  delivered  to 
these  cooperating  teachers  who  work  with  this  university,  there  are  only  25  percent  of  the  teachers  who  feel 
confident  in  integrating  technology  in  the  regular  curriculum.  With  these  figures  in  mind,  we  wonder  how  many  of 
our  children  in  the  classrooms  missed  out  on  the  opportunities  to  use  technology  as  a powerful  learning  tool  and  are 
being  under-prepared  for  the  digital  era. 

Another  area  that  presents  a problem  is  the  low  percentage  of  educational  technology  integration  in  courses 
that  prospective  teachers  take  from  the  Arts  and  Science  Department  (30%)  and  the  Teacher  Education  Program 
(35%)  at  this  university.  Not  all  faculty  members  who  teach  these  courses  have  the  knowledge  and  skills  to  model 
the  integration  of  technology  into  curriculum  as  inquiry  and  instructional  tools.  As  a result,  the  prospective  teachers 
face  double  jeopardy.  Not  only  can’t  they  see  much  modeling  from  their  cooperating  teachers  during  field 
experiences,  neither  do  they  see  much  modeling  within  their  teacher  preparation  curriculum  with  the  exception  of 
the  two  educational  technology  courses. 
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To  correct  the  deficiency  of  the  teacher  preparation  process,  a systemic  approach  is  in  order.  This  approach 
entails  a partnership  among  the  stakeholders  including  the  Teacher  Education  Program,  institutions  that  host  the 
program,  faculty  members  in  the  Teacher  Education  Program  and  the  Arts  and  Science  Department,  public  schools 
administrators,  classroom  teachers,  prospective  teachers,  and  community  members.  With  the  understanding  that  the 
partners  involved  in  this  project  will  have  far  more  impact  than  the  initial  scope  of  this  project,  the  next  section  will 
explain  the  design  by  which  this  partnership  improves  and  eventually  resolves  the  aforementioned  problems. 

Design  of  the  Project 

A.  Goal  and  Expected  Outcomes 


The  goal  of  this  project  is  to  svstemicallv  improve  the  field  experiences  of  the  Teacher  Education  Program 
and  the  degree  to  which  cooperating  public  schools  and  the  university  faculty  members  integrate  educational 
technology  (ET)  into  regular  curriculum.  The  expected  outcomes  include  but  are  not  limited  to  the  following: 

• Increased  degree  of  educational  technology  integration  in  field  experiences  for  the  prospective 
teachers  at  the  university. 

• Increased  confidence  of  prospective  teachers  in  integrating  educational  technology  in  the 
classroom  settings. 

• Increased  skills  and  knowledge  of  prospective  teachers  in  integrating  educational  technology  in 
the  classroom  settings. 

• Increased  degree  of  educational  technology  integration  in  participating  public  schools. 

• Increased  degree  of  educational  technology  integration  in  the  Teacher  Education  Program  and  Arts 
and  Science  Department  in  the  university. 

• Increased  confidence  levels  for  participating  professors  in  the  use  and  integration  of  educational 
technology  in  the  content  courses. 

• Increased  confidence  levels  for  participating  cooperating  teachers  to  sustain  the  interest  and  the 
skill  of  educational  technology  integration. 

• Increased  collaboration  among  teachers  and  prospective  teachers  in  educational  technology 
integration. 

• A consortium  Web  site  will  be  built  for  this  project  so  that  all  participants  can  use  the  collective 
information  (i.e.,  lesson  plans,  integration  ideas,  published  student  works,  etc)  and  collaborate 
their  efforts.  This  Web  site  will  be  designed  in  ways  that  can  accommodate  people  with 
disabilities. 

• A documentary  videotape  will  be  made  to  share  the  successful  experiences  and  preliminary  model 
with  other  universities  and  k- 1 2 schools. 

B.  Description  of  the  Project  Design 

This  partnership  will  be  developed  in  the  following  four  stages -preparation:  commitment  of  partners; 
assessment  and  training;  adaptation;  summative  evaluation  and  diffusion.  Formative  evaluations  will  be  conducted 
in  each  stage  to  evaluate  the  effectiveness  of  the  process.  A consortium  Web  site  will  be  built  to  contain  the 
information  of  this  project.  A videotape  will  be  produced  documenting  the  entire  process  to  be  used  in  the  diffusion 
stage  of  the  project. 

Stage  I:  Preparation-Commitment  of  Partners 
Time  Frame:  Pre-Project  Time 

In  the  first  stage  of  the  partnership,  each  partner  will  need  to  commit  to  the  shared  vision  and  provide  the 
necessary  resource  (i.e.,  service,  time,  money,  equipment,  etc)  to  facilitate  this  partnership.  The  shared  vision  is  to 
systemically  improve  the  quality  of  field  experience  of  the  Teacher  Education  Program,  the  quality  of  teacher 
preparation  curriculum,  and  the  competency  of  classroom  teachers  in  the  area  of  educational  technology  integration, 
in  turn  to  produce  competent  teachers  and  richer  educational  experience  in  the  classrooms.  The  partners  in  this 
project  include  the  Teacher  Education  Program  at  the  university;  the  Educational  Technology  Program  in  the 
university;  6 school  districts;  faculty  members  in  the  Teacher  Education  program;  faculty  members  in  the 
Educational  Technology  program;  faculty  members  in  the  Arts  and  Science  Department  who  teach  methods  courses 
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to  prospective  teachers;  cooperating  classroom  teachers  from  6 school  districts;  and  prospective  teachers  who  are 
enrolled  in  the  teacher  education  program  at  the  university. 

Examples  of  commitment  include  but  are  not  limited  to  the  following: 

1.  Participating  school  districts  will  raise  3%  of  the  budget  in  the  area  of  educational  technology. 

2.  Participating  school  districts  have  to  continue  to  support  the  existing  in-service  training  in  the  area 
of  technology  integration. 

3.  Public  schools  administrators  will  commit  to  obtaining  video  conferencing  facilities  (i.e.,  cable 
TV,  Polycom  unit,  or  Satellite  dish)  for  cooperating  teachers  to  receive  consultation  at  their  school 
sites. 

4.  Educational  technology  faculty  members  in  the  Teacher  Education  Program  will  be  given  released 
time  and  incentives  to  offer  training  to  faculty  members  in  the  Teacher  Education  Program  and 
methods  teachers  in  Arts  and  Science  Department. 

5.  The  university  will  have  a video  conferencing  facility  and  video  capturing  and  editing  equipment 
for  the  Teacher  Education  Program  since  these  are  the  areas  that  have  not  been  covered  in  the 
curriculum  due  to  the  lack  of  facilities.  In  addition  to  necessary  on-site  consultation,  educational 
technology  faculty  members  and  selected  professionals  will  need  this  facility  to  deliver 
consultation  to  classroom  teachers  in  different  school  districts. 

6.  Faculty  members  in  the  Teacher  Education  Program  and  faculty  members  in  the  Arts  and  Science 
Department  who  teach  methods  courses  will  commit  to  receive  training  or  consultation  in  the  area 
of  educational  technology  integration. 

7.  Participating  school  teachers  who  are  skilled  in  the  area  of  educational  technology  are  given 
incentives  to  coach  peer  teachers  or  to  conduct  in-service  training 

8.  Although  not  as  official  members  in  this  consortium,  educational  technology  professionals  from 
the  Tri-County  Computer  Service  Association  (TCCSA)  and  Tri -River  Educational  Computing 
Association  (TRECA)  will  be  contracted  to  provide  consultation  to  the  cooperating  teachers  and 
prospective  teachers  either  on-site  or  via  video  conferencing  facilities  regarding  how  to  integrate 
educational  technology  during  the  field  experience  sessions. 

9.  Prospective  teachers  who  have  taken  educational  technology  courses  in  the  university  will  assist 
the  cooperating  teachers  in  needed  technology  skills  while  working  with  consultants  to  integrate 
educational  technology  into  field  experiences. 

10.  Classroom  teachers  who  receive  consultation  as  a result  of  this  partnership  will  provide 
consultations  to  their  peer  teachers  given  limited  incentives. 

Stage  11:  Assessment  and  Training 

Time  Frame:  First  Project  Year  (June,l,  2001-IMay  31,  2002) 

In  the  second  stage,  the  focus  of  this  project  is  to  identify  the  areas  where  the  faculty  members  in  the 
Teacher  Education  Program  and  the  Arts  and  Science  faculty  members  who  teach  methods  courses  lack  knowledge 
and  skills,  and  provide  training  that  is  needed.  The  areas  where  cooperating  classroom  teachers  need  consultation 
from  the  trainers  are  also  identified  during  this  stage. 

Based  on  the  national  standards  (ISTE),  the  contents  of  training  and  consultation  sessions  for  participating 
university  faculty  members,  cooperating  teachers,  and  prospective  teachers  would  include  but  not  be  limited  to  the 
following: 

• Basic  concepts  and  skills  of  using  computer  systems 

• Basic  concepts  and  skills  of  using  computer  related  peripherals,  i.e.,  scanner,  digital  camera, 
projectors,  etc) 

• Using  productivity  tool  (word  processor,  spreadsheet,  database)  effectively  as  teaching,  learning, 
and  management  tools. 

Examples 

• Proficient  use  of  word  processor  to  produce  document  for  specific  purpose,  such  as 
newsletter,  communication,  and  professional  documents. 

• Power  point  presentation-  Instructional  tool  (teacher),  learning  tool  (students) 

• Spreadsheet  application-inquiry  project,  grading  sheets,  etc. 

• Database-inquiry  project,  resource  management,  test  bank,  etc. 
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• Using  Hypermedia  tools  such  as  Hyper  Studio  and  KidPix  as  instructional  and  learning  tools. 
Examples:  tailored  tutorial,  drill  and  practice  for  specific  topic  & students;  student  created 
projects  for  specific  topics  under  study. 

• Using  Internet  applications  for  research,  communication,  instruction,  learning,  and 
management. 

Examples: 

• Advanced  email  communication  and  listserv 

• Locate  and  utilize  resource  database 

• Develop  Web  pages  that  include  instructional  element  (tutorial,  information),  learning 
element  (research  project,  collaborative  learning  projects),  management  element  (class  Web 
site,  syllabus,  and  communication  with  students  and  parents). 

• Be  able  to  search  useful  Web  sites  and  create  annotated  list  of  good  software  and  useful  sites 
to  share  with  colleagues. 

• Be  able  to  provide  description  of  criteria  for  selecting  and  evaluating  educational  software. 

• Be  able  to  provide  a written  guide  and  example  of  using  Web  sites  and  software  for  inquiry 
projects. 

• Using  existing  assistive  technology  features  for  inclusion  classrooms. 

• Using  video  conferencing  facilities  to  communicate  and  to  collaborate. 

A needs  assessment  will  be  conducted  to  identify  the  gap  between  the  desired  training  outcomes  and  the 
current  skill  and  knowledge  of  the  participating  university  faculty  members.  After  the  needs  are  identified,  the 
university  faculty  members  need  to  commit  to  signing  on  for  training  sessions  so  that  they  can  model  the  use  of 
technology  in  the  context  of  their  subject  matters. 

The  Educational  technology  faculty  members  in  the  university  and  selected  professionals  will  provide 
training  to  the  participating  faculty  members  either  on  sites  or  via  video  conferencing.  Similar  needs  assessment 
will  be  conducted  for  participating  cooperating  teachers.  With  the  pre-existing  in-service  training  that  individual 
participating  school  offers,  the  trainers  will  consult  and  scaffold  cooperating  teachers  and  prospective  teachers  to 
integrate  educational  technology  into  field  experience  sessions.  The  classroom  teachers  will  be  informed  of  the 
schedule  when  the  skill  sessions  will  be  provided  for  university  faculty  members.  The  cooperating  teachers  are 
welcomed  to  join  the  sessions  via  video  conferencing  facilities  or  on  site  where  the  facilities  permit. 

Knowing  the  cooperating  teachers  and  prospective  teachers  have  integrated  educational  technology  into 
field  experiences,  faculty  members  who  teach  educational  technology  courses  in  Teacher  Education  Program  will 
incorporate  the  field  experience  sessions  into  the  courses  using  video  conferencing  facilities.  With  permission  from 
the  administrators  and  cooperating  teachers,  the  field  experiences  sessions  can  also  be  videotaped  and  later  shown  as 
examples  for  prospective  teachers.  The  prospective  teacheis  who  have  not  engaged  in  field  experiences  will  have  the 
opportunity  to  see  what  is  done  in  the  classroom  context.  This  can  provide  an  authentic,  real  world,  and  motivating 
experience  for  the  prospective  teachers  to  learn  how  to  use  and  to  integrate  educational  technology  in  the  classroom 
settings. 

Surveys  will  be  conducted  to  examine  the  effectiveness  of  each  training  session  and  consultation  process. 
The  training  sessions  for  university  faculty  members  can  be  videotaped  for  review  by  participating  faculty  members 
and  for  evaluation  purposes.  All  participating  classroom  teachers,  prospective  teachers,  and  faculty  members  will  be 
requested  to  provide  reflection  papers  on  their  progress  in  integrating  educational  technology  with  provided 
guidelines  throughout  the  year.  Prospective  teachers  will  continue  to  apply  what  they  have  learned  and  observed 
from  field  experience  to  their  student  teaching.  They  can  even  apply  the  consultation  skills  observed  from  the 
trainers  to  consult  with  their  colleagues  in  the  future.  In  addition  to  the  reflection  papers,  participating  classroom 
teachers  also  have  to  provide  records  on  the  noticeable  change  in  students’  performance  and  motivation  for  learning 
or  any  noticeable  changes  in  other  aspects  of  learning.  Integration  lessons  and  curriculum  examples  are  collected. 

The  collected  examples  will  be  put  onto  the  consortium  Web  sites  so  that  all  the  participants  in  this  project  can 
collaborate  and  share  the  information  and  integration  ideas.  These  experiences  will  also  be  shared  and  demonstrated 
in  the  showcase  conference. 

Surveys  will  be  conducted  to  evaluate  specific  aspects  of  the  overall  progress.  The  progress  results  will  be 
tallied,  documented,  and  analyzed  by  professional  evaluation  analysts. 

Stage  III:  Adaptation 

Time  Frame:  Second  Year  (June,  1,  2002-May  31,  2003) 
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As  a result  of  training  sessions  during  the  first  year,  more  participating  university  faculty  members  are 
better  prepared  to  model  the  integration  of  educational  technology  in  different  subject  matters.  Some  of  the  faculty 
members  can  also  assume  leadership  roles  to  offer  training  for  their  colleagues. 

In  the  participating  schools  districts,  some  of  the  participating  classroom  teachers  start  to  serve  as 
consultants  to  their  peers.  Under  the  supervision  and  with  the  consultation  of  their  trainers,  these  cooperating 
teachers  can  even  conduct  in-service  training  to  their  peer  teachers.  This  process  is  to  make  these  cooperating 
teachers  feel  confident  in  integrating  educational  technology  into  the  classrooms  and  make  public  schools  a natural 
place  for  collaboration  and  professional  development. 

The  gap  between  prospective  teachers  and  classroom  teachers  starts  to  diminish  at  this  stage.  A 
collaborative  relationship  is  fully  developed  between  prospective  teachers  and  classroom  teachers.  Prospective 
teachers  who  go  on  to  student  teaching  become  proficient  peers  with  their  cooperating  teachers.  In  the  mean  time,  a 
new  group  of  prospective  teachers  will  work  with  the  trained  cooperating  teachers  and  will  have  better  field 
experiences  since  the  cooperative  teachers  demonstrate  the  effective  integration  of  educational  technology  into  the 
classrooms.  The  TCCSA  and  TRECA  professionals  and  educational  technology  faculty  members  will  continue  to 
consult  a new  group  of  cooperating  teachers  with  a new  group  of  prospective  teachers  to  integrate  technology  into 
field  experiences.  The  faculty  members  who  teach  educational  technology  courses  in  the  Teacher  Educational 
Program  can  incorporate  the  field  experience  session  via  video  conferencing  with  the  prospective  teachers  who  have 
not  yet  engaged  in  field  experiences.  The  cooperating  classroom  teachers  can  also  serve  as  guest  speakers  in  the 
courses  to  demonstrate  and  to  motivate  prospective  teachers  even  before  the  prospective  teachers  start  their  field 
experiences. 

Although  the  provision  of  training  is  still  needed,  half  of  the  responsibility  of  the  trainers  shifts  to 
consultation  and  scaffolding  rather  than  conducting  training  to  participating  university  faculty  members. 

Participating  public  schools  become  more  self  -sufficient  and  continue  to  grow  in  the  degrees  of  educational 
technology  integration.  More  and  more  teachers  are  able  to  contribute  to  the  collective  growth  of  their  schools  in  the 
area  of  technology  integration.  Less  external  training  and  consultation  resources  are  needed  for  participating  public 
schools. 

Additional  training  areas  are  explored  for  participating  university  faculty  members  during  this  stage.  The 
professional  trainers  and  educational  technology  faculty  members  will  provide  new  training  topics  depending  on  the 
new  developments  in  educational  technology  and  how  the  needs  evolve. 

Showcase  conferences  will  be  held  annually.  The  participating  university  faculty  members,  cooperating 
teachers,  and  prospective  teachers  will  share  their  experiences  and  progress  in  the  conference.  Evaluations  of 
training  sessions  are  conducted  whenever  training  is  offered.  Throughout  the  year,  all  participating  classroom 
teachers,  prospective  teachers  and  faculty  members  will  submit  progress  reports  and  reflection  papers  based  on 
provided  guidelines.  Educational  technology  integration  lessons  and  curriculum  examples  are  collected  and  put  onto 
the  consortium  Web  site.  Classroom  teachers  continue  to  document  the  changes  in  students’  performance  and 
motivation  for  learning.  Surveys  will  be  conducted  to  evaluate  specific  aspects  of  the  overall  progress  of  this  stage. 
The  progress  results  will  be  tallied,  documented,  and  analyzed  by  professional  evaluation  analysts. 

Stage  IV:  Summative  Evaluation  and  Diffusion 

Time  Frame:  The  third  year  (June,  1 2003-May  31,  2004) 

While  more  and  more  faculty  members  and  classroom  teachers  are  being  trained  and  become  trainers,  and 
prospective  teachers  continue  to  develop  collaborative  relationships  with  classroom  teachers,  the  last  stage  of  this 
project  focuses  on  evaluating  the  project  and  diffusing  the  successful  experiences  of  this  project.  The  collected 
progress  reports  and  reflection  papers,  classroom  teachers’  documentation  on  students’  performance  and  motivation 
to  learn,  the  evaluation  of  training  sessions,  the  overall  effectiveness  and  progress  during  each  stage,  and  grant 
resource  utilization  are  all  systematically  reviewed  and  tallied.  Reports  will  be  generated  to  illustrate  the  effects  of 
this  project.  The  Educational  technology  faculty  members  will  offer  a capstone  course  that  focuses  on  evaluating 
the  effectiveness  of  educational  technology  integration  and  will  invite  graduate  students  in  educational  technology  to 
develop  capstone  experiences  for  the  evaluation  phase  of  this  project. 

The  results  of  the  project  will  be  first  shared  with  all  partners  in  this  project.  As  a whole,  the  partners  will 
evaluate  the  effectiveness  and  efficiency  of  the  project,  the  long-term  benefits,  and  the  future  dissemination  goals. 

The  results  of  this  project  will  also  be  shared  in  major  teacher  educator  conferences  or  educational 
technology  oriented  conferences  such  as  ATE  (Association  for  Teacher  Educators),  AECT  (Association  for 
Educational  Communication  and  Technology),  and  NECC  (National  Educational  Computing  Conference).  With  the 
documentary  videotape  and  consortium  Web  sites  produced  in  this  project,  this  project  can  serve  as  a model  to  help 
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other  universities  and  public  schools  systemically  improve  teacher  preparation  programs  and  produce  more 
competent  classroom  teachers  in  the  area  of  educational  technology  integration. 

Conclusion 

Enhancing  the  quality  of  teacher  preparation  programs  in  the  area  of  educational  technology  integration  is 
by  no  mean  an  easy  task.  It  entails  systemic  approaches  to  adjust  the  conditions  of  all  the  stakeholders  to  make  the 
effort  fruitful.  The  design  of  this  project  is  one  of  the  many  endeavors  undertaken  by  many  educators  and 
practitioners.  The  author  believes  that  with  the  continuous  effort,  we  will  see  the  difference  in  the  quality  of  our 
future  teachers  who  ensure  our  children  to  be  well  prepared  for  the  21*^  Century. 
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Introduction 

Schools  face  significant  challenges  in  implementing  computing  technology  within  their  curriculum.  Federal 
funding  programs  such  as  e-Rate  are  helping  schools  surmount  the  first  obstacle  — initial  technology  purchases. 
According  to  a recent  article  in  Electronic  School,  the  average  school  district  now  has  800  computers,  or  one  for 
every  4.9  students  (Kongshem,  2001 ).  The  National  Center  for  Education  Statistics  reports  that  98%  of  public 
schools  are  now  connected  to  the  Internet  (Cattagni,  2001),  and  while  this  does  not  always  mean  individual 
classroom  connections,  it  is  nonetheless  one  of  several  strong  indicators  of  the  prevalence  on  technology  in  schools.. 
However,  still  remaining  as  formidable  obstacles  are  the  costs  of  maintaining  this  technology  and  providing  training 
for  its  effective  use.  As  s Electronic  School  article  points  out,  computers  are  no  longer  concentrated  in 

computer  labs.  Increasingly,  they  are  located  in  classrooms  where  technical  support  staff  may  have  difficulty 
providing  adequate  support.  Teachers  are  fast  developing  skills  and  strategies  for  integrating  these  classroom 
computers  into  the  curriculum.  As  technology  becomes  a more  powerful  tool  for  delivering  the  curriculum,  teachers 
are  relying  more  and  more  on  quick  and  reliable  repair  of  their  classroom  computers. 

In  the  business  world,  one  full-time  person  has  responsibility  for  maintaining  50-75  computers  (Consortium 
for  School  Networking,  1999).  In  contrast,  most  school  districts  tend  to  provide  one  support  person  for  every  500 
computers.  Typically  in  the  business  world  one  computer  is  used  by  a single  user.  But  in  the  classroom,  computers 
serve  the  needs,  and  suffer  the  abuses,  of  many  users.  It  is  no  wonder  that  sometimes  technology  support  in  the 
schools  falters. 

And  when  technology  support  falters,  the  integrity  of  a school  district’s  entire  technology  program  is  at 
risk.  Teachers  who  have  invested  time  to  develop  lesson  plans  using  technology,  especially  those  who  are  still 
newcomers,  are  less  likely  to  continue  to  invest  their  energies  if  they  cannot  count  on  their  computers  to  be  up  and 
running.  When  parents  ask  their  children  how  computers  are  used  in  the  classroom,  or  when  parents  visit  the 
classroom,  they  may  discern  little  or  no  technology  use.  These  lapses  may  diminish  the  community  support  that  is 
necessary  for  continuation  or  expansion  of  the  district’s  technology  program. 

The  purpose  of  this  article  is  to  describe  how  Student  Technology  Assistant  (STA)  programs  can  help 
schools,  in  particular  rural  school  districts,  solve  some  of  these  problems.  Small  rural  school  districts  are  less  likely 
than  large  urban  school  districts  to  be  able  to  purchase  and  implement  the  technology  management  solutions 
recommended  for  these  new  decentralized  networks  of  computers.  Instead,  rural  school  districts  like  Plymouth 
(Wisconsin)  School  District  are  turning  to  creative  programs  where  students  help  do  the  work:  “It's  not  a formal 
program,  but  we  recruit  kids  when  they  hang  around  and  when  we  know  they're  interested.  If  a kid  puts  in  a full 
class  worth  of  time,  we'll  pay  around  five  or  six  dollars  an  hour”  (Kongshem,  2001).  Sedgwick  (Kansas)  School 
District  for  several  years  has  selected  five  top  Computer  Tech  students  to  serve  as  Tech  Apprentices,  offering  the 
students  an  elective  credit  for  their  participation.  In  an  interview  with  HPR*TEC’s  KidSpeak  webzine,  one  Tech 
Apprentice  noted,  “If  they  just  come  in  and  hire  a bunch  of  people,  the  people  they  hire  don’t  go  to  school  here 
everyday.  They  don’t  know  the  teachers.  With  students  doing  it...  the  teachers  know  us”  (Brown,  2000).  While  the 
Plymouth  and  Sedgwick  programs  are  fairly  informal,  many  of  the  STA  programs  highlighted  here  are  quite  formal. 
All  of  them  work  by  providing  students  with  opportunities  to  gain  credit  for  technical  experience,  at  the  same  time 
providing  technical  and  sometimes  instructional  or  community-related  services  to  others.  In  exchange  for  providing 
these  experiences,  the  schools  receive  technical  support,  training  for  their  teachers,  and  good  communication  with 
and  support  from  the  community  for  their  efforts.  The  increased  visibility  and  “buy-in”  also  give  a boost  to  the 
adoption  and  diffusion  of  technology  throughout  the  school. 

One  danger  in  implementing  an  STA  program  is  that  the  educational  needs  of  the  students  may  become  lost 
in  the  shuffle  of  administering  the  program.  Before  turning  to  specific  programs,  we’ll  examine  this  danger  more 
closely.  Then  we’ll  define  a successful  STA  program,  discuss  first  steps  in  establishing  such  a program,  point  out  a 
few  operational  considerations,  and  finally  take  a look  at  six  successful  models. 
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Education  First 


At  home,  more  and  more  school-aged  children  are  gaining  access  to  computers  and  the  Internet.  By  August 
2000,  the  percentage  of  homes  with  computers  had  risen  to  more  than  50%.  Household  Internet  access  has  also 
increased  dramatically  (NTIA,  2000).  There  is  little  doubt  that  computer  access  is  becoming  less  of  a problem  while 
our  students  are  coming  into  the  classroom  with  more  technical  know-how  each  year. 

Some  headlines  have  called  attention  to  the  potential  exploitation  of  these  technology-savvy  students  as 
technology  workers  in  their  schools.  Jamieson  McKenzie,  editor  of  the  online  education  technology  journal  From 
Now  On,  draws  similarities  between  some  computer  helpdesk  tasks  and  mowing  the  school  lawn  or  washing 
cafeteria  dishes.  “It’s  exploiting  children,”  he  says,  “There  are  lots  of  jobs  that  need  to  be  done,  but  we  expect 
adults  to  do  them”  (Vail,  1 999). 

Similarly,  some  are  concerned  that  students  may  have  inappropriate  access  to  data.  A Los  Angeles  Times 
article  reported  that  in  1997  members  of  a local  California  School  Educators  Association  chapter  filed  a complaint 
with  the  Irvine  School  District  because  students  had  unsupervised  access  to  “the  most  sensitive  material  we  store: 
grades,  personnel  records,  attendance,  personal  e-mail”  (Huffstutter,  1998). 

Advocates  of  STA  programs  note  that  several  of  the  larger  STA  programs  include  carefully  designed 
curricula  that  address  ISTE’s  National  Educational  Technology  Standards.  Other  programs  incorporate  coursework 
that  leads  to  A+,  Cisco,  Microsoft,  or  Novell  certification.  Advocates  stress  that  students  should  not  be  penalized  for 
their  technical  knowledge.  In  the  words  of  Dennis  Harper,  the  director  of  Generation  YES,  “Schools  must  stop 
operating  like  factories,  and  start  to  work  like  modem  companies.  The  kids  are  the  ones  in  power  because  they  have 
the  knowledge.  And  if  you  can’t  trust  your  kids,  you’re  in  trouble”  (Huffstutter,  1 998).  Michael  Milone  of 
Technology  and  Learning  Magazine  praises  one  STA  program,  SWAT,  for  its  encouragement  of  students  as 
stakeholders:  “Students  who  participate  in  SWAT  gain  a sense  that  they  are  active  participants  in  the  education 
process  and  recognize  that  their  contributions  are  valued,  thus  they  develop  a stronger  sense  of  ownership  of  the 
process”  (http://www.iit.edu/^swat/overview.html). 

Perhaps  the  most  often  mentioned  advantage  of  STA  programs  is  that  they  give  students  marketable  skills 
and  real-world  problem-solving  experience.  In  the  January  2001  cover  story  of  Electronic  School,  “The  New 
Networkers:  The  Path  to  Hot  IT  Jobs  Begins  in  High  School,”  Kevin  Bushweller  notes  that  according  to  recent 
industry  projections  “about  1.6  million  new  IT  workers  will  be  needed  this  year,  but  hiring  managers  predict  that 
about  850,000  positions  probably  won’t  be  filled  by  appropriately  skilled  workers.” 

To  keep  your  program  aligned  with  your  school’s  educational  goals,  you  may  want  to  keep  in  mind  the 
advice  offered  by  Kathleen  Vail  in  her  article  “Kids  at  Work:  The  Pros  and  Cons  of  Using  Students  as  Technology 


• Make  sure  the  community  knows  what  you’re  doing. 

• Make  sure  the  program  has  an  educational  element. 

o Offer  class  or  internship  credits 

o Allow  students  to  work  only  during  scheduled  times 

o Balance  a technology  apprenticeship  with  apprenticeships  in  other  careers 

• Make  sure  the  students  are  supervised  and  mentored. 

• Keep  the  pressure  off 

• Keep  an  eye  on  security. 


What  are  STA  Programs? 

Students  have  helped  maintain  computers  since  computers  first  entered  schools  in  the  1970s.  However,  the 
start  of  formal  large-scale  STA  programs  can  be  traced  to  1989  when  Mike  Bookey,  a parent  of  a middle -school 
student  in  Issaquah  (Washington)  School  District,  agreed  to  help  his  daughter’s  school  with  a computer  problem. 
Issaquah  is  home  to  many  families  who  work  for  Microsoft  and  other  high-tech  companies.  Yet  the  schools  were, 
according  to  Bookey,  like  “a  tribe  cut  off  from  the  outside  world  for  so  long  that  they  didn’t  know  about  telephones, 
voice  mail,  computers,  e-mail,  television,  Internet  and  other  tools  of  the  information  age” 
(http://www.svi.org/connect96/Profiles/Issaquah.htm).  With  the  help  of  concerned  educators,  the  support  of 
taxpayers,  and  the  volunteer  time  of  many  high-tech  community  members,  Bookey  started  the  TIP  program 
(Technology  Information  Project).  At  Liberty  High  School  in  Issaquah,  a small  group  of  TIP  students  began  meeting 
after  school  to  learn  about  different  network  operating  systems  and  to  build  prototype  networks.  TIP  students  and 
other  volunteers  did  the  “heavy  work”  of  installing  and  troubleshooting  equipment  within  the  school  district.  A 
formal  curriculum  based  on  the  TIP  students’  experience  and  modeled  after  similar  corporate  programs  was  put  into 
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place  in  Issaquah  in  1992  (http://www.Issaquah.wednet.edu/district/technology.htm).  Issaquah’s  TIP  program  has 
since  been  replicated  in  other  Washington  districts,  and  many  of  the  later  STA  programs  drew  inspiration  from 
presentations  which  Issaquah  students  made  around  the  country  in  the  1990s  and  through  well-attended  “open 
houses.”  TIP  students  worked  as  consultants  for  the  State  of  Kentucky  Department  of  Education  in  planning 
Kentucky’s  first  STA  conference  in  February  1995. 

Issaquah,  the  community,  is  unlike  most  U.S.  communities  in  its  concentration  of  high-tech  expertise  and 
its  taxpayer  support  of  technology  initiatives.  In  most  of  the  programs  discussed  in  this  article,  leadership  for  STA 
programs  did  not  come  from  community  members  but  instead  from  educators,  and  in  some  cases  from  the  students 
themselves.  For  this  article  we’ve  selected  a range  of  programs,  each  with  its  own  areas  of  emphasis.  But  in  all  of 
the  cases  discussed  here,  student  technology  assistants  (ST  As)  are  K-12  students  who,  under  the  leadership  of  their 
school,  provide  technological  assistance  to  others.  Usually,  the  assistance  provided  by  such  students  is  focused  on 
instructional,  technical  or  community-related  activities.  Students  in  such  programs  learn  technology -related  skills 
while  providing  a service  to  others.  The  work  of  STAs  is  often  hailed  as  being  beneficial  for  all  involved. 

Starting  an  STA  Program 

Given  the  relatively  inexpensive  start-up  and  maintenance  costs  of  an  STA  program,  many  schools  are 
looking  to  start  their  own  program.  The  logical  question,  then,  is  how  to  proceed.  The  following  sections  detail  some 
ideas  for  starting  your  own  STA. 

Determining  What  Kind  of  STA  Program  You  Want 

The  first  step  is  to  determine  what  kind  of  STA  program  you  want.  The  Educators’  Technology  Center  of 
Indiana  suggests  that  you  begin  by  asking  yourself  who  needs  help  and  what  kind  of  help  are  needed 
(http://etc.iupui.edu/pyop.html).  Will  your  program  focus  solely  on  providing  technical  support  to  teachers  and 
instructional  computer  equipment?  This  is  one  of  the  most  common  and  cost-effective  models  for  an  STA  program. 

However,  STA  programs  that  address  technical  issues  AND  professional  development  can  promote 
technology  integration  throughout  the  school  and  sometimes  even  save  professional  development  costs.  Such 
programs,  however,  require  additional  management  and  resources  including,  respectively,  a coordinator  for  the 
program  and  monetary  support  from  the  school,  district,  and/or  state.  Such  programs  also  do  little  to  promote 
community  awareness.  Thus  extra  effort  may  be  needed  to  publicize  the  good  work  the  school  is  doing. 

STA  programs  that  combine  all  three  types  of  service  --  technical  service,  professional  development,  and 
community  projects  - are  generally  the  most  beneficial  (see  Figure  1 for  examples  of  the  three  types  of  service). 

The  downside,  of  course,  is  that  these  programs  require  a significant  commitment  of  resources  in  terms  of 
management  and  coordination.  Somebody  has  to  serve  as  liaison  to  the  community,  evaluate  possible  projects, 
assign  people  to  projects,  monitor  progress,  and  provide  publicity  in  addition  to  generating  the  technical  and 
professional  development  projects  done  in-house.  There  is  no  reason  a school  cannot  begin  with  one  type  of 
program  and  expand  as  the  school’s  needs  and  resources  grow. 
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Figure  I.  Three  types  of  service  provided  by  STAs. 


Analyzing  Existing  Resources 

Once  you  have  decided  what  kind  of  program  you  want  to  have,  you  need  to  examine  what  resources  are 
currently  available.  You  may  want  to  gather  some  data  through  surveys  and  analysis  of  repair  times  and  costs. 
However,  it  is  important  to  remember  that  costs  are  not  just  measured  in  dollars;  they  are  also  measured  in  attitudes 
and  use  of  technology,  so  include  items  about  perceptions  and  use.  Additional  questions  to  answer  about  your  school 
include: 

1.  Who  provides  technical  support  now? 

2.  Are  the  support  goals  being  met? 

3.  What  kinds  of  hardware  and  software  do  you  have  now? 

4.  What  kinds  of  resources  or  programs  are  available  at  other  schools  in  your  area? 

5.  Are  there  ways  to  pool  resources  with  other  schools? 

Talk  to  local  businesses  about  your  program;  see  if  they  may  be  willing  to  support  it  with  initial  donations 
of  equipment,  money,  or  staff  Are  there  employers  in  your  area  who  have  expressed  an  interest  in  helping  prepare 
students  for  technology -related  jobs?  You  may  have  to  limit  the  program  scope  initially  and  focus  on  some  high- 
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impact  projects  that  will  help  sell  the  program  to  your  teachers  and  to  the  community.  In  exchange,  this  may  aid  you 
in  finding  additional  resources  that  you  can  then  use  to  take  on  additional  projects. 

An  STA  program  is  not  likely  to  be  successful  in  the  long  run  if  its  coordination  is  simply  piled  on 
someone  who  is  already  overworked.  Whom  will  you  get  to  coordinate  the  program?  What  will  that  person  have  to 
give  up  in  order  to  do  it?  Who  will  pick  up  the  slack?  Are  there  parents  or  community  members  who  can  help? 
Perhaps  in  the  initial  year  of  the  program,  the  school  will  need  to  provide  a substitute  teacher  for  one  class  to  allow 
the  coordinator  to  develop  the  program. 

Make  sure  the  people  involved  are  dedicated  to  the  idea  of  the  program.  Kentucky’s  statewide  STA 
program,  SLTP,  emphasizes  that  coordinators  must  be  prepared  to  serve  as  communicators,  facilitators/mentors,  and 
also  managers.  The  state  STLP  web  site  (http://www.kde.state.ky.us/oet/customer/stlp/coor.asp)  urges  the  selection 
of  coordinators  who  possess  the  following  characteristics: 

• Knowledgeable  and  enthusiastic  about  technology 

• Energetic  in  pursuing  student  participation 

• Patient  in  pursuit  of  accomplishments 

• Dedicated  to  the  success  for  all  students  and  the  program 

• A leader  with  excellent  organizational  skills 

• Willing  and  able  to  solicit  the  support  of  administrators,  peers,  parents,  students,  professionals, 
and  community  organizations 

• Adaptive,  innovative,  and  reliable  in  fulfilling  duties 

Initial  Planning 

There  are  other  questions  that  merit  early  consideration.  A partial  list  is  offered  by  the  Educators’ 
Technology  Center  of  Indiana: 

• How  will  students  be  selected/recruited? 

• What  kinds  of  help  will  teachers/technicians  be  comfortable  with? 

• What  age  students  will  be  involved? 

• How  will  students  be  trained? 

• What  responsibilities  will  students  have?  Who  will  they  report  to? 

• Will  students  be  paid?  Given  credit?  Otherwise  recognized?  (http://etc.iupui.edu/pyop.html) 

Erica  Peto,  Esther  Onishi,  and  Barbara  Irish  have  published  a handbook  Tech  Team:  Student  Technology 

Assistants  in  the  Elementary  & Middle  Schools  (1997),  which  is  available  from  Amazon.com 
(http://www.amazon.com)  and  from  the  publisher’s  web  site  (http://www.linworth.com).  The  book,  which  is  based 
on  Peto’s  masters  thesis  describing  the  program  in  use  at  her  Kent  (Washington)  elementary  school,  outlines  how  to 
publicize,  organize,  and  manage  an  STA  program.  It  provides  lesson  plans,  sample  forms,  and  strategies.  On  her 
school  web  site  (http://www.kent.wednet.edu/staff/epeto/tech_team/),  Peto  offers  ClarisWorks  versions  of  many  of 
the  forms  in  use  at  her  school  as  well  as  early  drafts  of  lesson  plans  and  memos  included  in  the  book.  Particularly 
useful  are  her  suggestions  regarding  the  selection  process  of  student  participants.  She  describes  the  many  purposes 
of  the  interview  and  written  application  question: 

• To  determine  the  strengths  of  the  student  in  the  area  of  technology 

• To  determine  the  areas  in  which  the  student  may  need  more  instruction 

• To  determine  the  extent  of  experience  the  student  has  in  technology 

• To  determine  the  ability  the  student  has  in  working  with  adults  and  other  students 

• To  determine  the  perception  the  student  has  as  to  what  is  expected  of  him/her  as  a Tech  Team 
member 

• To  determine  if  the  student  has  given  some  thought  as  to  how  this  activity  will  fit  into  his/her 
overall  schedule 

• To  determine  the  ability  of  the  student  to  communicate  orally  and  in  writing 

• To  begin  to  build  rapport  with  prospective  Tech  Team  members 

Similarly  Lucy  Miller,  founder  of  the  SWAT  approach  which  is  used  heavily  in  North  Carolina  and 
elsewhere  around  the  world,  is  preparing  a manual  and  CD-ROM  which  will  be  available  for  purchase  through  the 
SWAT  website  (http://www.swatweb.net/).  Miller  encourages  e-mail  inquiries  (lmillerl@nc.rr.com)  regarding  her 
fee  schedule.  Early  versions  of  many  of  her  forms  were  previously  available  on  the  SWAT  website,  including  one 
form  which  contained  a detailed  and  explicit  parental  approval  section,  “I  understand  that  my  child  may  leave  their 
assigned  classes  for  short  periods  of  time,  with  teacher  permission,  to  assist  other  teachers  and  students  with 
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computer  related  activities.  These  activities  align  with  the  state  and  local  student  technology  competencies. 
Membership  on  the  SWAT  Team  is  an  enrichment  activity  and  will  be  monitored  by...” 

(http://prometheus.educ. ncat.edu/users/swat/application.htm).  Miller  stresses  the  importance  of  gaining  parental 
approval  in  the  first  stages  of  the  student’s  application  process. 

Another  helpful  checklist  of  initial  considerations  is  offered  by  Kristin  Kuntz  on  the  Intel  in  Education  web 
site  (http://www.intel.com/education/teachtech/classroom/tech_teams.htm). 

Measuring  Success 

Regardless  of  the  type  or  scope  of  STA  program  you  decide  to  implement,  there  are  several  factors  that  are 
considered  key  to  success.  This  section  outlines  some  of  these  factors  in  more  detail. 

Open  Participation 

One  of  the  false  assumptions  held  about  STA  programs  is  that  they  require  a highly  skilled  student  base, 
and  that  they  are,  therefore,  comprised  of  an  elite  corps  of  students.  In  actuality,  STA  programs  are  most  successful 
when  they  are  open  to  all  students,  even  those  with  poorly  developed  technology  skills.  When  students  participate  in 
an  STA  program,  they  are  likely  to  learn  from  each  other  and  from  the  training  they  receive  as  part  of  the  STA 
program.  There  is  often  a snowball  effect  created  once  a majority  of  the  students  are  sufficiently  competent. 

Students  who  enter  the  program  later  are  trained  faster  and  often  “invisibly”  by  their  peers. 

Successful  STA  programs  set  specific  goals  to  include  students  who  would  not  traditionally  participate  in 
such  activities.  Specifically,  females,  those  without  computers  at  home,  and  those  who  are  academically  less 
successful  should  be  encouraged  to  participate.  It  is  important  to  realize  that  girls  tend  to  shy  away  from  technology 
around  the  same  time  boys  begin  playing  computer  games.  Too  often  these  behaviors  lead  to  a technology  gap  in 
high  school  that  is  never  closed.  Involvement  in  an  STA  program  may  address  the  long-term  goal  of  promoting 
technology  skills  for  girls.  Similarly,  students  without  computers  at  home  often  do  not  develop  good  technology 
skills.  Including  such  students  in  an  STA  program  can  often  help  close  the  digital  divide  we  read  about  every  day, 
STA  participants  who  are  at-risk  academically  may  develop  more  positive  attitudes  towards  school  and  begin  to 
view  themselves  as  valuable  members  of  the  community. 

Clearly  Identified  Mission,  Goals,  & Objectives 

When  starting  an  STA  program,  you  need  to  be  very  clear  about  the  mission,  goals,  and  objectives  for  your 
program.  The  mission  and  goals  guide  how  you  will  develop  your  program.  They  help  you  communicate  to  the 
community  and  other  potential  sources  of  funding  exactly  how  you  have  been  successful  and  what  you  intend  to  do 
with  any  donated  equipment,  time,  or  funds.  They  also  ensure  that,  when  asked,  individual  meni^ers  of  the  program 
give  consistent  and  accurate  descriptions  of  the  program  to  those  who  may  question  what  the  value  or  purpose  of  the 
STA  is  for  the  participants. 

External  Motivation 

While  the  intrinsic  motivation  provided  by  STA  programs  can  be  high,  some  schools  offer  external 
motivation  in  the  form  of  an  hourly  wage,  typically  in  the  range  of  $5  or  $6  an  hour.  Another  possibility  is  to  offer 
scholarships  or  awards  such  as  the  two  BellSouth  STEP  Scholarships  given  each  year  to  seniors  who  have  excelled 
in  one  of  Kentucky’s  STEP  programs  (http://162.1 14.158.30/scholar/new.htm).  The  STEP  annual  conference  for 
students  provides  students  the  opportunity  to  travel  and  to  be  recognized  for  their  work.  The  STEP  model  also 
provides  external  motivation  for  schools  and  school  districts  in  the  form  of  Diamond,  Gold,  and  Silver  recognition. 
Recognition  of  community  participants  can  also  help  boost  community  involvement. 

Training  & Support 

While  some  students  and  teachers  who  want  to  participate  in  your  STA  progra  m may  have  skills,  not  all 
will.  And  someone  who  has  technical  know-how  may  not  know  how  to  apply  those  skills  within  the  context  of  your 
STA  program.  Regular  training  for  current  and  new  STAs  is  required  for  the  long-term  success  of  a program.  In 
addition,  those  who  coordinate  the  program  must  get  the  support  and  time  they  need  to  manage  the  program 
effectively. 
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Organized  and  Effective  Coiiaboration 

Your  STA  program  needs  to  have  a visible  presence  and  contact  person  in  the  community.  Interested 
individuals  need  to  have  easy  access  to  program  leaders  in  order  to  propose  ideas  or  volunteer  services.  Students  will 
grow  through  exposure  to  your  community’s  multiple  points  of  view  and  cultures.  Again,  this  can’t  be  done  easily 
by  a teacher  or  staff  member  who  is  coordinating  the  program  in  addition  to  other  duties.  It  is  probably  best 
accomplished  by  the  involvement  of  several  community  and  business  members  who  serve  as  leaders  in  the  program. 

Six  Successful  STA  Programs 

Table  1 provides  six  examples  of  STA  programs  in  place  in  K-12  schools  today,  along  with  a brief 
description  of  their  areas  of  focus.  This  is  followed  by  a more  in-depth  analysis  of  these  programs  and  their 
components. 


583 


1072 


Table  I Profiles  of  Six  Student  Technolo^  Assistant  Programs 


Program 

Type  of  program 

What  it  does/more  information 

Issaquah  School 
District  Technology 
Information  Project 
(TIP) 

Primarily  Technical 

Probably  the  first  formal  STA  program,  TIP  is  offered  as  a middle 
school  and  high  school  course  that  includes  work  toward  technical 
certification. 

http://www.issaquah.wednet.edu/district/tip.htm 

Students  Recycling 
Used  Technology 
(StRUT) 

Primarily  Technical 

Founded  by  Intel  and  the  Northwest  Regional  Education  Service 
District,  StRUT  became  a statewide  Oregon  program  in  1997-98. 
StRUT  students  evaluate,  repair  and  refurbish  donated  computers 
and  in  turn  donate  those  computers  to  local  schools.  The  program 
has  been  replicated  in  other  states. 

http://www.strut.org 

Generation  Yes 

Primarily 

Instructional 

An  outgrowth  of  Olympia  (Washington)  Network  Navigators  clubs, 
this  program  has  received  federal  funding  and  recognition.  Its 
curriculum  teaches  students  the  technology,  presentation,  and 
mentoring/teaching  skills  that  they  then  use  to  help  teachers 
integrate  technology  into  their  classrooms.  It  has  recently  expanded 
to  include  three  additional  curricula  - one  focused  on  community 
projects,  one  focused  on  network  maintenance,  and  one  addressing 
issues  of  students  who  “don’t  like  computers.” 

http://www.genyes.org 

Kentucky  Student 
Technology 
Leadership  Program 
(STLP) 

Technical, 
Instructional,  and 
Community- Related 

This  well-organized  program  is  administered  on  the  state  level  and 
promoted  in  districts  and  schools  across  the  state.  Each  year 
students  participate  in  a statewide  conference. 

http://www.kde.state.ky.us/oet/customer/stlp 

Students  Working  to 
Advance  Technology 
(SWAT) 

Technical, 
Instructional,  and 
Community- Related 

SWAT  is  a network  of  schools  that  embrace  the  organization  model 
developed  by  Lucy  Miller  in  which  students  are  assigned  to  task 
forces.  Many  SWAT  schools  are  in  North  Carolina  where  students 
participate  in  a free  statewide  summer  program. 

http://www.swatweb.net 

Tech  Team 

Technical, 
Instructional,  and 
Community- Related 

This  program’s  details  have  been  well -documented  by  its  founder 
Erica  Peto  and  two  classroom  teachers  who  are  active  in  the 
program.  Documentation  is  geared  toward  elementary  and  middle 
schools. 

http://www.kent.wednet.edu/staff/epeto/tech  team/ 
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Issaquah  School  District  Technology  Information  Program  (TIP) 

The  Technology  Information  Project  (TIP)  varies  in  its  implementation  from  school  to  school  within  the 
Issaquah  School  District  (http://www. issaquah, wednet.edu/district/tip.htm).  However,  in  general  all  of  the  TIP 
programs  follow  an  apprenticeship  model  of  training.  One  former  TIP  coordinator,  quoted  in  a Microsoft  in 
Education  case  study  (http://www.microsoft.com/education/planning/implement/system_issaquah.asp),  described  his 
motto:  “Every  computer  here  is  equipped  with  a teenager.” 

At  the  middle  school  level,  all  students  are  welcome  regardless  of  their  prior  experience  with  technology. 
Students  pledge  to  teach  each  other.  At  the  high  school  level,  students  are  expected  to  bring  prior  experience  and 
skills.  They  are  carefully  screened  during  an  application  process  that  includes  an  ethics  test  and  a teacher 
recommendation.  Students  accepted  into  the  program  receive  course  credit.  In  some  cases,  basic  skills  are  taught  not 
only  through  authentic  learning  assignments  but  also  through  a certification  preparation  program.  In  addition  to 
these  basic  skills,  TIP  programs  stress  “an  appetite  for  life-long  learning  and  goal-setting” 
(http://www.ims.Issaquah.wednet.edu/isdtip.htm). 

Sample  TIP  projects  include  the  following: 

• Provide  afternoon  access  to  the  school  computer  lab  and  library  computer  facilities 

• Provide  technical  support  for  feeder  schools  and  community  centers 

• Maintain  the  school’s  user  network  and  mail  accounts 

• Help  monitor  security  and  space  issues  on  the  school’s  network 

• Maintain  and  install  all  network  wiring  for  the  school 

• Provide  routine  maintenance  and  technical  support  for  the  school 

• Help  train  staff  and  students  on  the  use  of  the  school’s  computer  workstations 

The  student-maintained  TIP  page  for  Liberty  High  School  describes  the  philosophy  of  that  school’s 
program  and  its  emphasis  on  public  presentation  of  student  work 

(http://www.liberty.issaquah.webnetxduyTiPpage/philosophy.htm):  “Imagine  how  it  feels  for  a sixteen  year  old  to 
get  up  in  front  of  a group  of  ‘suits*  some  300  strong  and  actually  have  them  TAKING  NOTES  as  you  speak.” 

The  following  individual  schools  maintain  TIP  web  sites: 

• Issaquah  Middle  School 

o http://www.ims.issaquah.wednet.edu/isdtip.htm 

• Issaquah  High  School 

o http://www.ihs.issaquah.wednet.edu/ihstip/ 

• Liberty  High  School 

o http://www.liberty.issaquah.webnet.edu/TiPpage/ 

Students  Recycling  Used  Technology  (StRUT) 

Another  technology -focused  STA  program,  StRUT,  was  co-founded  by  Intel  and  the  Northwest  Regional 
Education  Service  District  in  Oregon  in  1995  (http://www.strut.org).  The  program  was  designed  to  refurbish  and 
make  use  of  donated  computer  equipment.  Specifically,  the  goals  of  the  program  are  as  follows: 

1.  To  develop  programs  where  students  gain  valuable  technical  and  business  management  skills  by 
assembling,  testing  and  loading  software  on  donated  computers. 

2.  To  place  these  computers  in  schools  throughout  the  area  to  supplement  those  purchased  by  the 
district. 

3.  To  work  with  business  partners  to  reduce  the  barriers  preventing  schools  from  accessing  the 
Internet  and  other  communication  technologies. 

StRUT  has  expanded  to  each  of  the  21  districts  in  Oregon.  To  date,  more  than  1,000  students  have  been 
involved  in  the  StRUT  program,  and  these  students  have  refurbished  over  15,000  computers  and  donated  them  to 
more  than  70  schools.  In  1999,  StRUT  received  Oregon’s  SOLV  Citizenship  Award. 

Most  StRUT  programs  are  run  as  after  school  clubs  that  emphasize  field  trips  and  rigorous  participation  in 
certification  programs  such  as  A+  and  Cisco.  Some  StRUT  programs  have  joined  forces  with  Cisco  Networking 
Academy  to  enhance  this  training.  One  such  program  is  West  Albany  High  School’s  Computer  Networking 
Program.  Its  coordinator  Dave  Hudson  is  a 1999  recipient  of  the  Oregon  Innovators  in  Education  Award.  His  report 
on  the  program  is  available  online  (http://www.osba.org/salute/2000/sal0002.htm). 

The  StRUT  alliance,  as  an  example  of  a technical  STA  program,  has  proven  to  be  so  successful  that  it  has 
expanded  beyond  Oregon’s  boundaries.  StRUT  is  being  implemented  in  six  other  states  - Arizona 
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(http://www.azstrut.org/),  California,  Georgia  (http://www.sandersvilletech.org/gastrut/),  Massachusetts,  New 
Mexico,  and  Texas  (http://www.txstrut.org/). 

The  following  individual  Oregon  schools  are  among  those  maintaining  StRUT  web  sites: 

• Sam  Barlow  High  School 

o http://www.gresham.kl2.or.us/strut/default.html 

• Spray  High  School 

o http://www.spray.kl2.or.us/strut/strut.htm 

• Nestucca  High  School 

o http://www.nestucca.kl2.or.us/strut.htm 


Generation  YES 

Generation  YES  (Youth  and  Educators  Succeeding),  a third  ST  A program  with  roots  in  the  Pacific 
Northwest,  initially  focused  exclusively  on  instructional  issues  (http://www.genyes.org).  Even  now,  the  primary 
curriculum  offered  by  Generation  YES  (variously  called  Gen  YES,  Gen  Y,  and  Gen  www.Y)  is  a training  program 
that  prepares  students  in  grades  3 through  12  to  serve  as  mentors  to  teachers  who  are  integrating  technology  into 
their  classroom  teaching.  Graduates  of  this  semester-long  program  can  now  continue  with  the  Gen  DID  curriculum 
(which  focuses  on  a major  community  project).  Students  who  demonstrate  maturity  and  technical  skills  may  enroll 
in  the  Generation  SCI  curriculum  (Students  Caring  for  Infrastructure,  which  focuses  on  computer  network 
maintenance).  A fourth  curriculum,  Gen  GIT  (Girls’  Issues  and  Technology),  is  designed  to  meet  the  needs  of  girls 
who  “don’t  like  computers.”  The  more  than  500  schools  in  41  states  affiliated  with  Generation  YES  now  constitute 
probably  the  most  widespread  ST  A network  in  the  country. 

An  outgrowth  of  the  Olympia  School  District’s  Network  Navigator  mentoring  program.  Generation  YES 
received  a federal  Technology  Innovation  Challenge  Grant  in  1996.  Dennis  Harper,  author  of  this  initial  grant,  still 
serves  as  executive  director  for  Gen  Yes,  which  is  now  administered  by  the  Office  of  Educational  Research  and 
Improvement  (OERI).  The  Generation  YES  curriculum  kits,  which  are  aligned  with  the  International  Society  for 
Technology  in  Education  (ISTE)  National  Educational  Technology  Standards,  are  available  for  purchase  through  the 
ISTE  bookstore  (http://www.iste.org/Bookstore/index.html).  The  ISTE  magazine  Learning  and  Leading  with 
Technology  profiled  Generation  YES  in  October  1999 

(http://www.iste.org/L&L/archive/vol27/no2/features/harper/index.html)  and  regularly  features  the  writing  of  Gen 
YES  students  in  its  “Student  Voices”  column.  In  2000,  the  U.S.  Department  of  Education’s  Educational  Technology 
Expert  Panel  recognized  Gen  YES  as  one  of  only  two  “Exemplary  Programs” 

(http://www.ed.gOv/offices/OERI/ORAD/LTD/newtech_progs.html).  Most  recently,  USA  highlighted  the 

program  in  an  August  8,  2001,  back-to-school  article  (http://www.usatoday.com/life/cyber/tech/2001  -08-06- 
students  -tutor-teachers.htm). 

Students  who  participate  in  the  basic  program  enroll  in  a semester-long  (18 -week)  course  to  learn  necessary 
technical  skills  and  integration  techniques  to  help  their  teachers  with  technology-related  lesson  planning.  The 
program  does  not  require  students  to  have  prerequisite  technology  skills,  but  the  challenging  nature  of  the 
curriculum  does  require  significant  student  effort.  Each  student  is  paired  with  a teacher  to  assist  with  a lesson  or 
project’s  technology  integration. 

In  one  such  project  for  a social  studies  class,  an  STA  helped  a teacher  develop  a unit  on  Graham  County 
history  that  made  use  of  a scanner,  digital  video  camera.  Avid  Cinema  editing  software,  and  Photo  Deluxe  image 
editing  software.  The  teacher  and  the  STA  filmed  different  historical  sites  around  the  county  using  a digital  video 
camera.  Students  in  the  teacher’s  class  meanwhile  collected  pictures  of  Graham  County  that  were  then  scanned  in 
and  combined  with  the  video,  which  was  edited  to  include  music.  The  video  was  then  shown  to  the  class  as  part  of 
the  unit  on  the  history  of  Graham  County,  and  students  were  tested  on  the  content. 

Another  project  developed  for  a science  class  took  a chapter  from  the  class  textbook  and  created 
storyboards  for  the  chapter.  These  storyboards  called  for  pictures  and  images  for  illustration,  which  were  gathered 
from  the  Internet,  the  textbook,  and  other  print  resources.  Images  were  scanned  as  necessary  and,  combined  with  the 
storyboards,  were  then  used  to  create  PowerPoint  slides.  The  teacher  presented  the  unit  to  the  class. 

In  a third  project,  for  an  English/Language  Arts  class,  an  STA  worked  with  the  teacher  to  teach  students 
how  to  use  a digital  camera  to  take  pictures  and  how  to  edit  those  pictures  in  an  image  editing  program.  As  a 
practice  assessment,  the  teacher  and  students  went  on  a scavenger  hunt,  taking  pictures  of  specific  objects  and  then 
editing  them.  The  teacher  and  the  STA  developed  “story  starters”  which  the  students  then  elaborated,  continuing  the 
stories  and  illustrating  them  with  digital  photos.  More  projects  and  information  can  be  found  at 
http  ://www.  genyes.  org/gen  wwwy/. 
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Betow  are  just  some  of  the  web  sites  of  participating  Gen  Yes  schools: 

• Argyle  Central  School  (Argyle,  New  York) 

o http://www.nheeep.org/SchoolWebs/argyleweb/Programs/GenY.htm 

• Birmingham  High  School  (Los  Angeles,  California) 

o http://www.lausd.kl2.ca.us/Birmingham_Magnet__HS/GENYWEB/ 

• Thurgood  Marshall  Middle  School  (Olympia,  Washington) 

o http://kids.osd.wednet.edu/Marshall/homepage/thurgoodmarshall.html 

Kentucky  Student  Technology  Leadership  Program  (STLP) 

Kentucky’s  Student  Technology  Leadership  Program  (STLP)  exemplifies  a state-organized  STA  program 
(http://www.kde.state.ky.us/oet/customer/stlp).  According  to  its  web  site,  828  Kentucky  schools  in  165  Kentucky 
school  districts  currently  participate.  Although  at  first  glance  its  vastness  and  organizational  complexity  may  seem 
intimidating,  many  of  STLP’s  features  are  noteworthy  even  for  rural  schools  who  are  considering  implementing  a 
small-scale  STA  program.  Specifically,  Kentucky’s  program  involves  students  in  technical,  instructional,  and 
community- related  projects,  making  its  usefulness  broader  than  one  specific  area.  Additionally,  the  program 
provides  local  districts  with  state-level  guidance,  a feature  that  helps  insure  that  proper  instructional  objectives  and 
standards  are  being  met.  The  state-organized  nature  of  the  program  allows  for  more  uniform  development  and 
collaboration  across  the  state.  State-run  competitions  and  student  conferences  assure  that  students  work  hard  to 
maintain  a high  quality  of  work. 

Kentucky’s  STAs  have  performed  a number  of  technical-related  tasks.  Specifically,  they  have  created  and 
maintained  web  pages,  wired  classrooms  and  laboratories,  produced  videos,  and  assisted  faculty  members  with 
technology-related  problems.  These  projects  are  varied  and  numerous  and  go  beyond  simple  technical 
troubleshooting. 

The  instructional  projects  of  Kentucky’s  program  are  also  broad  and  diverse.  Students  have  developed 
brochures,  mentored  younger  students,  developed  electronic  portfolios,  evaluated  software,  and  provided  technical 
instruction.  One  STA  school  developed  a PowerPoint  presentation  on  the  state  of  Kentucky  using  graphics  and 
sounds  of  the  state  bird  and  the  state  song.  Another  STA  school  implemented  electronic  portfolios  of  HyperStudio 
work,  studied  and  reported  on  technology  related  issues  such  as  Y2K,  developed  web-based  curriculum,  illustrated 
poems  with  digital  pictures,  built  WebQuests  for  several  topics,  created  audio/video  broadcast  coverage  for 
campaigns  and  a shuttle  launch,  and  generated  a variety  of  print-based  materials  such  as  news  letters  and 
newspapers.  In  another  school,  students  reviewed  software  programs  and  then  held  a showcase  to  introduce  the 
software  to  teachers.  Another  project  enlisted  STAs  in  helping  parents  learn  how  to  use  the  Internet  during  weekly 
“Internet  night”  workshops.  STAs  in  yet  another  program  developed  a program  called  “Technology  Opportunities  in 
the  Library.”  Teachers  attending  this  program  learned  how  to  use  a digital  camera,  a scanner,  HyperStudio,  and  a 
Laserdisc  player.  Ideas  for  integrating  these  technologies  into  classroom  curriculum  were  also  presented.  There  are 
also  scores  of  instances  where  STAs  have  helped  teachers  and  community  members  learn  to  use  programs  like 
Excel,  PowerPoint,  and  HyperStudio;  to  troubleshoot  and  maintain  classroom  computers;  and  to  use  a variety  of 
hardware. 

STLP  has  generated  hundreds  of  community  projects.  Students  in  this  program  have  helped  maintain  city 
web  sites  and  build  and  maintain  web  sites  for  community  organizations  and  schools.  They  have  converted 
networking  systems  and  installed  computers  for  neighboring  schools,  conducted  research  and  web  page  maintenance 
for  local  companies,  and  taught  community  education  classes  in  PowerPoint  and  other  software  applications.  They 
have  made  holiday  cards  and  large-print  phonebooks.  They  have  held  workshops  for  senior  citizens  in  assisted- 
living  facilities,  managed  and  supported  a lending  library  for  software  and  hardware  open  to  the  community,  and 
created  business  cards  and  pamphlets  for  local  organizations. 

Providing  technical,  instructional,  and  community-related  assistance  to  others  follows  the  mission  of 
Kentucky’s  Student  Technology  Leadership  Program.  The  STLP  mission  aims  to  advance  the  individual  capabilities 
of  all  students,  to  motivate  all  students,  and  to  create  leadership  opportunities  through  the  use  of  technology.  The 
specific  goals  of  this  mission  are  as  follows: 

• The  STLP  will  develop  activities  that  enhance  the  academic,  social  and  emotional  growth  of  the 
student. 

• The  STLP  will  provide  leadership  opportunities  for  all  students. 

• The  STLP  will  experience  multi-age  collaboration  by  forming  innovative  learning  partnerships. 

• The  STLP  will  form  learning  partnerships  between  students  with  different  technology  skills. 
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• The  STLP  will  develop  activities  that  benefit  communities. 

• The  STLP  will  develop  instructional  activities  which  integrates  technology  and  benefits  the  school 
and  support  KETS  (Kentucky  Education  Technology  System) 

From  its  inception,  these  six  goals  have  guided  STLP.  In  her  \ 999  Learning  and  Leading  with  Technology 
article,  Elaine  Harrison  describes  the  advisory  council  that  formulated  these  goals  and  structured  the  program  in  its 
early  days.  She  notes,  “The  Kentucky  Department  of  Education  gave  grants  ($1,000  per  year)  to  the  initial  STLP 
schools  for  the  1994-95  and  1995-96  school  years.”  Online  manual  materials  describe  the  program’s  structure  - an 
adult  STLP  coordinator  for  the  school  and  student  cluster  coordinators  for  individual  projects  - and  the  process 
followed  to  create  a unified  action  plan  that  meets  the  program’s  six  goals. 

One  notable  aspect  of  Kentucky’s  program  is  its  inclusiveness.  The  program  strives  to  include  a population 
fiilly  representing  the  school’s  diversity.  This  means  that  the  program  aims  to  draw  females,  minorities,  and  special 
education  students.  Activities  are  scheduled  with  a flexibility  that  assures  that  no  student  is  excluded  on  the  basis  of 
schedule.  As  a means  of  assuring  inclusiveness,  the  STLP  program  stresses  four  important  roles  for  students 
involved  in  the  program  - starter,  liaison,  trainer,  and  provider. 

Below  is  a sampling  of  individual  STLP  web  sites: 

• Charles  Russell  Elementary  School 

o http://crussell.ashland.kl  2.ky.usZstlp\stlp.htm 

• Paul  Lawrence  Dunbar  High  School 

o http://www.pldhs.com/stlp.html 

• Region  6 

o http://www.kl2.ky.us/oapd/rsc6/stlp.asp 
SWAT  - Students  Working  to  Advance  Technology 

Another  nationally  recognized  STA  program,  SWAT,  focuses  its  efforts  on  student  leadership  training, 
technology  integration,  and  community  outreach.  Although  the  program  has  been  implemented  across  the  nation, 
individual  programs  are  expected  to  each  have  their  own  personality.  SWAT  teams,  as  many  implemented  programs 
have  been  called,  are  supported  by  guidelines  (http://www.fetc.org/fetcon/!  199/swat.html),  yet  each  of  the  programs 
are  expected  to  assess  the  technological  needs  of  their  local  school  and  community. 

Students  who  are  interested  in  participating  in  a SWAT  program  are  usually  required  to  complete  a Job 
application  and  complete  an  interview.  In  this  way,  teachers  can  more  appropriately  evaluate  the  types  of  programs 
to  embrace,  and  perhaps  more  importantly,  to  effectively  match  students  with  technology  tasks. 

After  assessing  the  technology  needs  of  the  local  school  and  community,  those  who  are  implementing  the 
SWAT  program  are  encouraged  to  do  the  following: 

• Communicate  concerns  with  administrators,  teachers,  and  the  community 

• Develop  a program  mission,  with  goals,  priorities,  a plan  of  action,  and  a method  of  evaluation 

• Announce  the  plan  to  the  students 

• Distribute  applications  and  obtain  parental  permission 

• Conduct  interview  to  build  SWAT  teams 

• Assess  the  technology  needs  of  the  local  school  and  community 

• Train  the  students 

• Monitor  and  evaluate  the  program 

Task  Force  Teams,  the  organizational  name  given  to  a job  that  a student  is  assigned  to  complete,  can 
include  any  number  of  responsibilities.  Examples  of  these  teams  include  the  following:  TV/Weather  Internet  Crew, 
Internet  Researcher,  Web  Master,  and  Computer  Buddy.  Of  course,  the  possibilities  are  limitless  for  teams  and 
responsibilities.  But  again,  local  needs  should  drive  the  development  of  teams. 

During  summer  2001,  the  second  annual  SWAT  Camp  was  offered  to  North  Carolina’s  SWAT  students  at 
seven  CyberCampuses  <http://www.dlt.ncssm.edu/swat/swat_camp.cfm>.  The  program,  which  is  free  to  students,  is 
funded  through  a Technology  Innovation  Challenge  Grant. 

One  web  site  hosted  by  the  North  Carolina  School  of  Science  and  Mathematics  serves  as  a resource  for  ten 
high  school  SWAT  programs  in  North  Carolina  <http://www.dlt.ncssm.edu/SWAT/index.cfm>.  This  consortium  of 
SWAT  programs  is  currently  planning  a portfolio  process  by  which  students  can  become  “SWAT  Team  certified”  in 
web  page  design:  “Certification  implies  that  the  student  is  capable  of  designing  pages  for  school  community 
members  and  organizations.” 

Below  are  a sampling  of  other  school  web  sites  affiliated  with  SWAT: 
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• Narragansett  Elementary  School  (Narragansett,  Rhode  Island) 

o http://www.ri.net/schools/Narragansett/NES/SWAT/swat.html 

• Eastview  Elementary  School  (Connersville,  Indiana) 

o http://fayette.kl  2.in.us/eastview/computerclub.html 


Tech  Teams 

Erica  Peto’s  model  for  TechTeams  has  been  cited  as  an  influence  for  a variety  of  programs,  including  the 
Ballard  Tech  Team,  which  is  a featured  School  Web  Clubs  web  site 

<http://supportnet.merit.edu/webclubs/featuredl.html>.  In  a June  2000  Education  World  article,  Ballard  teacher 
Marcia  Cousins  explains,  “As  the  tech  people  in  our  building,  we  were  becoming  more  and  more  overtaxed  and 
overwhelmed  with  trying  to  teach  and  also  be  the  'techies’...  As  we  added  more  and  more  computers,  the  task 
increased.  Hence,  the  idea  for  a tech  team  evolved.  We  had  the  opportunity  to  see  a presentation  on  tech  teams  and 
had  the  resource  of  a book  --  Teams,  Student  Technology  Assistants  in  the  Elementary  and  Middle  School^  published 
by  Linworth  Publishing  [ISBN  0-938865-60-9].  This  book  is  geared  just  towards  a tech  team  and  not  a Web  club. 
Ours  has  developed  into  both"  <http://www.educationworld.com/a_tech/tech035.shtml>. 

Similarly,  Mississippi’s  new  CREATE  (Challenging  Regional  Educators  to  Advance  Technology  in 
Education,  http://www.create4ms.org)  program  draws  from  Peto’s  model  in  designing  student  Techno  Teams  who 
provide  support  for  the  program’s  technology  carts.  Schools  within  the  CREATE  program  are  encouraged  to  gain 
ideas  and  inspirations  from  Peto’s  workbook  but  to  create  tech  teams  suited  for  their  own  schools. 

Peto’s  model,  as  described  on  the  web  site  <http://www.kent.wednet.edu/staff/epeto/tech_team>,  is 
designed  to  meet  the  needs  of  her  own  school,  Daniel  Elementary,  as  well  as  other  schools  in  the  Kent  School 
District.  As  she  explains,  “The  main  function  of  the  Tech  Team  is  to  assist  during  media  block  (a  combination  of 
library  and  computer  classes).  Tech  Team  members  also  help  during  preschool  and  Head  Start  preschool  computer 
center  time.” 

Tech  Team  members  at  Daniel  are  fourth,  fifth  and  sixth  grade  students  who  volunteer  to  miss 
approximately  one  hour  of  class  time  per  week  and  make  up  that  work  on  their  own:  “During  this  time,  they  assist 
students  who  are  learning  to  use  technology  and  working  on  technology-related  projects.  In  addition.  Tech  Team 
students  agree  to  attend  a training  session  once  per  week.  Last  year  this  training  time  was  held  after  school  but  this 
year  it  is  held  during  lunch  recess.” 

The  school’s  web  site  <http://www.kent.wednet.edu/KSD/DE/st_proj/st_proj.html>  offers  a glimpse  of  a 
variety  of  impressive  student  projects  supported  by  Daniel’s  Tech  Team  members. 

Conclusion 

As  these  models  demonstrate,  STA  programs  can  be  an  excellent  way  for  schools  to  promote  technology 
integration,  for  teachers  and  students  to  gain  technology  skills,  for  students  to  become  more  involved  and  committed 
to  their  communities,  and  for  communities  to  gain  services  they  could  not  otherwise  get  and  a more  educated 
workforce.  In  most  cases,  the  start-up  costs  are  small. 

Before  embracing  one  of  these  models  or  creating  your  own,  you  should  first  explore  what  is  being  done  in 
other  schools  in  your  state.  Have  several  schools  within  your  state  adopted  the  Gen  Yes  curriculum?  Does  StRUT 
operate  in  your  state?  Is  there  a large  employer  in  your  state  interested  in  helping  coordinate  a StRUT-like  program? 
Is  your  State  Department  of  Education  considering  a statewide  initiative  such  as  Kentucky’s?  Perhaps  your  state  can 
achieve  the  synergy  Mississippi  schools  achieved  when  they  applied  for  and  acquired  federal  funding  for  the 
CREATE  project. 

The  key  to  any  successful  program  must  be  that  it  meets  the  needs  of  your  school  and  your  students.  And 
though  STA  programs  have  proven  helpful  in  addressing  schools’  budgetary  crises,  no  STA  program  should  be 
viewed  as  a permanent  replacement  for  budgeted  technical  support  dollars.  For  the  full  potential  of  STA  programs  to 
be  achieved  - technical,  instructional,  and  community  - the  focus  must  firmly  centered  on  students’  academic  needs. 
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ABSTRACT 

At  no  time  in  history  has  the  ability  to  locate,  organize,  evaluate,  manage  and  use  information,  skills 
collectively  referred  to  as  information  literacy,  been  more  important  to  today’s  learners.  Classroom  and 
technology  teachers  and  library  media  specialists  are  challenged  to  find  effective,  innovative  techniques  for 
teaching  research  and  information  skills,  especially  to  young  children.  This  paper  summarizes  the  research 
conducted  for  a U.S.  Department  of  Education  Phase  I SBIR  award.  The  project  utilized  digital  video, 
database,  and  information  technologies,  to  design  a proof  of  concept  prototype  for  a comprehensive  Web- 
based  tool,  5.O.5.,  for  improving  instruction  in  this  critical  area.  Educators  identify  relevant  situation- 
specific  variables  (S.)  and  desired  instructional  outcomes  (O.).  Suggested  instructional  strategies  (S,)  are 
subsequently  generated.  These  strategies  are  linked  to  a database  of  real-world  video  and  multimedia 
examples.  S,0,S,  will  be  responsive  to  advancing  technology  and  include  system  feedback  mechanisms  as 
well  as  direct  user  input  for  continuous  formative  evaluation  and  improvement.  By  integrating  sound 
pedagogical  principles  with  real-world  practice  presented  in  video  and  multimedia  demonstrations,  the 
project  will  make  a valuable  contribution  to  the  quality  of  information  literacy  skills  instruction. 

Introduction 

This  paper  highlights  the  main  activities  and  results  of  an  SBIR  Phase  I project  to  research  and  design  a 
proof-of-concept  prototype  for  improving  instruction  in  information  literacy.  It  utilized  digital  video,  database,  and 
information  technologies  in  a comprehensive  tool,  S.O.S.  for  Information  Literacy^  for  improving  instruction  in  this 
critical  area.  Educators  identify  relevant  situation-specific  variables  and  desired  instructional  outcomes  (O.). 
Suggested  instructional  strategies  (S.)  are  subsequently  generated.  These  strategies  are  linked  to  a database  of  real- 
world  video  demonstrations. 

A high  degree  of  success  was  achieved  for  all  Phase  1 objectives.  Top  level  objectives  included  1) 
performing  a front-end  analysis  to  determine  how  best  to  design  a Web-based  information  system  that  meets  the 
needs  of  its  target  audience  (elementary  library  media  specialists  and  teachers)  2)  designing  the  Phase  I 
specifications  (e.g.,  scope/organization  of  content,  project  curriculum,  and  design  specifications  for  the  proof-of- 
concept  prototype,  3)  producing  the  Phase  I prototype  including  data  structure,  sample  videos,  and  user  feedback 
mechanisms,  and  4)  evaluating  the  proof-of-concept  prototype  across  a number  of  variables  including  ease  of  use, 
technical  reliability,  quality  of  content,  interface  design  and  general  appeal.  Major  accomplishments  included: 

• The  development  of  an  online  Research  Management  Site  provided  an  excellent  environment 
within  which  the  Phase  1 research  activities  occurred  and  will  prove  useful  as  the  project  continues 
into  its  development  phase. 

• A survey  of  192  practitioners  supplied  critical  information  on  the  target  audience’s  needs  and 
preferences  as  well  as  clarification  of  a number  of  content  issues. 

• A questionnaire  to  faculty  and  leaders  (n  = 8)  illuminated  the  need  to  add  several  subskills  to  the 
search  parameters  in  Phase  11,  and  provided  useful  information  for  dissemination  of  S.O.S.  project 
information. 

• Input  evaluation  resulted  in  guidelines  for  software  development  and  video  production. 

• Production,  post  production,  and  compression  of  18  sample  videos  demonstrated  that  quality  video 
could  be  delivered  via  the  web  at  high  compression  schemes. 

• Sophisticated  data  structure  and  query  systems  were  the  outcomes  of  software  development  over 
two  iterations  of  the  proof-of-concept  prototype. 
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• A Focus  Group  (n  = 1 1)  of  potential  users  helped  to  elucidate  the  results  of  the  earlier 
practitioners’  survey  and  to  beta  test  the  first  iteration  of  the  prototype. 

• Evaluation  of  2"^^  iteration  prototype  by  a Progressive  Feedback  Panel  (n  = 1 1)  yielded  valuable 
information  for  Phase  II  development  relating  to  functionality,  technical  quality,  and  interface  and 
design  issues. 

The  remainder  of  this  paper  will  briefly  discuss  each  of  the  above  accomplishments,  and  conclude  with 
information  on  current  development  efforts. 

Development  of  Research  Management  Site 

The  Research  Management  Site  (RMS)  was  a controlled  online  environment  within  which  the  Focus  Group 
and  Progressive  Feedback  Panel  activities  took  place.  The  Focus  Group  (which  will  be  described  later)  tested  the 
first  iteration  prototype  while  the  Progressive  Feedback  Panel  tested  a later  version.  The  RMS  was  first  developed  in 
HTML  and  then  brought  into  the  WebCT  Courseware  so  that  threaded  discussions  could  take  place.  Additionally,  it 
served  as  a jumping  off  spot  to  link  to  the  S.O.S.  beta  test  site  and  as  a return  point  to  continue  in  the  Focus  Group 
or  Progressive  Feedback  Panel  mode.  The  success  of  conducting  research  within  the  RMS  environment  in  Phase  I 
has  set  the  stage  for  successful  research  efforts  in  Phase  II. 

Results  of  Online  Practitioners’  Survey 

A 40-item  survey  was  designed  to  gather  feedback  on  potential  curriculum  content,  perceived  usefulness  of 
the  product,  and  plans  for  current  and  future  technology  implementation  in  respondents’  schools  and  homes.  The 
survey  was  designed  as  a comprehensive  set  of  items  including  multiple  choice,  short  answer,  and  open-ended 
questions.  The  first  draft  of  the  survey  was  piloted  with  32  graduate  students  enrolled  in  a class  called  “Instructional 
Strategies  and  Techniques  for  Information  Professionals”  at  Syracuse  University,  School  of  Information  Studies. 

The  pilot  test  served  three  purposes:  1)  test  the  appropriateness  of  the  survey’s  dissemination  medium,  2)  test  the 
robustness  of  the  dissemination  medium,  and  3)  gamer  item  feedback.  After  some  modifications  the  survey  was 
announced  on  several  listservs  (e.g.  LM__NET,  ED -TECH)  and  192  library  media  specialists,  teachers,  and 
technology  coordinators  responded. 

Ninety-four  percent  (94%)  of  respondents  indicated  that  the  S.O.S.  system  as  described  would  be  either 
“useful”  or  “very  useful”  to  library  media  specialists  and  85%  felt  it  would  be  “useful”  or  “very  useful”  to 
classroom  teachers.  Participants  were  given  a number  of  potential  uses  for  S.O.S.  and,  while  the  majority  indicated 
the  importance  of  all  the  options,  using  S.O.S  to  help  with  lesson  planning  (85%;  163)  was  the  highest  rated 
response.  Stimulating  ideas  (80%;  152),  providing  background  information  (67%;  127),  and  allowing  practitioners 
to  compare  strategies  provided  with  current  practice  (65%;  124)  were  also  highly  rated  uses.  Fifteen  (8%)  indicated 
“other”  (e.g.  professional  development,  use  in  education  courses).  Information  was  also  gathered  on  information 
skills  and  subskills  to  include  in  the  system.  Respondents  overwhelmingly  agreed  that  the  inclusion  of  lesson  plans 
would  be  a highly  motivating  component  of  S.O.S.  and  that  standards  tie-ins  would  also  be  important. 


Questionnaire  to  Selected  College  Educators  and  Leaders  in  the  Field 

The  purpose  of  the  questionnaire  was  to  gather  consensus  on  the  appropriate  scope  of  content  to  include  in 
the  S.O.S.  tool  and  to  gather  feedback  on  the  proposed  features  of  the  tool.  A group  of  1 1 individuals  was  initially 
targeted  for  participation.  Eight  of  1 1 responses  were  received.  The  results  of  this  survey  were  clarifications  of 
several  skills  and  the  addition  of  several  subskills  to  search  options  for  the  tool  in  Phase  II.  This  group  provided 
useful  suggestions  for  publicizing  and  motivating  educators  to  use  S.O.S.  Respondents  also  provided  a number  of 
suggestions  for  S.O.S.  features  which  included: 

• A listserv  to  allow  people  to  trade  comments  and  make  suggestions  as  well  as  contribute  lessons 

• A way  to  add  comments/reflections  to  the  site  which  would  then  be  available  to  others  to  peruse 

• Printable  lesson  plans 

• Links  to  content  standards 

• Lessons  that  reflect  collaboration  with  classroom  teachers  in  specific  subject  areas 

• Printed  transcripts  of  video/audio  clips 
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Input  Evaluation 


Early  on,  an  input  evaluation  was  conducted  in-house  and  extended  to  several  outside  consultations  with 
technology  persons  who  frequently  work  with  Creative  Media  Solutions.  Technical  specifications  for  software 
development  were  considered  and  a preliminary  web-based  data  entry  structure  was  planned  using  an  SQL 
client/server  environment.  Many  video  issues  were  also  considered  such  as  what  features  were  accessible  by  the 
majority  of  educators  that  would  be  testing  our  product  and  what  long-range  features  could  be  anticipated  for  the 
ultimate  release  of  the  product.  Decisions  about  frame  size,  delivery  format,  and  compression  algorithms  were  made 
after  testing  factors  such  as  download  time  of  various  options  with  typical  users’  computer  systems.  These  decisions 
are  discussed  further  in  the  sections  below. 

Video  Production  and  Acquisition 

Only  original  materials  produced  by  Creative  Media  Solutions  were  planned  for  the  prototype.  Eighteen 
videos  were  included  in  the  prototype  ranging  in  length  from  approximately  :25  to  4:30  minutes.  Teachers  and 
librarians  from  4 Northeast  states  participated:  New  York,  Pennsylvania,  Massachusetts,  and  New  Hampshire.  In 
addition  to  videos  produced  entirely  by  the  company,  two  videos  were  compiled  from  still  photos  sent  in  by  a school 
in  a Boston,  Massachusetts  suburb.  Photos  covering  a six-week  unit  by  a teacher  in  New  Hampshire  were  also 
compiled  into  a multimedia  PowerPoint  presentation. 

The  style  of  production  is  similar  to  ENG  production  (electronic  news  gathering)  in  that  the  production 
team  consists  of  the  producer  and  videographer  and  sometimes  an  assistant.  This  is  necessary  because  what  happens 
in  the  course  of  a teaching  episode  is  not  entirely  predictable;  sometimes  the  camera  is  on  the  tripod  and  other  times 
it  is  simply  handheld.  The  production  team  attempts  to  be  as  non -intrusive  as  possible  during  the  videotaping.  There 
is  generally  little  discussion  ahead  of  time  as  to  where  in  the  library  media  center  the  librarian  or  students  will  be  at 
any  one  time.  The  videographer  simply  documents  the  events. 

With  several  exceptions,  the  production  team  was  not  allowed  to  videotape  children’s  faces.  This  posed  a 
challenge  but  did  not  compromise  the  quality  of  production. 

The  library  media  specialist  was  generally  interviewed  after  the  teaching  episode.  By  conducting  the  on- 
camera  interview  afterwards,  the  producer/interviewer  could  frame  her  questions  in  the  context  of  the  actual  lesson 
and  the  reactions  from  children  she  observed. 

Each  raw  videotape  (unedited  field  tape)  was  reviewed  and  shots  or  soundbites  (also  referred  to  as  “clips”) 
logged  using  timecode  that  would  help  to  demonstrate  a particular  strategy.  Scripts  were  written  to  guide  the  editing 
process  providing  detail  on  all  shots  selected  including  their  sequence  and  sometimes  duration.  Interview  soundbites 
were  included  and  narration  was  written  where  needed.  Any  transitions  (e.g.,  dissolves,  wipes,  etc.),  music/effects, 
and  text  were  also  noted  in  the  script.  Voiceovers  were  recorded  when  appropriate. 

The  selected  clips  and  voiceovers  were  then  captured  into  the  computer  using  Firewire  and  digital  capturing 
software  made  by  Pinnacle.  Next,  the  captured  clips  were  brought  into  editing  software  such  as  Media  100  or  Adobe 
Premiere  (for  this  example,  Adobe  Premiere)  and  laid  out  on  a timeline.  Titles  and  identifying  text  were  created 
including  a summary  of  the  strategy  used.  Voiceovers  were  added  when  indicated.  Transitions  were  created. 

The  timelines  with  all  their  elements  were  rendered  as  complete  movies  in  AVI  uncompressed  format  at 
360  X 240  pixels  down  from  their  original  size  of  720  X 480  pixels.  As  planned,  the  finished  AVI  movies  were 
taken  into  another  industry  standard  program  called  Media  Cleaner  Pro  for  compression  and  . reduced  to  a 240  X 
1 80  pixel  frame  size  and  rendered  with  Sorenson  compression  at  15  frames  per  second  with  audio  at  22  kh. 

The  final  step  was  adding  the  video  to  the  video  demonstration  database  to  be  linked  to  strategies  and 
lesson  plans. 

Two  professional  looking  albeit  simple  (in  terms  of  video  production  techniques)  videos  that  the 
Progressive  Feedback  Panel  felt  represented  an  acceptable  alternative  to  only  motion  videos  in  the  database  were 
created  from  still  photos  sent  in  by  a teacher  in  Massachusetts,  proving  to  be  a viable  and  cost-effective  alternative 
to  only  motion  videos. 

A number  of  implications  for  future  acquisitions  of  video  content  were  determined.  They  include  but  are 
not  limited  to  loaning  library  media  specialist/teacher  teams  digital  video  or  digital  still  cameras  for  2 - 3 months  at  a 
time  to  capture  teaching action  clips.  The  equipment  would  be  bicycled  to  different  areas  to  expand  geographic 
coverage.  The  alternative  production  method,  mentioned  earlier,  of  using  still  photos  to  create  a video  would  also 
open  up  participation  to  many  individuals.  All  footage  and  stills  would  be  sent  to  Creative  Media  Solutions  for  post- 
production including  adding  voiceovers,  titles,  etc.,  helping  to  maintain  a consistent  look.  Online  training  resources 
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to  be  developed  in  Phase  11  could  help  prepare  teams  for  acquiring  video  and  preparing  their  work  for  submission  to 
the  S.O.S.  project. 

Development  of  Data  Structure  and  Query  System  for  Initial  Prototype 

Phase  1 technical  development  goal  was  to  collect  information  from  a variety  of  sources,  provide  a 
mechanism  to  enter  or  query  data,  and  deliver  a wide  variety  of  media  as  efficiently  as  possible  to  a diverse  market. 
This  solution  was  then  integrated  into  a comprehensive  Web  site. 


Data  Structure . A relational  database  was  custom  developed  to  insure  all  facets  of  the  project  goals  could 
be  realized.  A number  of  changes  and  additions  were  added  to  the  database  over  the  development  cycle.  Bringing 
the  vision  of  the  academic  information  literacy  professional  and  translating  into  technical  development  introduced  a 
series  of  challenges  that  were  overcome.  The  solution  consists  of  over  thirty  data  tables,  hundreds  of  data 
relationships  and  a client  server  interface  as  well  as  a Web  interface.  The  final  product  has  almost  no  artificial  limits 
to  data  entry.  Strategies  can  be  related  to  lesson  plans,  lesson  plans  can  be  related  to  strategies  and  an  unlimited 
number  of  resources  can  be  attached  to  either.  Supported  resources  include  but  not  necessarily  limited  to; 

QuickTime  video.  Power  point,  text,  QuickTime  audio,  pictures,  and  Web  links.  Special  care  was  taken  to  allow 
content  to  be  cross-referenced. 

Another  important  aspect  to  consider  was  data  entry.  The  user  interface  was  designed  to  allow  professbnal 
staff  to  quickly  learn  the  necessary  skills  and  offer  automated  entry  functions  when  possible.  Feedback  on 
navigation,  view,  data  required  on  screen  and  the  complexities  of  identifying  all  the  associated  links  were  all 
considered.  This  is  an  evolving  process,  but  the  current  solution  has  met  the  data  entry  requirements  to  this  point. 

Query.  Query  screens  fall  into  two  categories.  Within  the  client  server  environment,  queries  have  relatively 
unlimited  options  and  occur  very  quickly.  The  query  is  an  integrated  function  of  the  development  software.  As  the 
complex  data  relationships  that  empower  the  information  are  ported  to  the  web,  significant  slowdowns  occurred. 

This  was  an  anticipated  result  and  a number  of  solutions  are  possible.  The  initial  investment  was  made  in 
understanding  the  data  flow  and  allowing  for  the  data  to  be  entered  correctly.  As  indicated  above,  this  led  to  a 
number  of  adjustments.  Optimization  of  code  occurs  in  logical  sequence,  and  every  adjustment  has  a potential 
ripple  effect  throughout  the  code  that  is  not  always  possible  to  predict.  The  next  phase  of  development  will  focus  on 
additional  data  indexing;  relationship  based  queries,  and  a continued  focus  on  utilizing  the  best  technology  tools 
available  at  the  time  of  product  delivery.  During  Phase  1,  the  focus  was  on  utilizing  Active  Server  Pages  with  SQL 
queries  embedded  into  the  HTML  page  and  CHTML  tags  with  proprietary  software.  We  feel  that  all  issues  of  query 
speed  from  a Web  centric  view  will  be  resolved  transparent  to  the  user.  It  is  also  anticipated  that  more  features  will 
be  added  to  the  data  structure  that  will  impact  the  solution  as  development  continues. 

Data  delivery.  Data  delivery  from  the  Web  site  introduced  a number  of  variables  inherent  in  the  diverse 
world  of  Internet  users.  The  expectation  was  data  would  be  accessed  by  a current  Web  browser,  would  require  Flash 
and  Quicktime  plug-ins,  and  the  user  would  have  dedicated  Internet  access.  This  expectation  was  not  what 
occurred.  This  solution  requires  the  ability  to  develop  and  distribute  complex  pieces  of  information.  Multiple 
versions  of  multiple  browsers  demand  constant  day  to  day  troubleshooting.  From  our  research,  we  can  see  just  how 
problematic  this  can  be.  For  example,  Netscape,  particularly  in  earlier  versions,  did  not  fare  as  well  as  Internet 
Explorer  with  our  prototype.  There  is  a wide  array  of  variables  unique  to  each  browser  and  specific  version.  The 
problem  then  becomes  “which  version  do  we  adjust  code  to  support”?  As  mentioned  previously  the  effect  of 
software  adjustments  at  one  level  can  and  will  affect  results  in  other  levels.  Some  panel  members  using  both  old 
browsers  and  old  “systems”  required  technical  support  from  our  staff  in  order  to  complete  the  research.  The 
compatibility  problem  is  further  complicated  by  the  fact  that  competition  has  the  various  browsers  introducing 
newer,  better,  faster  features  but  that  the  different  browsers  (while  trying  to  maintain  competitiveness)  still  do  so  at 
different  paces. 

A decision  that  faces  every  developer  is  whether  to  adjust  code  to  the  lowest  common  denominator 
approach  or  choose  a solution  somewhere  between  the  lowest  and  the  highest  common  denominator.  This  project 
demands  fairly  complex  searches  on  multiple  variables.  It  includes  videos,  some  of  wh  ich  can  be  up  to  four  or  five 
minutes  in  length.  What  our  research  is  indicating,  from  a technical  perspective,  is  that  the  lowest  common 
denominator  approach  will  not  be  the  best  one  for  this  project.  While  some  web  databases  do  just  fine  with  this 
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approach,  they  do  not  have  the  many  additional  variables  to  deal  with  as  one  such  as  the  S.O.S.  project  with  its 
multimedia  content. 

Judging  from  the  results  provided  by  users,  the  complexity  of  supporting  multiple  browsers  limits  delivery 
options  to  an  unacceptable  level.  Internet  Explorer  version  5 or  greater  has  proven  to  be  the  most  reliable  with  this 
project.  Current  versions  of  Internet  Explorer  and  Netscape  will  likely  be  recommended  as  viewing  engines. 

Perhaps,  by  the  time  this  resource  is  available,  the  browsers  will  have  achieved  an  overall  consistency  that 
eliminates  this  issue.  For  now,  the  delivery  goal  will  dictate  browser  selection  and  support.  Browsers  are  typically 
free,  are  upgraded  regularly,  and  have  many  settings  and  options  to  enhance  web  experiences. 

Will  our  decision  eliminate  many  potential  users?  The  question  is  not  whether  the  site  can  be  viewed,  but 
more  precisely  can  the  site  be  experienced  to  the  full  potential.  It  is  not  unusual  today  to  see  messages  such  as  “best 
viewed  by...”  While  we  may  lose  some  users,  we  will  try  to  minimize  that  loss  by  making  the  necessary  software  or 
upgrades  easily  available  to  the  user.  Furthermore,  most  current  computer  operating  systems  include  versions  of 
both  Netscape  and  Internet  Explorer.  Navigating  the  site  in  the  preferred  browser  should  be  less  of  a problem. 
Information  reduces  anxiety.  Providing  information  to  the  user  about  technical  needs  for  an  optimum  experience 
will  also  reduce  possible  frustration  when  problems  are  encountered  but  users  don’t  know  why. 

Results  of  Focus  Group 

The  Focus  Group  consisted  of  1 1 pre -service  and  in-service  educators  (potential  future  users  of  S.O.S.) 
most  of  whom  were  library  media  specialists.  They  were  from  New  York  State,  North  Carolina,  and  Virginia. 
Because  of  the  distributed  geographic  location  of  focus  group  participants,  all  focus  group  sessions  were  conducted 
online  using  the  Research  Management  Site  and  WebCT’s  asynchronous  discussion  feature  rather  than  the 
traditional  face-to-face  method.  The  Focus  Group  provided  valuable  feedback  on  important  issues  that  related  to  the 
scope  of  content,  inclusion  of  standards,  value  of  lesson  plans,  variety  of  videos,  amount  of  information  presented  in 
videos,  links,  and  a number  of  other  issues.  At  this  point  in  the  research,  only  content  and  curriculum  related  issues 
were  explored.  Interface  and  design  issues  were  not  addressed  in  the  Focus  Group.  They  reviewed  content  issues  in 
the  context  of  a very  plain  and  simple  interface  on  a beige  background.  This  group  also  provided  input  that  resulted 
in  the  adoption  of  the  national  information  literacy  standards  put  forth  \n  Information  Power:  Building  Partnerships 
for  Learning  published  by  the  American  Association  of  School  Librarians  (AASL)  and  the  Association  of 
Educational  Communications  and  Technology  (AECT)  in  1998,  as  the  basis  of  the  standards  search.  Quality  control 
of  content  was  also  considered  an  important  attribute  of  the  future  tool.  The  Focus  Group  provided  a number  of 
possible  ways  to  insure  quality  control  of  content  including  a quality  assurance  committee,  expansion  of  the  online 
feedback  mechanism,  and  the  development  of  an  evaluation  tool.  The  consensus  seemed  to  be  that  anything 
uploaded  to  the  S.O.S.  site  should  be  considered  a quality  product. 

Evaluation  with  Progressive  Feedback  Panel 

Based  on  feedback  and  recommendations  from  the  Focus  Group,  the  development  team  made 
modifications  and  additions  to  the  initial  prototype  for  testing  with  another  group  of  educators  described  below.  In 
this  iteration,  videos  were  linked  to  the  strategy  or  lesson  plan  generator  in  which  search  variables  included  specific 
situation  (grade  and  context)  and  outcome  (information  skills  and  subskills)  variables.  The  results  of  such  a search 
are  strategies  or  lesson  plans  which  have  links  to  a video/multimedia  database.  We  also  added  a video  “quick 
search”  where  a user  can  search  on  a video  topic  using  only  a topic  keyword. 
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A comprehensive 

questionnaire  was  developed  by  the 
principal  investigator  and  senior  project 
consultant  that  would  elicit  feedback  in 
two  main  areas  1)  Searching 
functionality  across  strategies  and 
videos,  lesson  plans,  topics,  and 
standards,  and  2)  Interface  and  design. 

This  site  was  then  presented  to  the 
Progressive  Feedback  Panel  who  helped 
us  evaluate  the  2"^  iteration  of  the 
proof-of-concept  prototype  in  the  above 
areas  and  on  specific  issues  that 
included:  ease  of  use,  features  that 
might  attract  educators,  quality  of 
content,  technical  quality,  menu 
buttons,  color  appeal,  etc. 

(Whereas  the  Focus  Group  was  Figure  I:  Screen  capture  showing  top  half  of  homepage  illustrates 

concerned  with  content  issues  and  interface  design  of  2nd  iteration  prototype 

functionality,  this  group  was  also  concerned  with  interface  issues.) 

The  Panel  consisted  of  several  members  from  the  original  Focus  Group  plus  new  members.  This 
configuration  was  chosen  to  provide  a balance  between  continuity  of  feedback  and  fresh  input  from  persons  not 
previously  connected  with  the  project.  Eight  library  media  specialists,  2 classroom  teachers,  and  1 district-wide 
technology  coordinator  comprised  the  group.  They  were  given  instructions  for  reviewing  the  site  individually,  and 
subsequently  filled  out  a comprehensive  questionnaire. 

In  addition  to  rating  various  aspects  of  the  tool  on  a scale,  participants  provided  many  useful  suggestions 
via  open-ended  comments.  The  Panel  gave  S.O.S.  high  scores  (4.1;  n = 11)  on  the  intuitiveness  of  searching  with  the 
Strategy  and  Lesson  Generators  (1  = not  intuitive;  5 = very  intuitive).  They  gave  an  even  higher  score  (4.4;  n = 1 1) 
on  the  usefulness  of  the  “related  resources’’  which  included  videos,  lesson  plans,  graphics,  etc.  Such  resources  are 
intended  to  help  clarify,  enrich,  or  reinforce  the  teaching  strategies  suggested  by  the  strategy  generator. 

In  terms  of  quality  of  video,  most  comments  reflected  high  enthusiasm  for  the  videos.  “This  is  some  of  the 
highest  quality  [video]  I have  seen  on  the  web! ! !”  wrote  one  respondent.  Another  wrote:  “It  is  always  valuable  to 
hear  from  colleagues.  When  I see  the  enthusiasm  of  the  person  in  the  video,  it  makes  me  more  likely  to  try  their 
strategy.  The  videos  are  a valuable  part  of  S.O.S.”  There  were  some  comments,  however,  that  indicated  concern 
about  the  speed  of  access  of  the  videos.  Some  of  the  longer  videos  took  an  extremely  long  time  to  load  if  the  user 
did  not  have  a direct  connection  to  the  Internet. 
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Participants  commented  that  the 
voiceovers  which  were  added  to  some  of  the 
videos  were  of  professional  quality,  helped  to 
clarify  points  made  by  the  speaker  (educator),  and 
were  effective  in  providing  transitions. 

After  seeing  two  examples  demonstrating 
an  alternative  video  production  technique,  almost 
all  Panel  members  agreed  that  allowing  educators 
to  submit  photos  that  could  be  edited  into  a video 
with  a voiceover  added  would  provide  a valuable 
alternative  to  all  motion  video.  An  important  point 
was  made  that  many  educators  still  feel  more 
comfortable  with  a still  camera  than  a video 
camera.  Finally,  the  opinions  were  mixed  on  the 
value  of  including  video  transcripts  but  one 
important  comment  was  made  that  transcripts  may 
be  useful  for  the  hearing  impaired. 

Some  Panel  members  felt  the  current  size 
of  the  video  frame  used  in  the  prototype  was 
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adequate;  others  felt  it  should  be  twice  as  large.  Figure  2 illustrates  the  current  frame  size  in  relation  to  the  page.  The 
user  arrives  at  the  screen  in  Figure  2 after  selecting  a particular  video  from  the  list  of  Related  Resources  (to  the 
strategy  or  lesson  plan)  which  includes  brief  descriptions  of  the  available  videos.  As  technology  advances  allow,  we 
anticipate  increasing  the  size  of  the  video  at  least  two -fold.  We  selected  the  current  size  in  order  to  conduct  research 
with  users  who  would  not  have  been  able  to  accommodate  the  larger  video  size  at  this  time  due  to  unreasonable 
loading  times. 

Comments  were  solicited  on  the  usefulness  of  the  search  function  for  information  literacy  standards.  This 
search  is  based  on  the  national  standards  as  presented  \r\  Information  Power,  Building  Partnerships  For  Learning 
(1998).  Responses  were  generally  positive  and  included:  “This  is  a good  search.  Often  we  are  looking  for  a specific 
idea  to  teach  a standard  required  in  our  curriculum  and  this  will  be  a good  resource  for  it.”  Another  wrote:  “This  will 
be  handy  when  my  co-librarian  and  1 are  in  the  midst  of  our  curriculum  and  benchmarks  we  are  writing  based  on  the 
ILS  [information  literacy  standards]  from  Information  Power.  Great  examples.” 

The  user  can  select  one  of  the  broader  standards  or  narrow  the  search  to  include  only  those  lessons  that 
relate  to  a specific  indicator  of  the  standard.  Future  development  will  broaden  the  search  to  include  all  video  and 
related  resources  as  well. 

Although  not  implemented  in  the  prototype,  participants  were  asked  to  assess  how  useful  it  would  be  to 
search  S.O.S.  by  nationally  recognized  content  area  standards  such  as  those  compiled  by  the  Mid -Continent 
Research  for  Education  and  Learning  (McREL)  in  addition  to  searching  by  information  literacy  standards.  On  a 
scale  of  1 (not  useful)  to  5 (very  useful),  this  item  received  a 3.7.  One  Panel  member  wrote:  “1  believe  the  national 
standards  are  adequate.  Users  can  interpolate  where  the  standard  they  are  looking  for  is  within  the  national 
standards.” 

Panel  members  offered  suggestions  for  improving  the  Topic  Search  function  including  providing  drop- 
down menus  of  topics,  clarifying  directions,  providing  the  ability  to  browse  available  topics,  and  adding  a help 
menu.  Positive  comments  included  those  who  liked  the  “quickness”  and  “ease”  of  the  search  in  addition  to  its 
potential  helpfulness  in  searching  for  ideas  to  integrate  into  classroom  instruction. 

From  a design  and  navigational  perspective,  S.O.S.  was  well  received.  It  had  received  a complete  overhaul 
from  the  simple  design  and  layout  of  the  initial  prototype  beta  tested  by  the  Focus  Group.  Panel  members 
commented  positively  on  the  colors,  banner,  the  appealing  look,  and  layout.  One  member  wrote:  “1  love  the  design 
and  colors  used  in  the  site  ...  the  way  the  menu  buttons  move  is  great!  1 like  the  logo  of  the  earth  and  the  banner. 
Information  is  shared  all  over  the  world.”  That  is  the  message  that  the  banner  was  meant  to  reflect  and  it  seemed  to 
be  readily  recognized. 

Finally,  comments  from  both  the  Focus  Group  and  the  Faculty/Leaders  Questionnaire  had  previously 
mentioned  the  desirability  of  including  a section  where  visitors  could  find  news,  features,  special  videos,  or  other 
information.  This  was  included  in  the  2*^^  prototype  iteration  which  the  Progressive  Feedback  Panel  reviewed.  The 
“Spotlight”  section  of  the  site  was  enthusiastically  accepted.  Responses  were  almost  universally  positive.  One 
person  commented  on  its  potential  value  for  encouraging  collaboration  which  is  one  of  the  project’s  underlying 
goals.  *ln  the  Spotlight  ” for  this  prototype  included  a feature  story  on  a teacher  from  New  Hampshire  who  cleverly 
wove  eight  information  skills  into  a 6 -week  Social  Studies  unit  on  communities.  In  fact,  her  students  actually  built  a 
play  community  in  the  woods  behind  their  school  as  a way  of  learning  the  concepts.  Also  the  Spotlight*' w2ls  a 
library  media  specialist  talking  about  ways  to  foster  collaboration  between  classroom  teachers  and  library  media 
specialists.  This  was  presented  as  a video  interview.  This  component  of  S.O.S.  will  help  the  site  maintain  a current 
feel  and  may  be  expanded  in  future  development  efforts. 


From  Phase  1 to  Current  Development 


Creative  Media  Solutions,  Inc.  is  continuing  to  develop  5. 0.5.  ybr  Information  Literacy  through  a strategic 
alliance  with  Grant  Systems,  Inc.,  as  new  sources  of  funding  are  explored.  Since  the  authors  have  always  planned  on 
making  this  resource  freely  available  to  educators,  creative  means  of  sustaining  the  project  are  necessary. 
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Already,  a number  of 
improvements  have  been  accomplished  as 
a result  of  the  input  from  the  Phase  I 
research.  Search  screens  are  now  much 
more  streamlined  and  require  no 
scrolling.  After  selecting  grade  levels, 
subject  area,  and  type  of  search  (see 
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Figure  3),  for  example,  the  user  can  select  both  information  skill(s)  and  subskill(s)  in  one  easy  step.  The  color 
scheme,  considered  pleasant  (blue  and  gold  tones)  by  the  Progressive  Feedback  Panel,  has  been  retained  but  the 
screen  design  has  been  simplified.  An  online  submission  template  which  will  allow  educators  to  upload  media  such 
as  digital  stills  and  PowerPoint  presentations  along  with  their  lesson  plans  is  currently  being  developed.  As  soon  as 
completed,  library  media  specialists  and  classroom  teachers  will  be  recruited  to  submit  materials  to  populate  the 
database.  Once  the  database  has  sufficient  materials  for  meaningful  queries,  it  will  be  made  available  to  educators  at 
large. 

Evaluation  of  all  materials  submitted  to  the  database  will  be  an  important  aspect  of  S.O.S.  Before  being 
made  available  online,  new  submissions  will  be  reviewed  by  a two-person  team  of  evaluators  using  an  evaluation 
rubric  similar  to  that  used  in  the  U.S.  Department  of  Education’s  Gateway  to  Educational  Materials  (GEM)  project. 
A Web-based  interface  for  evaluators  to  review  and  score  submissions  is  currently  being  developed. 

Finally,  the  S.O.S.  for  Information  Literacy  information  system  will  be  expanded  to  include  middle  school 

grades. 

Summary 

The  successful  completion  of  all  technical  objectives  for  Phase  I not  only  demonstrated  the  project’s 
feasibility  but  also  provided  a strong  framework  for  the  successful  continuation  of  the  research  and  development 
effort.  Most  importantly,  there  was  a high  degree  of  agreement  by  educators  in  each  of  the  research  components 
conducted  that  S.O.S.  for  Information  Literacy  would  be  a valuable  and  needed  addition  to  improving  the  teaching 
of  information  literacy  at  the  elementary  and  middle  school  grade  levels. 


599 


Online  Resources  for  Teaching  Widely-Used  Secondary  School  Texts 


Thomas  J.  Reinartz 
Lauren  A.  Liang 
University  of  Minnesota 

Gregory  C.  Sales 

Seward  Learning  Systems,  Inc. 


Scaffolded  Reading  Experiences 

The  scaffolded-reading-experience  (Graves  & Graves,  1994,  Tierney  & Readence,  2000;  Watts  & 
Rothenberg,  1997)  instructional  framework  provides  both  lesson  designers  and  lesson  users  with  a flexible  structure 
for  text -specific  lessons.  This  framework  presents  a set  of  options  for  creating  coordinated  sets  of  pre-,  during-,  and 
post-reading  activities  (Figure  1).  While  the  framework  is  appropriate  for  various  combinations  of  texts  and 
purposes,  the  specific  activities  employed  in  any  particular  situation  will  differ  greatly  depending  on  the  difficulty  of 
the  reading  selection  and  the  purposes  for  which  students  are  reading. 

Figure  1 


Method  Used  in  Constructing  the  Website  and  Scaffolded  Reading  Experiences  The  website  (Figure  2) 
was  constructed  by  professional  design  and  development  staff  at  Seward  Learning  Systems,  Inc.  (SLS),  using 
standard  web  development  tools  and  procedures.  A designer,  a subject-matter  expert,  a graphic  artist,  and  a 
programmer  worked  as  a team  to  develop  the  website.  The  scaffolded  reading  experiences  (SREs)  were  created  by 
two  secondary  teachers;  university  faculty  members  in  literature,  social  studies,  and  reading  education;  and  a 
research  assistant.  This  team  used  the  SRE  instructional  framework  as  a model  for  the  instruction. 


Figure  2 
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Methods  of  Evaluating  the  Website  and  SREs 

The  website  and  the  SREs  themselves  were  evaluated  by  10  high  school  English  teachers  and  10  high 
school  social  teachers.  Half  of  these  teachers  were  observed  and  interviewed  by  the  project  staff  as  they  worked  on 
the  website  and  completed  a written  survey.  The  other  half  of  the  teachers  completed  only  the  written  survey. 

Two  SREs  were  evaluated  in  high  school  classrooms.  In  one  school,  three  10th  grade  classes  read  and 
studied  The  Great  Gatsby  over  a three-week  period.  Because  of  a last-minute  school  decision  that  different  classes 
not  receive  different  activities,  all  three  classes  used  the  SRE.  All  three  classes  also  received  pretests  and  posttests 
on  knowledge  about  Gatsby  and  higher  order  thinking  skills  as  well  as  a posttest  in  which  they  demonstrated  higher 
order  thinking  with  Gatsby. 

In  the  other  school,  two  1 2th  grade  classes  read  and  studied //aw/ef  over  a three-week  period.  Here,  we 
conducted  a mini  experiment,  with  one  class  receiving  the  SRE  and  the  other  class  receiving  the  teacher's  typical 
instruction.  These  two  classes  received  pretests  and  posttests  paralleling  those  given  to  the  classes  that  read  Gatsby. 

Data  Sources  for  Evaluating  the  Website  and  SREs 

The  data  for  evaluating  the  website  and  SREs  were  the  field  notes  taken  by  the  researchers  as  teacher- 
reviewers  were  exploring  the  website  and  SREs  and  the  results  of  the  written  survey.  The  data  for  evaluating  the 
effectiveness  of  the  SREs  were  the  results  of  the  pretests  and  posttests  administered  to  students  who  read  Gatsby  and 
those  who  read  Hamlet. 

Results  with  Respect  to  the  Website  and  SREs  Constructed 

The  website  included  five  major  pages: 

• About  SREs  - approximately  1000  words  in  length.  It  explains  what  SREs  are,  the  SRE 

framework,  the  possible  components  of  an  SRE,  and  the  content  and  organization  of  each  SRE. 
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• About  Higher-Order  Reading  and  Comprehension  Skills  - approximately  1300  words  in 
length.  It  includes  a definition  of  higher  order  thinking  (Resnick,  1987),  a description  of  analytical, 
creative,  and  practical  thinking  (Sternberg,  1 996)  and  a discussion  of  teaching  for  understanding 
(Perkins,  1998,1992).  This  section  also  includes  a link  to  the  Center  for  Critical  Thinking 
(http://www.criticalthinking.org/default.html),  which  includes  an  extensive  discussion  of  critical 
thinking  and  definitions  and  examples  of  the  types  of  higher  order  thinking  skills  emphasized  in 
the  SREs 

• Search  - allows  users  to  search  the  site  by  Author,  Title,  and  Topic  (all  open  fields),  as  well  as  by 
Type  of  Text  (Fiction,  Non-Fiction,  Poetry,  Drama),  Ethnic  Group  Featured  (Black,  Asian  or 
Pacific  Islander,  American  Indian  or  Alaskan  Native,  Hispanic,  White),  and  Grades  (9-10,  11-12, 
13-14). 

• Contact  Us  - 

• Help- 

During  the  Phase  1 development  the  site  was  populated  with  the  following  six  SREs: 

• James  Baldwin's  "Sonny's  Blues,"  (1948) 

• William  Faulkner's  "A  Rose  for  Emily,"  (1930) 

• F.  Scott  Fitzgerald's  The  Great  Gatsby,  (1925) 

• Russell  Freedman's  Eleanor  Roosevelt , ( 1 994) 

• Shakespeare’s //a/w/e/,(  c.  1600) 

• United  Nations'  Universal  Declaration  of  Human  Rights,  (1948) 

Each  SRE  was  comprised  of  the  following  component: 

• Introduction 

• Table  of  Contents 

• Objectives 

• Higher  Order  Reading  and  Comprehension  Skills  Emphasized 

• Chronological  List  of  Activities 

• Detailed  Description  of  Activities 

• Student  Materials 

• Resources 

Thus,  once  a user  is  familiar  with  the  organization  of  one  SRE,  she  is  familiar  with  the  organization  of  all 

of  them. 

Evaluation  Results 

Observations  and  interviews  with  the  10  English  teachers  showed  a very  favorable  attitude  toward  the  site. 
The  most  valued  features  were  (1)  that  teachers  could  modify  the  material,  (2)  the  inclusion  of  student  materials,  and 
(3)  the  teacher  resources,  which  included  online  resources.  Teachers  also  commented  that  the  search  capabilities 
would  be  very  useful  once  the  website  included  a large  number  of  SREs.  Observations  and  interviews  also  revealed 
that  the  site  generally  functioned  well. 

The  most  consistent  responses  on  the  survey  were  that  (1)  shorter  descriptions  of  the  website  would  be 
desirable,  (2)  more  graphics  on  both  the  website  and  the  SREs  would  be  desirable,  and  (3)  having  as  many  teacher 
resources  as  possible  would  be  useful. 

Tests  of  the  1 0th  graders  comprehension  of  Gatsby  showed  that  on  the  pretest  only  16  percent  of  the 
students  had  any  knowledge  about  the  novel  and  that  no  students  could  name  any  themes  from  the  novel.  On  the 
posttest  96  percent  of  the  students  gave  an  acceptable  or  strong  summary  of  the  novel,  and  students  named  an 
average  of  2 themes  from  the  novel. 

Tests  of  the  10th  graders  knowledge  of  higher  order  skills  showed  that  on  the  pretest  only  7 percent  of  the 
students  had  any  knowledge  about  higher  order  thinking  and  no  students  could  describe  any  of  the  three  main  types 
of  higher  order  thinking.  On  the  posttest,  84  percent  of  the  students  gave  an  acceptable  or  strong  definition  of  higher 
order  thinking,  and  students  named  an  average  of  2 of  the  three  main  types  of  higher  order  thinking.  Additionally,  on 
the  posttest  72  percent  of  the  students  were  able  to  use  the  higher  order  skills  they  had  been  taught  to  interpret  a 
previously  unread  passage  from  the  novel.  (See  Table  1.) 
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Item 

Pre-test 

Post-test 

Summarize  the  plot  in  three  paragraphs  or  less. 

0.0 

2.3 

Identify  as  many  significant  themes  in  the  novel  as  you  are  familiar  with. 

0 

1.7 

Define  Higher  Level  Thinking. 

0 

2.3 

List  the  three  main  types  of  Higher  Level  Thinking. 

0 

1.7 

Define  Analytical  Thinking. 

.1 

2.3 

List  three  Analytical  Thinking  Skills. 

0 

1.9 

Identify  the  section  on  pages  188-189  in  which  Nick  makes  these  final  comments  by  giving  the  beginning  4-5 
words  and  the  ending  4-5  words. 

- 

2.2 

Select  a sentence  of  phrase  from  that  section  that  you  find  particularly  informative  about  Gatsby.  (2x) 

- 

3.7*  ■ 

Explain  what  this  sentence  of  phrase  tells  us  about  him.  (2x) 

- 

2.8* 

Table  1 

Educational  Importance  of  the  Study 

Everything  we  have  learned  thus  far  supports  the  feasibility  of  putting  these  online  resources  on  the  Web  so 
that  at  least  some  of  the  time  teachers  can  begin  with  quality  basic  lesson  plans  and  spend  their  very  limited  time 
modifying,  improving,  and  tailoring  the  lessons  to  their  students  rather  than  doing  basic  lesson  design.  Doing  so 
should  result  in  a huge  saving  of  time  and  in  better  lessons. 
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Introduction 


Advances  in  technology  offer  the  possibility  of  new  methods  for  delivering  instruction.  Learning  via  the 
Internet  is  being  heralded  by  many  as  the  new  pedagogical  model  for  training.  Recent  issues  of  training,  computer, 
and  management  magazines  all  suggest  that  web-based  training  (WBT)  is  the  best  way  to  reach  geographically 
dispersed  employees  quickly  and  at  low  costs.  Some  articles  offer  examples  of  companies  that  have  successfully 
implemented  WBT.  However,  the  same  articles  that  tout  WBT  as  the  next  wave  in  the  instructional  technology 
revolution  often  fail  to  provide  the  reader  with  specific,  instructionally  sound  steps  for  implementing  WBT  in  a 
corporate  setting. 

Furthermore,  many  companies  eager  to  embrace  WBT  have  focused  their  attention  on  the  features  of  the 
new  technology.  The  trainers  and/or  instructional  designers  who  fall  into  this  trap  expect  WBT’s  features  alone  to 
provide  effective  instruction  (Alexander,  1995).  As  a result,  the  advantages  that  WBT  offers  are  often 
overshadowed  by  poor  design  and  cumbersome  navigation  (Cohen  & Rustad;  1998;  Filipczak,  1997b;  Henke,  1997; 
Strandberg,  1999).  Learners  are  set  adrift  in  cyberspace  with  little  or  no  instruction. 

The  purpose  of  this  paper  is  to  explore  the  many  unique  characteristics  of  WBT  and  to  discuss  important 
issues  related  to  the  use  of  WBT  in  a corporate  setting.  These  issues  include  the  cost  effectiveness  considerations  of 
using  WBT,  the  design,  implementation  and  delivery  of  WBT,  and  the  impact  that  WBT  has  on  both  learners  and 
trainers.  Examples  of  how  WBT  is  being  used  in  American  corporations  will  be  introduced,  too. 


WBT  refers  to  the  communication  of  information  over  the  World  Wide  Web  (WWW)  with  the  intent  of 
providing  instruction  (Kurtus,  1997).  WBT  makes  it  possible  for  students,  the  instructor,  the  course  content  and,  at 
times,  professional  groups  and  subject  matter  experts,  to  be  in  different  locations  at  possibly  different  times  and  yet 
still  be  brought  together  through  the  use  of  on-line  technology  (Dede,  1996;  Heckman  & Owens,  1996;  Mclntrye, 
1996;  Saltzberg  & Polyson,  1996;  Smith,  Tyler  & Benscoter,  1999;  Stenerson,  1998;  Williams,  1998).  WBT  allows 
learners  who  cannot  have  face-to-face  training  to  attend  classes  in  their  homes  or  offices  through  the  use  of  the 
Internet  ora  company’s  intranet  (Henke,  1997;  Levin,  Levin  & Waddoups,  1999). 

WBT  uses  Web  technologies,  such  as,  Web  browsers  including  Microsoft  Internet  Explorer  or  Netscape 
Navigator,  Hypertext  Transfer  Protocol  (HTTP),  Transmission  Control  Protocol/Intemet  Protocol  (TCP/IP) 
protocols,  and  Hypertext  Markup  Language  (HTML)  as  its  programming  language.  HTML  is  a language  that  allows 
computers  linked  to  the  Internet  to  transfer  information  to  one  another,  thus  enabling  the  computer  to  retrieve  all  or 
Just  parts  of  the  information  located  on  a Web  site  (Hall,  1997;  Polyson,  Saltzberg  & Godwin -Jones,  1996; 
Rutherford,  1996).  By  using  HTML,  the  learner  can  explore  additional  resources  in  other  web  sites  in  the  order  and 
at  the  pace  that  he  or  she  prefers.  This  experience,  for  most  WBT  learners,  is  similar  to  staying  after  class  to  talk 
with  other  students  or  the  instructor. 

WBT  can  be  synchronous  or  asynchronous.  Synchronous  training  is  instantaneous  and  allows  the 
instructor  and  learners  to  interact  via  the  Web  in  “real-time”  (Driscoll,  1998;  Kruse  & Keil,  2000).  This  can  be 
accomplished  through  on-line  discussions,  real-time  audio,  videoconferencing,  and  application  sharing  where  two  or 
more  people  can  work  on  the  same  file,  such  as  a shared  electronic  whiteboard  or  a spreadsheet,  simultaneously.  In 
contrast,  asynchronous  training  is  not  instantaneous;  the  learner  logs  on  to  the  course  to  complete  the  lesson  or  post, 
receives  and  responds  to  messages  at  his  or  her  own  pace  and  at  the  time  that  is  convenient  to  him  or  her(Bearman, 
1997;  Driscoll,  1998;  Kruse  & Keil,  2000).  In  using  either  type  of  WBT,  it  offers  learners  a higher  degree  of 
anonymity,  as  compared  to  face-to-face  instruction.  The  anonymity  enables  the  learner  to  interact  with  other 
learners  in  an  environment  free  from  bias  or  discriminations  due  to  age,  gender,  economic  status,  or  appearance 
differences. 

In  theory,  any  type  of  content  can  be  presented  via  WBT  (Beer,  2000;  Driscoll,  1998).  For  example,  WBT 
can  be  a good  venue  for  presenting  verbal  information.  WBT  is  also  appropriate  for  delivering  instruction  related  to 
concept  and  rule  learning  and  problem  solving  skills.  However,  motor  techniques  and  attitudinal  skills  are  not  well 
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suited  to  WBT.  This  is  due  to  the  limitations  of  WBT  in  allowing  the  learner  to  practice  the  sequence  involved  in 
mastering  the  motor  skills  as  well  as  procedures  required  for  attitude  or  behavioral  changes. 

The  Status  of  WBT  in  Corporate  America 

Many  corporations  now  use  some  form  of  WBT  to  deliver  training.  According  to  a Fortune  magazine 
survey,  corporations  in  year  2000  spent  approximately  20%  of  the  $66  billion  allocated  for  training  annually  on 
WBT  (Gotschall,  2000).  That  percentage  is  expected  to  increase  to  40%  by  the  year  2003.  In  addition,  a report 
provided  by  Masie  in  1997  stated  that  in  1997  alone,  71  percent  of  major  U.S.  companies  had  an  objective  of  using 
their  intranets  to  pilot  the  delivery  of  corporate  training  presentations.  Masie  also  reported  that  major  software 
publishers,  including  IBM  and  Microsoft,  are  working  on  online  learning  and  training  products  and  services. 
Companies  who  published  multimedia  authoring  tool  companies  like  Macromedia  and  Allen  Communications,  are 
also  building  Web  delivery  capability  into  its  products.  Finally,  Masie  detailed  that  the  majority  of  classroom-based 
training  companies,  such  as  CBT  Systems  and  Gartner  Group,  has  begun  to  establish  online  learning  products 
(Anonymous,  1997). 

Other  corporate  examples  in  using  WBT  include,  PricewaterhouseCoopers,  a consulting  firm,  is 
introducing  a new  learning  technology  called  Continua.  This  technology  allows  employees  to  form  virtual 
professional  communities  based  on  areas  of  interest,  abilities,  or  client  relations.  Services  such  as  peer  dialogue, 
chat  groups,  online  events,  and  online  training  are  part  of  Continua  (Koonce,  1998).  In  addition,  in  a recent  study  by 
Hall,  it  is  reported  that  the  Cisco  Systems  had  the  most  pervasive  e-leaming  implementation  (Hall  & LeCavalier, 
2000).  Cisco  Systems  offers  some  form  of  WBT  in  all  30  of  its  training  organizations,  with  its  major  emphasis  on 
sales  training.  However,  Cisco  itself  notes  that  it  uses  WBT  to  deliver  only  5%  of  its  total  training  (“Learning  on 
Demand”). 

Siemens  Business  Communications  launched  Siemens  Virtual  University  (SVU)  in  1997  and  now  offers 
more  than  64  courses,  from  Local  Area  Networks  and  Wide  Area  Networks  (LAN/WAN)  technology  to  customer 
relations,  to  its  7,500  employees  worldwide  (Frieswick,  1999).  Dell  Computer  Corporation  aggressively  pursued 
Web-based  training  as  a solution  to  its  employee  education  needs  (Cone,  2000;  Galagan,  2000).  The  company 
patterned  its  training  after  its  business  model,  which  emphasizes  a direct-to -customer  approach,  with  on-line  training 
available  24  hours  a day,  7 days  a week.  Delfs  Vice  President  of  Learning,  John  Cone,  expects  90%  of  its  courses  to 
be  delivered  online  by  2001 . In  1 998,  Dow  Chemical,  headquartered  in  Midland,  MI,  began  the  process  of 
converting  its  700,000  employee-hours  of  industry -related  training  into  Web-based  training  modules  (Barron, 

1999a).  It  anticipates  that  by  the  end  of  2001,  it  will  offer  more  than  60  courses  via  WBT,  representing  three- 
quarters  of  its  industry -related  training  and  an  additional  300,000  hours  of  employee-development  training. 

In  addition,  IBM,  Xerox,  Siemens  Business  Communications  and  Eli  Lilly  Corporation  employ  a 
combination  of  individual  assignments  coupled  with  group  discussions  and  projects  in  their  WBT  courses 
(Frieswick,  1999;  Hibbard,  1998;  “IBM  builds  its  manager  training,”  2000;  “Siemens  saves  $2000,”  2000).  Most  of 
these  courses  are  asynchronous.  Only  Chrysler,  Herman  Miller,  Siemens  Business  Communications  and  Aetna  offer 
real  time  learning  to  their  employees  (Barron,  1999a;  “Herman  Miller  turns,”  2000;  Hibbard,  1998). 

The  topics  most  commonly  taught  via  WBT  are  company-specific  procedures  and  products.  Southern 
California  Edison,  Siemens  Business  Communications,  Xerox,  Herman  Miller,  Dow  Chemical,  General  Motors, 

Ford  Motors,  Cisco  Systems,  Eli  Lilly,  and  Dell  all  use  WBT  to  train  employees  on  company-specific  processes  and 
products  (Barrett  & Lewis,  2000;  Barron,  1999a;  Beer,  2000;  Frieswick,  1999;  “Herman  Miller  turns,”  2000; 
Hibbard,  1998;  “Southern  California  Edison  uses,”  2000;  “Smarter  sales  force,”  2000).  Technology-based  courses, 
such  as  software  programs  or  how  to  navigate  the  Internet,  appear  to  be  second  to  product  and  processes  in  course 
offerings.  Dell,  Microsoft,  KPMG,  Charles  Schwab  and  Siemens  Business  Communications  all  offer  technology- 
based  courses  to  their  employees  (Balu,  2000;  Dobbs,  2000b;  Frieswick,  1999;  Gotschall,  2000;  Hall,  1997).  Dell 
and  Xerox  also  offer  an  orientation  program  to  new  employees  via  WBT  (Beer,  2000;  Cone,  2000)  and  Dell  and 
IBM  offer  management  courses  via  WBT  (IBM  builds  its  manager  training,  2000). 

Important  Considerations  of  Using  WBT  in  Corporations 

A company  must  consider  many  factors  when  deciding  whether  or  not  the  move  to  WBT  would  be  cost 
effective.  These  factors  are  explained  as  follows. 

Considerations  of  Cost  Effectiveness 
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The  first  consideration  is  the  cost  that  a company  will  spend  on  equipments,  for  delivering  WBT  courses. 
Normally,  a Web  server  dedicated  to  supporting  the  materials  can  range  from  less  than  $5,000  to  more  than 
$100,000  (Beer,  2000;  Hall,  1997)  and  in  some  cases,  may  also  require  an  access  license  fee  for  each  user  (Wells, 
1999).  Companies  may  also  need  to  pay  for  special  “plug-in”  software  for  special  features,  such  as  video, 
animation,  and  sound. 

Companies  also  need  to  consider  if  they  could  afford  allocating  internal  resources  to  create  WBT  courses. 
Normally,  it  would  take  between  100  — 200  hours  to  develop  one  hour  of  on-line  instruction  (Beer,  2000;  Frieswick, 
1999).  For  some  companies,  it  may  be  more  practical  to  contract  with  an  outside  vendor  to  develop  the  WBT 
courses.  Most  WBT  design  vendors  charge  anywhere  from  $7,000  to  $50,000  per  finished  hour,  depending  on  the 
course’s  complexity  and  use  of  multimedia  plug-ins  (Hall,  1997;  Kruse  & Keil,  2000).  Consequently,  whether  the 
WBT  is  developed  internally  or  externally,  companies  can  spend  thousands  of  dollars  on  the  development  stage.  In 
addition  to  the  development  of  WBT,  maintaining  the  course  is  another  issue.  Maintenance  costs  often  include 
revisions  to  the  course  and  subsequent  Web  site  maintenance  (Beer,  2000;  Hall,  1997;  Kilby,  1997;  Sanders,  1998). 
Web  server  maintenance  and  course  upgrades  can  cost  upwards  of  $500,000  per  year. 

Furthermore,  how  a WBT  course  will  be  used  also  affects  a company’s  overall  investment  considerations. 
For  example,  a course  that  will  be  delivered  once  or  twice,  then  abandoned,  is  not  a good  candidate  for  WBT  (Kruse 
& Keil,  2000).  Companies  must  consider  the  total  number  of  learners  who  will  be  using  the  course  and  consider  if 
the  course  can  help  reducing  the  amount  of  time  the  learners  will  be  pulled  out  from  their  regular  jobs. 

However,  while  initial  costs  for  WBT  can  run  anywhere  from  $75,000  to  $1.6  million  (Barron,  1999a; 
Frieswick,  1999),  Hall  (1997)  asserts  that  WBT  “reduces  the  cost  of  training  when  compared  to  instructor-led 
training. ..primarily  from  a reduction  in  training  time  and  the  elimination  of  travel”  (p.  108).  Wells  (1999)  also 
emphasized  that  the  initial  costs  to  implement  WBT  are  often  offset  by  the  long-term  gain  in  empbyee  productivity. 
By  using  WBT,  companies  can  train  a larger  number  of  employees  in  a shorter  time  frame  and  at  reduced  costs 
when  the  materials  are  delivered  via  WBT  (Barron,  1999a;  Beer,  2000;  Driscoll,  1998;  Hall,  1997;  Salopek,  1998; 
Wells,  1999). 

For  example,  Hewlett  Packard  reports  that  WBT  has  reduced  its  new  sales  representative  training  from  20 
weeks  to  eight  days  per  year  (Reinhardt,  1995).  Hewlett  Packard  estimates  saving  $800,000  per  year  since  its  use  of 
WBT.  Before  using  the  Web,  Hewlett  Packard  must  budget  travel  expenses  both  for  its  trainees  and  trainers,  and 
must  spend  money  on  renting  many  convention  centers  in  different  cities  for  its  sales  training.  Additional  examples 
include,  Aetna  U.S.  Healthcare  estimates  that  it  saved  $3  million  in  travel  and  associated  costs  the  first  18  months  of 
its  on-line  training  programs  (Barron,  1999a).  Steelcase,  the  largest  office  furniture  manufacturer  in  the  United 
States,  has  cut  its  training  costs  by  30%  through  the  use  of  Web-based  training  (“Steelcase  to  save  resources,”  2000). 
Siemens  Business  Communications  estimates  that  WBT  has  saved  the  company  $4  million  in  travel  costs  alone 
(Frieswick,  1999)  while  IBM  estimates  that  for  every  1000  employees  who  use  WBT,  the  company  saves  $500,000 
in  travel-associated  costs  (Kiser,  1999).  MCI  estimates  that  it  realized  a savings  of  $2.8  million  in  reduced  travel, 
facility  and  labor  costs  in  a 20-month  period  after  the  introduction  of  WBT  (Neilson,  Pasternack  & Viscio,  2000). 
Rockwell  Collins,  a company  that  manufactures  cockpit  instruments,  estimates  that  its  newly -implemented  WBT 
delivery  system  will  save  the  company  $14  million  in  three  years  in  travel  costs  alone  (Fister,  2000).  Price 
Waterhouse  realized  an  86%  cost  savings  using  WBT  over  traditional  classroom  delivery  (Hall,  1997).  Dow 
Chemical  estimates  its  $1.6  million  WBT  start-up  costs  are  yielding  a return  of  $4-$5  million  annually  (Barron, 
1999a).  And,  Budget  Rent-A-Car  reduced  its  per  employee  cost  from  $2000  to  $150  by  placing  its  new  employee 
training  on  the  Web  (Schaaf,  1997). 

Considerations  for  Course  Consistency  and  Delivery 

Other  factors  that  a company  needs  to  consider  when  using  WBT  include,  if  the  course  content  need  to  stay 
consistent  over  time,  and  if  the  course  need  to  be  delivered  to  the  learners  instantly.  If  the  answers  to  these 
questions  are  “yes’,  WBT  would  be  a good  choice.  WBT  offers  a higher  level  of  content  consistency  that  is  difficult 
to  be  attained  by  human  trainers.  It  can  also  prevent  from  a course  being  changed  in  re  mote  offices.  In  other  words, 
companies  can  use  WBT  to  provide  standardized  training  for  all  trainees  regardless  of  different  locations  or 
instructors  (Reinhardt,  1995).  WBT  can  also  provide  instant  distribution  of  the  course  content.  The  content  of  most 
WBT  can  be  updated  instantly,  which  is  not  possible  with  any  other  educational  technologies.  Technologies  such  as 
CD-ROMs,  printed  materials,  floppy  disks,  videos,  etc.  must  be  mechanically  reproduced,  then  mailed  to  the 
learners.  With  WBT  materials,  the  designer  simply  has  to  send  the  course  material  or  updates  to  the  server  and  it 
will  be  available  to  the  learner  instantly  (Driscoll,  1998;  Hibbard,  1998;  Kruse  & Keil,  2000;  Sanders,  1999;  Staley, 
1999). 
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Considerations  of  the  Bandwidth  Limitation 


One  of  the  frustrations  related  to  the  use  of  WBT  in  many  corporations  is  the  bandwidth  limitation. 
Bandwidth  determines  how  much  information  can  be  sent  across  the  web  and  how  fast  it  will  be  sent  (Filipczak, 
1997a;  Hall,  1997;  Schaaf,  1999a;  Williams,  1998),  Bandwidth  problems  translate  to  slow  transmission  and  delayed 
response  time  for  learners.  Learners  may  become  frustrated  and  bored  when  response  time  is  very  slow  (Kiser, 

1999;  Kurtus,  1997). 

Bandwidth  can  become  an  issue  in  one  of  three  places:  the  originating  computer,  the  “pipeline”  over  which 
the  information  is  sent  (in  most  cases,  a telephone  line),  or  the  receiving  computer.  The  most  common  bandwidth 
problems  focus  on  the  “pipeline.”  The  “pipeline”  refers  to  the  data-carrying  capacity  of  the  computer-to-computer 
delivery  channel,  Common  pipeline  sizes  range  from  relatively  small  28,8  kilobytes  per  second  (kbps)  modems  to 
optical  fiber  that  can  handle  more  than  10  gigabytes  (gbps)  per  second.  The  problem  of  pipeline  capacity  arises 
when  designers  add  data-intensive  multimedia  such  as  videos,  complex  audio,  live  transmissions,  and  animation  to 
WBT  programs  (Bassi  et  al.,  1998;  Filipczak,  1997a;  Kurtus,  1997;  Schaaf,  1999a;  Schaaf,  1997;  Williams,  1998). 
The  increasing  use  of  the  Internet  also  adds  to  bandwidth  problems  (Bassi  et  al.,  1998;  Schaaf,  1997).  In  1996, 
bandwidth  demand  was  approximately  200  trillion  bits  per  day  (bpd).  This  figure  is  expected  to  rise  to  9,000  trillion 
bpd  as  early  as  2001  and  to  220,000  trillion  by  2006,  creating  the  potential  for  a network  overload. 

However,  bandwidth  problems  can  be  lessened  by  creating  separate,  smaller  programs  that  are  executed 
within  the  primary  application  (“Applets,”  2000:  Beer,  2000).  These  smaller  programs,  called  applets,  are  designed 
to  be  downloaded  from  the  Web  and  launched  by  the  primary  application  to  support  simulations  and  other  media- 
intensive applications  without  adding  to  the  size  of  the  primary  program.  For  example,  the  designer  or  programmer 
could  create  a master  file  that  contains  the  rules  and  programming  language  necessary  to  run  all  simulations.  The 
master  file  is  then  stored  separately  from  smaller  files  that  contain  information  specific  to  individual  simulations. 
When  a simulation  is  started,  both  files  are  used,  but  because  they  are  stored  separately,  the  download  time  is  faster 
than  when  all  the  information  is  stored  in  one  large  file.  Recently  introduced  Digital  Subscriber  Lines  (DSL), 
satellite  transmission  systems  and  cable  modems  are  becoming  the  alternatives  to  slow  dial-up  modems  and 
bandwidth  limitations  (Bassi  et  al.,  1998;  Wiley,  South,  Basset,  Nelson,  Seawright,  Peterson  & Monson,  1999). 

These  technologies  offer  users  the  ability  to  connect  to  the  Web  via  standard  telephone  lines  and  receive  data  at  rates 
considerably  higher  than  33.6  kbs.  However,  because  DSL  and  cable  modems  use  shared  data  pathways,  increased 
use  of  these  technologies  eventually  will  result  in  reduced  transmission  speeds. 

Considerations  of  Learners'  Technical  Skills 

One  of  the  more  challenging  obstacles  when  implementing  WBT  is  the  learner's  lack  of  computer  and/or 
Internet  skills  (Wulf,  1996,  p.  54).  For  many  organizations,  this  could  mean  additional  technical  training.  In 
addition,  employees  with  limited  technical  background  and  skills  may  feel  apprehensive  and  confused.  They  may 
resist  WBT  training.  Another  possible  drawback  of  using  WBT  is  that  the  organization  must  rely  on  the  individual 
to  take  initiative  with  his  or  her  own  WBT  training  (Wulf,  1996,  p.  54).  With  WBT,  although  the  company  does  not 
need  to  require  an  employee  to  be  at  a certain  place  at  a particular  time,  however,  the  company  must  communicate 
with  employees  regarding  when  and  where  they  will  be  trained  and  the  level  of  participation. 

Other  Considerations 

In  addition  to  the  above  considerations,  other  considerations  include  security,  the  changing  role  of  the 
trainer  and  the  training  department,  and  the  impact  of  digital  relationships.  Security  has  become  a growing  concern. 
Since  the  Internet  was  originally  designed  as  a private  network  for  the  military  and  for  educational  use,  at  that  time, 
security  was  not  a problem.  With  the  growth  of  businesses  on  the  net,  the  need  for  security  has  become  greater  with 
a public  network  (Keen,  Mougayar,  & Torregrossa,  1998). 

There  is  also  a concern  regarding  the  skills  and  resources  needed  by  the  traditional  trainer  and  training 
department  if  the  organizations  focus  shifts  to  training  via  the  Web.  This  is  especially  true  since  close  to  50  percent 
of  training  departments  are  composed  of  classroom  trainers  (Appleton,  1998,  p.  1).  For  example,  the  online 
instructor  does  not  have  the  same  type  of  control  over  a group  of  learners  as  they  would  in  a typical  classroom 
environment.  The  instructor  must  become  more  of  a facilitator  or  moderator  (Kearsley,  1997).  Additionally,  an 
online  trainer  in  an  environment  that  does  not  incorporate  video  is  limited  by  not  being  able  to  see  the  learners.  By 
seeing  the  learners,  an  instructor  can  gauge  attention  levels  and  comprehension  of  material.  The  trainer  must  ensure 
that  the  learners  interact.  They  can  do  this  by  monitoring  chat  rooms,  virtual  white  boards,  etceteras  (Snell,  1998). 
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The  focus  of  training  departments  may  become  more  centered  on  using  Internet  tools  to  develop,  deliver,  and 
monitor  online  training  digitally,  as  opposed  to  developing  and  delivering  classroom  training. 

Finally,  there  is  concern  regarding  the  impact  of  the  prolonged  exposure  of  learners  and  instructors  to 
digital  environments  (Stone,  1996),  as  well  as  the  formation  of  digital  relationships  versus  the  traditional 
development  of  relationships  (Murphy,  1996).  Some  also  feel  that  with  the  loss  of  face-to-face  interaction, 
behavioral,  gestures,  and  tonal  cues  may  cause  misunderstanding  (Pennell,  1996). 

Organizational  Implications  in  Using  WBT 

The  first  implication  is  related  to  the  function  and  skills  of  trainers  or  instructional  designers  of  a company. 
While  developing  WBT  courses,  corporate  trainers  or  instructional  designers  need  to  be  proficient  in  skills  that  are 
not  normally  associated  with  classroom  or  face -to  face  instruction  design  (Alexander,  1995;  Beer,  2000;  Curtin, 

1997;  Dobbs,  2000b;  Hall  & LeCavalier,  2000;  Horton,  2000).  They  now  need  to  possess  a thorough  understanding 
of  HTML,  computers,  networking,  and  Internet  protocols  in  order  to  incorporate  materials  that  are  Web-compatible. 
Clearly,  the  Web  is  a largely  visual  medium;  it  would  be  helpful  for  corporate  trainers  to  be  proficient  in  graphic 
design  and  layout.  It  would  be  also  helpful  for  the  trainers  or  designers  to  have  an  artistic  perspective  and  a basic 
understanding  of  ratio  aspects. 

The  second  implication  concerns  the  relationship  between  the  Training  department  and  the  Information 
Technology  (IT)  department.  Clearly,  as  a company  moves  to  WBT,  the  Training  Department  needs  technical 
support  from  the  IT  department  (Curtin,  1997;  Driscoll,  1998;  Filipczak,  1997a;  Hall  & LeCavalier,  2000;  Kiser, 
1999).  The  two  departments  now  must  work  together  to  ensure  that  WBT  does  not  create  a drain  on  the  company’s 
network,  servers  and  IT  Help  Desk. 

One  result  of  this  collaboration  will  be  that  trainers  and/or  the  instructional  designers  instructional 
designers  will  have  a better  understanding  of  the  technology  used  for  WBT,  and  the  IT  professionals  will  benefit 
from  an  increased  understanding  of  instructional  design  process. 

Finally,  the  last  implication  is  about  merging  working  and  learning  among  employees.  As  WBT  becomes 
more  commonplace  in  organizations,  companies  will  find  more  ways  to  integrate  it  into  employees’  everyday  work 
lives  (Beer,  2000;  Cone,  2000;  Driscoll,  1998;  Ryan,  Neece  & Meyer,  2000).  Employees  will  have  WBT  available 
as  a means  of  quick  reference  at  their  desktops.  They  will  be  able  to  access  a WBT  course,  or  a small  piece  of  it,  to 
seek  the  information  they  need.  Employees  will  come  to  expect  instantaneous  information  and  to  access  to  subject 
matter  experts  as  a way  of  doing  business.  Employees  will  eventually  become  so  used  to  the  process  of  receiving 
information  “Just-in-time”  without  even  realizing  that  they  are  learning  while  working. 

Reload  buttons  should  be  available  to  make  the  learner’s  navigation  through  the  course  easier  (Kruse  & 
Keil,  2000).  In  addition,  page  transitions  should  be  simple  and  speedy  to  avoid  long  download  times  and  subsequent 
learner  frustration  (Curtin,  1997;  Williams,  1998).  Every  page  should  include  a link  to  the  course’s  home  page  to 
allow  a “lost”  learner  to  find  his  or  her  way  back  to  the  course  easily  (Nielsen,  1996).  Multimedia  should  be 
avoided  because  of  download  time  constraints,  unless  it  is  critical  to  the  learning  process  (Beer,  2000;  Clark  & 

Lyons,  1999). 

Finally,  storyboards  for  each  WBT  module  are  strongly  recommended  for  designing  and  revising  purposes 
(Driscoll,  1998;  Hall,  1997;  Kruse  & Keil,  2000;  Ward,  1998).  Storyboards  are  visual  representation  of  the 
information  that  will  be  included  on  the  screen.  One  storyboard  should  be  created  for  each  screen,  and  it  should 
include  thumbnail  sketches  of  all  onscreen  visuals  and  the  corresponding  text.  Storyboards  should  include 
identifying  information,  such  as  the  course  title,  the  date,  the  version  or  draft  number,  and  the  page  number.  It  will 
also  specify  the  names  of  the  files  (audio,  video,  graphic)  to  be  used,  file  numb  ering  schemes,  programming  notes, 
and  branching  instructions. 

Conclusion 

WBT  is  time  and  location  independent  and  thus  enabling  companies  to  train  their  widely  dispersed 
employees  without  incurring  exorbitant  travel  cost.  Employees  are  also  allowed  to  have  a higher  level  of  flexibility 
while  taking  a WBT  course.  However,  technical  and  design  issues  present  obstacles  to  the  effective  use  of  WBT  in 
many  corporations.  It  is  imperative  that  WBT  is  grounded  in  learning  theories  and  instructional  design  principles. 
Effective  WBT  must  also  engage  employees  and  provides  them  with  opportunities  to  apply  the  gained  knowledge 
and  skills  back  to  their  Jobs. 
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Introduction 

Computer  conferencing,  which  entails  "conducting  a conference  between  two  or  more  participants  at 
different  sites  by  using  computer  networks  to  transmit  any  combination  of  text,  static  pictures,  audio  and/or  motion 
video"  (Palloff  & Pratt,  1999,  p.189-190)  originally  started  as  a supplementary  instructional  mode  in  a conventional 
face-to-face  class.  Recently  however,  computer  conferencing  is  sometimes  used  as  a primary  mode  of  instruction 
and  it  is  even  considered  an  alternative  instructional  method  with  high  interest.  This  trend  naturally  leads  to  the 
question  of  its  educational  significance. 

There  are  extensive  studies  about  computer  conferencing  despite  short  history  and  most  studies  explain  or 
enumerate  its  characteristics  and  advantages.  However,  most  publications  are  theoretical  discussions  without  a 
concrete  case.  Recently,  a few  studies  describe  what  typically  happens  in  an  online  course  (e.g.,  Eastmond,  1995; 
McDonald,  1998).  Thus,  this  study  describes  what  really  happens  in  an  online  course  to  test  the  educational  value  of 
computer  conferencing.  Especially,  this  study  focuses  on  self-direction  among  many  advantages  of  computer 
conferencing  because  it  is  a usual  goal  of  distance  education. 

Computer  conferencing 

Many  studies  explain  the  characteristics  of  computer  conferencing  (e.g.,  Feenberg,  1989;  Harasim,  1989, 
1990;  Kaye,  1989;  Mason  & Kaye,  1989).  The  major  characteristics  of  computer  conferencing  include  many-to- 
many  communication,  place  and  time  independence,  text -based  communication,  and  high  interactivity.  Computer 
conferencing  makes  group  communication  possible  that  was  impossible  in  traditional  distance  education  modes. 
Computer  conferencing  synthesizes  advantages  of  face-to-face  and  distance  education  in  that  all  participants  interact 
with  each  other  and  this  interaction  is  time  and  place  independent.  In  addition,  instructors  manage  learning  programs 
and  courses  that  are  adaptive  and  responsive  to  the  needs  of  individual  learners,  by  ongoing  communication. 
Consequently,  computer  conferencing  is  a paradox  in  distance  education  by  eliminating  psychological  separation 
between  learner  and  instructor  through  interaction  (Saba,  1988). 

Interaction 

Distance  education  can  be  differently  defined  according  to  its  major  focus.  Most  distance  educators, 
however,  agree  that  distance  education  assumes  a physical  separation  between  instructor  and  learner  (e.g., 

Holmberg,  1995;  Keegan,  1996).  As  a result,  learners  engage  in  learning,  while  considering  convenient  times  and 
places  instead  of  scheduled  ones.  Thus,  learners  have  independence  in  time  and  place.  Traditional  distance  education 
modes,  however,  are  based  on  one-way  communication  in  which  the  teacher  controls  learning.  As  a result,  learners 
become  more  passive  in  the  learning  process  than  in  a face-to-face  class  considering  that  they  can’t  share  their 
opinions.  At  the  same  time,  learners  experience  isolation  because  of  psychological  distance  resulting  from  lack  of 
dialogue  between  learner  and  instructor  or  among  learners.  Consequently,  learners  do  not  have  independence  in 
controlling  and  directing  their  learning  process.  Especially,  considering  that  learning  is  a transactional  process, 
which  entails  exchanging  ideas,  thoughts,  and  feelings  among  people  (Palloff  & Pratt,  1999),  the  absence  of 
interaction  has  been  a great  weakness,  even  though  early  distance  education  served  the  solitary  learner. 

Therefore,  distance  educators  have  searched  for  new  educational  methods  to  increase  dialogue  among 
participants  for  effective  learning.  The  history  of  American  distance  education  supports  the  importance  of 
interaction  in  distance  education.  That  is,  continuous  efforts  providing  learners  with  independence  in  their  learning 
led  to  interest  in  interaction  among  participants  during  the  learning  process.  Thus,  the  concept  of  interaction  and  its 
role  in  distance  education  have  been  important  topics  in  distance  education  research  (Moore,  1995). 

Why  is  interaction  important  in  distance  education?  Interaction  in  distance  education  is  closely  related  to 
individualization  in  learning.  That  is,  when  learners  actively  interact  with  instructors  or  peers,  they  can  address  their 
consistently  changing  needs.  This  can  lead  to  suitable  learning  for  each  person.  This  provides  learners  with 
independence  in  the  learning  process  as  well.  Therefore,  interaction  is  very  important  in  distance  education  and  it 
explains  why  many  people  undoubtedly  welcome  computer  conferencing  characterized  by  high  interaction. 
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Self-direction 


For  a long  time,  adult  educators  have  emphasized  the  learner's  autonomy  or  independence  in  self-directed 
learning.  It  leads  to  the  understanding  of  self-directed  learning  in  isolation  excluding  external  assistance  or 
interaction.  The  self-directed  learner,  however,  is  not,  in  Moore’s  word,  “an  intellectual  Robinson  Crusoe,  castaway 
and  shut  off  in  self-sufficiency"  (1973,  p.669,  cited  in  Brookfield,  1986).  In  this  context,  many  adult  educators  argue 
the  importance  of  interaction  related  to  self-direction.  Brookfield  (1985)  states  the  importance  of  external  sources  of 
assistance  regarding  self-direction.  Garrison  (1987)  supports  the  idea  that  self-direction  is  highly  dependent  upon 
interaction  and  collaboration  between  a learner  and  facilitator.  Candy  (1991)  suggests  a new  view  defining  self- 
direction  as  a product  of  interaction  between  the  person  and  the  environments.  With  the  criticism  of  previous  studies 
focusing  on  external  control  in  the  learning  process,  recently  the  definitions  and  conceptualizations  have  moved 
away  from  equating  self-directed  learning  with  sociologically  independent  learning  to  an  interactive  concept. 

Garrison’s  model  of  self-directed  learning 

This  study  followed  current  research  trend  focused  on  multidimensional  and  interactive  aspect  of  self- 
direction.  In  this  context.  Garrison’s  comprehensive  model  (1997)  and  his  concept  of  control  construct  (1993) 
contributed  to  developing  the  conceptual  framework.  According  to  Garrison,  self-directed  learning  is  comprised  of 
self-management,  self-monitoring,  and  motivation  and  each  dimension  is  very  closely  connected.  Self-management 
means  "decision  of  learning  objectives  and  activities,  and  the  management  of  learning  resources  and  support"  (p.22). 
It  entails  the  learner's  external  control  over  the  learning  process.  However,  Garrison  explains  the  concept  of  self- 
management based  on  collaborative  relationships  between  teacher  and  learner  instead  of  independence.  Thus,  self- 
management does  not  mean  simply  learner’s  external  control.  That  is,  self-management  (control)  consists  of  three 
variables,  proficiency,  resources,  and  interdependence  (Garrison,  1993).  Proficiency  refers  to  ’’the  abilities  and  skills 
of  the  teacher  and  student  related  to  construction  of  knowledge".  Resources  refer  to  "diverse  support  from  teacher 
and  assistance  from  learner  in  educational  contexts".  Interdependence  means  "teacher  responsibilities  (institutional 
or  subject  norms)  as  well  as  learner  choice  (freedom)"  (p.3 1 -33).  That  is,  the  learner  has  independence  in 
transactions  with  the  teacher.  Consequently,  the  balance  of  each  variable  influences  the  extent  of  control.  Control  is 
dynamic  by  existing  in  the  circle  of  communication  between  teacher,  learner,  and  curriculum  as  well. 

Self-monitoring  means  the  internal  aspect  of  self-direction  by  referring  to  construction  of  meaning.  It 
assumes  that  the  learner  has  responsibility  to  construct  knowledge.  This  is  related  to  the  extent  of  the  learner's 
cognitive  ability  and  thus  it  leads  to  differences  in  the  degree  of  self-direction.  Self-monitoring  is  a metacognitive 
process  because  it  needs  critical  reflection  as  well.  Self-monitoring  includes  collaborative  confirmation.  That  is, 
constructed  knowledge  based  on  the  learner's  cognitive  ability  is  confirmed  by  others  in  the  learning  process.  Thus, 
self-monitoring  depends  on  shared  control  based  on  transactions  between  learner  and  teacher. 

Motivation  refers  to  "perception  and  anticipation  related  to  learning  goals  at  the  beginning  of  learning  and 
mediates  between  control  and  responsibility  during  the  learning  process"  (p.28).  Motivation  includes  entering  and 
task  motivation.  Entering  motivation  is  related  to  the  decision  to  participate  at  the  beginning.  Task  motivation  is 
related  to  the  will  to  continue  the  task  during  the  process. 

Research  objectives 

This  study  explored  the  extent  to  which  computer  conferencing  contributes  to  facilitating  self-direction  by 
describing  concrete  activities.  This  study  focused  on  interaction  among  participants  related  to  self-direction  because 
it  is  very  characteristic  of  computer  conferencing.  At  the  same  time,  it  is  necessary  to  study  self-direction  in  group 
learning  rather  than  individual  study. 

This  study  had  the  following  objectives.  First,  this  study  explored  the  possibility  that  computer 
conferencing  can  contribute  to  facilitating  self-direction  by  describing  interaction  patterns  in  an  online  course. 
Second,  this  study  sought  evidence  that  learners  are  self-directed  in  an  online  course  by  describing  concrete  self- 
directed  activities  that  occurred  in  computer  conferencing.  Third,  this  study  showed  the  extent  of  self-direction 
students  have  in  this  course.  Finally,  this  study  focused  on  interaction  to  explain  how  learners  are  self-directed  in  an 
online  course. 
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Conceptual  framework 


This  study  adapted  Garrison’s  model  (1997)  considering  the  characteristics  of  data  and  research  questions 
to  develop  conceptual  framework.  Consequently,  in  this  study  self-direction  consists  of  three  dimensions  (control, 
critical  reflection,  responsibility)  and  interaction  influences  self-direction.  Self-direction  means  both  overall  decision 
and  management  related  to  learning  and  knowledge  construction.  Control  means  the  opportunity  and  the  ability 
related  to  making  decisions  in  the  learning  process,  capability  to  manage  resources,  and  cognitive  ability  to  construct 
knowledge.  It  consists  of  interdependence,  proficiency,  and  resources.  Critical  reflection  means  the  process  of 
constructing  personal  meaning.  Responsibility  means  learner's  active  attitude  to  learn. 

Case  description 

A graduate  level  course  taught  adult  education  via  computer  conferencing  in  the  Spring  of  1998  at  a 
midwestem  university  was  chosen  as  the  case  for  this  study.  This  course  consisted  of  twenty-one  students.  Among 
them  were,  fifteen  females  and  six  males,  fifteen  majored  in  adult  education  and  six  other  majored,  seven 
experienced  people  of  FirstClass®,  used  as  computer  conferencing  software  in  this  course,  fourteen  nonexperienced. 
One  female  instructor  taught  this  course.  Students  had  one  face-to-face  meeting  at  the  beginning  of  course  to  receive 
information  related  to  course  and  to  practice  FirstClass®.  After  the  initial  face-to-face  meeting,  all  discussions 
depended  almost  entirely  on  asynchronous  communication.  The  instructor  separated  students  into  four  groups 
considering  previous  FirstClass®  experience,  gender,  distance  to  campus,  department,  and  similar  name.  The 
instructor  posted  an  overview  including  readings,  brief  summary  of  content,  questions  and  activities  for  each  Week 
folder.  Each  group  decided  on  a weekly  moderator  who  read  weekly  messages  posted  until  Thursday  midnight  and 
summarized  the  discussion.  The  moderator  posted  a summary  for  each  week  in  a Week  folder  so  all  four  groups  has 
a chance  to  read  the  summaries  of  all  the  groups'  discussion  of  the  past  week.  Students  proceeded  group  project  as 
well  as  individual  learning  activity  according  to  group  member's  interest. 

Methodology 

This  study  analyzed  the  content  of  transcripts  of  participants  to  explore  research  questions.  In  this  study, 
weeks  3,  8,  and  13  were  selected  based  on  time  period,  the  number  of  messages,  the  progress  of  group  projects,  and 
the  topic  of  the  week.  The  total  number  of  messages  during  weeks  3,  8,  13  was  1,333.  It  was  necessary  to  code  total 
messages  for  data  analysis.  In  this  study,  speech  segment  created  by  Henri  and  Rigault  (1996)  was  used  as  a single 
unit  for  data  analysis.  After  the  coding  work  was  finished,  inter-rater  reliabilities  were  assessed  to  decide  whether 
several  raters  had  a high  degree  of  consistency.  Cohen's  reliabilities  for  interaction  type,  function,  and  self- 
direction  were  .95,  .90,  and  .89,  respectively. 

Results  and  discussion 

Objective  #1:  Interaction  patterns 

The  degree  of  students’  participation  shows  whether  this  online  course  is  learner-centered  or  teacher- 
centered.  During  weeks  3,  8,  and  13,  participants  posted  1,333  messages.  Students  posted  1,159  and  instructor 
posted  174  messages.  In  addition,  21  students  generated  1,159  messages  during  3 weeks,  averaging  approximately 
19  messages/person/week.  Related  to  interaction  type,  students  provided  94%  of  initiation.  This  result  was  in  accord 
with  the  previous  studies  (Eastmond,  1995;  Garrison,  1987;  Mason,  1988;  Seaton,  1993),  insisting  that  an  online 
course  should  be  learner-centered. 

Objective  #2:  Self-Directed  activities 

This  study  categorized  self-directed  activities  which  occurred  in  this  online  course  based  on  three 
dimensions  of  self-direction:  control,  critical  reflection,  and  responsibility.  Selective  response,  autonomous  decision, 
providing  norms,  and  negotiation  were  classified  as  activities  of  interdependence,  one  component  of  control. 
Interpretation,  definition,  judgment,  and  challenging  or  questioning  were  classified  as  activities  of  proficiency,  one 
component  of  control.  Providing  help,  sharing  information,  providing  social  support,  and  confirmation  were 
classified  as  activities  regarding  resources,  one  component  of  control.  Revision,  correction,  finding  misconceptions, 
meeting  new  perspectives,  self-reflection,  connecting  with  previous  knowledge,  experience,  current  situation,  and 
knowledge  construction  were  classified  as  activities  of  critical  reflection.  Asking  for  help,  information,  clarification, 
confirmation,  and  notification  of  nonparticipation  were  classified  as  activities  of  responsibility. 
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Self-directed  activities  classified  in  this  study  confirmed  the  current  conclusion  that  self-direction  is 
multidimensional  and  is  based  on  interaction.  That  is,  self-directed  activities  confirmed  the  conclusion  that  learners 
had  self-direction  through  ongoing  interaction.  After  all,  students  had  self-direction  in  learning  in  the  process  of 
establishing  shared  agreement,  collective  knowledge,  and  shared  control.  Self-directed  activities  also  reflected  a 
characteristic  of  computer  conferencing.  That  is,  asynchronous  communication  and  text -based  communication 
influenced  self-directed  activities.  For  example,  students  interacted  with  each  other  through  written  messages 
because  computer  conferencing  is  based  on  text-based  communication.  Thus,  selective  response  was  a strategy  for 
controlling  learning  content.  At  the  same  time,  this  was  a very  important  tool  related  to  control  considering  that  this 
course  was  response-centered.  In  addition,  the  self-directed  activities  that  were  found  reflected  the  characteristics  of 
group  learning  based  on  interdependence  and  interaction.  Thus,  these  activities  had  limited  generalization  to  face-to- 
face  courses  and  individual  learning.  As  an  extension  of  this  study,  the  comparisons  between  face-to-face  and  online 
course  and  between  individual  and  group  learning  can  contribute  to  explaining  how  students  are  self-directed  in 
virtual  group  learning. 

Objective  #3:  Extent  of  self-direction 

There  were  1,852  speech  segments  related  to  self-direction  which  was  70%  of  the  total  speech  segments. 
This  means  that  this  course  had  a relatively  high  degree  of  self-direction.  Within  the  three  components  of  self- 
direction,  the  number  of  speech  segments  classified  as  control  was  1,442,  critical  reflection  was  214,  and 
responsibility  was  196.  Thus,  students'  self-direction  in  this  online  course  appeared  to  be  mainly  related  to  control. 
The  number  of  speech  segments  which  were  classified  as  interdependence  was  143,  proficiency  was  506,  and 
resources  was  793  among  the  three  components  of  control.  This  showed  that  students  participated  in  knowledge 
construction  during  the  course  and  they  received  diverse  resources  from  instructor  and  peers.  The  high  percentage 
classified  as  resources  implies  that  interaction  among  participants  contributed  to  developing  self-direction. 

Objective  #4:  Mechanism  of  facilitating  self-direction 

High  percentage  of  resources  related  to  self-direction  supports  the  importance  of  interaction.  Previous 
studies  assumed  interaction  influences  self-direction  without  evidence.  The  researcher  paid  attention  to  message 
threads  made  by  participants  to  examine  the  relationships  between  self-direction  and  interaction.  Students 
sometimes  opposed  or  questioned  previously  taken  for  granted  assumptions  and  this  engendered  other's  perspective 
transformation.  Students  experienced  critical  reflection  through  these  processes.  Students  constructed  knowledge 
and  the  instructor  or  peers  confirmed  it.  Thus,  interaction  connected  control  with  critical  reflection.  Interaction 
influenced  the  relationship  between  responsibility  and  control  as  well.  Students  asked  for  help  when  they 
experienced  technical  problems  or  needed  more  information  and  saw  this  as  their  responsibility.  An  instructor  or 
peers  provided  students  with  diverse  resources.  Interaction  connected  responsibility  with  critical  reflection.  Students 
sometimes  did  not  have  a clear  understanding  because  of  an  ambiguous  use  of  a word.  Students  asked  for  more 
clarification  to  the  author  of  a problematic  message.  The  author  automatically  had  an  opportunity  to  think  again 
about  his  or  her  own  message  and  revise  it.  This  helped  critical  reflection.  After  all,  three  components  of  self- 
direction:  control,  critical  reflection,  and  responsibility  are  connected  each  other  through  learner-learner  interaction 
and  learner-instructor  interaction. 

The  mechanism  students  can  be  self-directed  through  interaction  in  an  online  course  leads  to  the  conclusion 
that  an  instructor  and  peers  play  important  roles  related  to  facilitating  self-direction.  An  instructor  and  peers 
influenced  student’s  decision  process.  Instructor  participated  in  this  course  one  of  participants  and  she 
collaboratively  managed  course  with  students.  Related  to  collaborative  course  management,  the  degree  of  a structure 
of  course  is  very  important  because  it  influences  the  latitude  for  student's  choice.  This  course  was  managed 
according  to  a syllabus  made  by  the  instructor  before  course  began  and  weekly  overviews  during  the  course. 
However,  even  though  the  instructor  decided  the  overall  structure,  it  could  be  changed  according  to  students' 
response.  Actually,  students  sometimes  challenged  to  a rule  or  decision  made  by  instructor.  This  resulted  in  a change 
of  the  teacher's  structure  based  on  shared  agreement.  The  course  structure  was  maintained  not  only  by  interaction 
between  instructor  and  students  but  also  by  negotiation  among  students.  Students  sometimes  solved  problems 
themselves  based  on  collaborative  decisions.  The  instructor  provided  students  with  minimal  guidelines  and 
facilitated  their  learning  as  not  a lecturer  but  as  a facilitator  in  democratic  environment.  Thus,  students  frequently 
negotiated  with  the  instructor  and  peers  for  effective  course  management  and  have  shared  control  as  a result. 

Students  built  collective  knowledge  through  interaction.  An  instructor  and  peers  made  learner  challenge 
previous  or  current  knowledge  and  helped  to  build  new  knowledge.  The  role  of  instructor  and  peers  did  not  end  only 
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with  challenging.  The  instructor  and  peers  confirmed  constructed  knowledge.  This  helped  the  learner  to  make  his  or 
her  knowledge  meaningful  under  this  process.  In  addition,  the  instructor  and  peers  provided  learner  with  cognitive 
help  related  to  knowledge  construction.  When  students  needed  supplementary  explanations,  they  rephrased 
ambiguous  terms  and  provided  suitable  examples.  Both  instructor  and  peers  contributed  to  building  learning 
community  in  virtual  learning  environment. 

All  participants  shared  diverse  resources  in  a flexible  time  schedule.  Resources  that  participants  shared 
were  not  related  to  only  content  and  technical  aspects.  Students  shared  personal  experience.  Social  support  from  the 
instructor  and  peers  helped  learner  to  overcome  psychological  anxiety  and  isolation  related  to  using  computer 
conferencing.  Consequently,  students  share  same  feeling  and  experience  through  interaction  and  feel  their  learning 
community. 

Implications  for  practice 

This  study  confirmed  that  students  are  self-directed  in  an  online  course,  as  indicated  by  concrete  self- 
directed  activities  and  their  extent.  Furthermore,  this  study  explained  how  students  are  self-directed  in  an  online 
course  by  focusing  on  instructor  and  peer's  roles.  Major  findings  have  several  implications  for  practitioners. 

First,  this  study  showed  that  a low  level  of  structure  is  related  to  self-direction.  The  instructor  did  not  force 
students  to  follow  fixed  procedures  but  modified  the  structure  to  be  responsive  to  learners'  needs.  Students  had  room 
for  expressing  their  needs  and  making  decisions  in  this  environment  and  self-direction  was  a result.  Thus,  it  is 
desirable  for  students  to  have  an  opportunity  to  choose  or  decide  related  to  their  learning.  Instructors  should  modify 
traditional  authoritative  roles  and  interact  with  participants  in  order  to  create  such  an  environment. 

Second,  students'  self-direction  appeared  most  in  the  control  dimension.  Students  were  self-directed 
regarding  proficiency  and  resources.  This  supports  the  important  role  of  interaction  in  facilitating  self-direction. 

That  is,  students  collaboratively  construct  knowledge  and  frequent  feedback  from  instructor  and  peers  helps  students 
to  achieve  content  and  technical  mastery.  Therefore,  instructors  and  course  designers  should  pay  attention  to  how  to 
provide  students  with  frequent  feedback. 

Finally,  this  study  confirmed  that  the  learning  environment  is  very  important  in  developing  self-direction. 
Students  experienced  psychological  anxiety  related  to  a new  learning  environment  and  missed  features  of  a 
traditional  face-to-face  course.  However,  students  found  that  they  studied  not  alone  but  together  through  frequent 
interaction.  Social  support  from  instructor  and  peers  and  sharing  personal  experience  made  students  feel  comfortable 
in  the  new  environment.  Thus,  instructors  should  make  an  effort  to  develop  a comfortable  learning  environment  to 
facilitating  self-direction,  for  example  through  informal  discussion.  A comfortable  learning  environment  will  help 
encourage  lurkers  to  become  participants. 
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An  Analysis  of  Faculty  Concerns  Regarding  Distance  Education  at 

Canton  College 
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Suny  Canton 


PART  A:  INTRODUCTION 

Nanotechnology,  the  new  frontier  of  microscopic  technology,  is  shaking  up  the  scientific  world  with  the 
same  “promise  and  fear”  that  the  atomic  theory  first  shook  the  world  during  the  1950’s  ( USA  Today ^ 24  October 
2000).  The  building  of  mechanical  and  information  machines  on  a nanometer  scale  (one-billionth  of  a meter)  has 
attracted  both  its  supporters  and  detractors.  Advocates  point  to  the  promise  of  one-day  sending  cancer-detecting 
microscopic  machines  into  the  human  body  to  deliver  cancer-  fighting  drugs.  Opponents,  on  the  other  hand,  warn 
that  terrorists  may  someday  use  the  technology  to  create  nanosize  mechanical  germs  for  release  in  terrorist’s  attacks. 
Should  the  scientific  community  embrace  or  reject  this  new  technology?  What  stirs  its  debate?  Is  it 
misunderstanding  or  legitimate  concerns  regarding  its  use  or  abuse? 

New  technology,  whether  it  is  new  technology  in  the  scientific  field  or  new  technology  in  the  educational 
field,  spurs  debate.  In  a recent  article  entitled  “College  Online:  Rethinking  How  to  Learn,”  published  in  the 
Watertown  Times,  1 0 September  2000,  distance  learning  is  viewed  as  a revolutionary  force  that  “looms  large  on  the 
education  landscape.”  As  established  universities  race  ahead  with  online  programs,  faculty  members  continually 
debate  its  virtues  and  uses.  To  many  educators  distance  education  is  an  innovative  and  creative  instructional 
medium.  For  others,  it  is  the  commercialism  of  education,  an  educational  format  plagued  by  pedagogical 
uncertainties.  Many  college  campuses  across  the  country  are  currently  engaged  in  its  controversy,  and  the  State 
University  of  Canton  College  is  no  exception.  With  its  unique  blend  of  liberal  arts  and  technology  education.  Canton 
College  includes  faculty  that  both  support  and  deny  the  merits  of  distance  education.  No  where  is  this  more  evident 
than  at  faculty  meetings  where  heated  debates  often  erupt  between  a skeptical  liberal  arts  faction  and  a so-called 
“willing”  technology  and  science  faction.  Such  divergent  educational  viewpoints  in  one  institution,  I believe, 
provide  a rich  environment  in  order  to  research  faculty  concerns  related  to  distance  education. 

Overview  of  distance  education 

“Femunterricht”  or  “Tele-enseignement”  are  both  European  terms  for  one  of  the  fastest  growing  and  most 
controversial  segments  of  higher  education,  distance  education.  While  the  terms  distance  education  and  distance 
learning  are  often  used  interchangeably,  each  word  has  a distinct  meaning.  Distance  education  describes  the 
organizational  framework  and  process  of  providing  education  at  a distance.  In  this  framework,  teachers  and  students 
are  separated  by  physical  distance  and  technology  is  used  to  bridge  the  instructional  delivery  (Moore  1989). 

Distance  learning,  on  the  other  hand,  identifies  the  intended  instructional  outcomes  and  depicts  the  learning  that 
takes  place  at  a distance.  Every  effort  will  be  made  in  this  paper  to  adhere  to  these  specific  definitions. 

Historical  perspective 

Most  people  assume  that  distance  education  is  a relatively  new  phenomenon.  However,  the  growth  in  the 
area  of  distance  education  is  best  characterized  as  evolutionary  rather  than  revolutionary  (Willis  1994).  Learning  at  a 
distance  has  been  going  on  for  years;  it  is  the  mode  of  delivery  that  has  changed.  Today,  audio,  video  and  computer 
technologies  are  more  common  methods  of  delivery  than  the  historical  correspondence  courses  of  years  ago. 

Traditionally,  distance  education  students  have  been  adults,  studying  part-time  and  geographically 
distanced  from  the  enrolled  campus.  Access  to  continuing  and  higher  education  has  typically  been  the  primary 
motivating  factor  in  students  enrolling  in  distance  education  courses.  Recent  evidence  suggests  that  this 
demographic  trend  is  changing.  Younger,  local  students  with  full-time  course  loads  that  combine  distance  education 
courses  with  on-campus  courses  now  comprise  new  student  populations  (Wallace  1996).  While  students  of  the  past 
sought  access  to  continuing  and  higher  education;  students  of  today  are  attracted  to  distance  education  because  of 
the  ability  gain  control  over  the  time,  place  and  pace  of  their  learning.  Consider  the  following  article  entitled, 

“SUNY  Online  Classes  Giving  More  People  Access  to  Higher  Ed,”  appearing  in  the  Watertown  Times,  20  October 
2000: 


Ms.  Walker  is  a full-time  student  at  Jefferson  Community  College.  Four  days  a week 
she  commutes  to  take  four  night  classes.  There  were  not  enough  houre  in  her  schedule 
to  take  another  class,  so  he  decided  to  take  one  of  her  classes  online.  This  course  is 
working  wonderful  for  me  as  a mother  because  I can  do  it  at  home.  I wouldn’t  be  able 


to  take  a fifth  class  if  it  wasn’t  online.”  ..  q 
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Delivery  Technologies 


Distance  education  involves  two  main  delivery  methods:  synchronous  (interactive  video)  and  asynchronous 
(online  or  web-based).  Synchronous  learning  is  the  simultaneous  participation  of  student  and  instructor.  Instructional 
delivery  is  at  the  same  time,  but  at  different  locations.  For  example,  various  campuses  in  the  State  University  of 
New  York  (SUNY)  are  linked  to  one  another  via  a telecommunications  line.  This  telecommunication  line  transmits 
live  video.  Thus,  a SUNY  Canton  site  may  be  transmitting  a “live”  course  via  television  to  another  SUNY  campus, 
allowing  live  group  discussion  between  sites.  In  order  to  accommodate  synchronous  delivery,  special  distance 
learning  classrooms  must  be  constructed.  These  classrooms  are  equipped  with  television  monitors,  ceiling 
microphones,  sound  proof  walls,  as  well  as  computer  and  other  technological  equipment  necessary  to  conduct 
interactive  classrooms.  Such  classrooms  can  be  expensive.  They  require  a significant  initial  capital  investment,  and 
funding  for  on-going  equipment  maintenance  and  support. 

Asynchronous  instruction,  on  the  other  hand,  does  not  require  that  student  and  instructor  be  located  in  the 
same  location  at  the  same  time.  Instructional  delivery  is  at  different  times  and  different  locations.  Examples  of 
asynchronous  delivery  include  email,  listservs,  and  web-based  or  online  instruction.  Asynchronous  instruction  can 
occur  anytime,  anyplace,  as  long  as  the  student  has  a computer  and  access  to  the  course  through  a registration 
process.  Students  are  able  to  choose  their  own  instructional  time  frame,  and  studying  lectures  and  participating  in 
their  class  according  to  their  own  schedule.  The  primary  advantage  of  asynchronous  learning  is  flexibility;  students 
choose  the  location  and  the  time  for  learning.  However,  there  are  disadvantages  to  online  instruction.  These  include 
considerable  email -based  written  exchange  and  online  development  training  needs  of  the  faculty.  Both  forms  of 
instructional  delivery  will  be  included  under  the  general  term  distance  education  for  purposes  here. 

Trends  in  distance  education 

According  to  the  U.S.  Department  of  Education,  National  Center  for  Education  Statistics  (1999):  Nearly 
80%  of  public,  four-year  institutions  and  over  60%  of  public,  two-year  institutions  offered  distance  education 
courses  in  the  1997-1998  academic  year.  Overall,  U.S.  higher  education  institutions  reported  1,  661,100  students  are 
enrolled  in  distance  education  courses.  The  most  popular  delivery  technologies  used  were  asynchronous  Internet 
instruction  (58%),  two-way  interactive  video  (54%),  and  one-way  pre  recorded  video  (47%).  This  year  the  State 
University  of  New  York  (SUNY)  system  itself  enrolled  more  than  20,000  students  in  distance  learning  courses, 

7,000  more  than  last  year.  Five  years  ago  only  1 19  students  were  taking  distance  learning  courses.  It  is  estimated 
that  in  the  next  two  years  85  percent  of  American  colleges  will  be  offering  online  courses  to  over  two  million 
students  (Moore  2000). 

History  of  distance  education  at  Canton  College 

Canton  College  has  been  using  both  asynchronous  and  synchronous  methods  of  distance  learning  since 
1997.  The  college  uses  a 2-way  audio/visual  conferencing  system,  interactive  television,  to  send  and  receive  classes 
to  and  from  other  state  colleges.  More  than  2,000  students  have  participated  in  course  offerings  that  have  included 
Companion  Animal  Behavior,  Criminal  Law,  Business  Communications  and  Wine  Tasting.  Recently,  the  college 
has  begun  an  ambitious  project  with  China  utilizing  interactive  video  to  broadcast  courses  in  English  Conversation 
to  a class  of  twenty  Beijing  high  school  students.  Canton  also  participates  in  the  SUNY  Learning  Network,  an 
umbrella  organization  offering  online  courses  at  forty-five  SUNY  institutions  (see  Appendix  A). 

PART  B:  PURPOSE  AND  OUTLINE  OF  PAPER 

It  is  my  intent  to  study  distance  education  at  the  State  University  of  New  York  at  Canton  College, 
juxtaposing  viewpoints  from  a humanities  and  liberal  arts  perspective  with  those  of  a science,  medical  and  technical 
perspective.  It  is  hoped  that  such  an  analysis  will  provide  an  interdisciplinary  context  for  understanding  faculty 
concerns  and  a framework  for  applying  current  educational  research  to  specific  faculty  issues. 

Why  this  thesis? 

I developed  this  thesis  as  a result  of  witnessing  the  many  skirmishes  and  debates  among  fellow  faculty 
members  over  the  use  and  value  of  distance  education.  I listened  as  faculty  expressed  their  concerns  over  distance 
education  and  deliberated  its  merits.  While  many  of  my  colleagues  expressed  a substantial  amount  of  opinion,  I 
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heard  little  research  applied  to  such  concerns.  If  faculty  members  believe  that  student  learning  is  compromised  by 
distance,  I wanted  to  know  whether  or  not  educational  research  supports  this  claim.  I was  also  curious  about  the 
pattern  of  opinion  I observed  at  the  college.  It  seemed  that  those  faculty  representing  the  liberal  arts,  writing  and 
humanities  curricula  were  quite  vocal  against  distance  education,  while  those  members  of  the  science,  medical  and 
technical  aspect  of  the  campus  appeared  either  nonaligned  or  willing  to  explore  the  use  of  distance  education  in  their 
courses.  I wondered  if  the  various  schools  of  education  at  Canton  College  shared  any  common  ground  of  concerns, 
or  did  any  one  faction  harbor  specific  concerns  while  others  did  not.  More  importantly,  could  I explore  the  merit  of 
these  issues  through  the  use  of  educational  research?  The  answers  to  these  questions  form  the  basis  of  my  paper. 

Biases  Acknowledged 

I think  it  important  to  acknowledge  my  personal  background  and  biases  regarding  distance  education.  I 
have  been  instructing  one  course  via  three  different  instructional  modalities,  two  of  which  are  taught  in  a distance 
learning  format.  In  the  fall  semester,  I teach  the  course  on  campus,  while  in  the  spring  and  I teach  the  course  via 
interactive  video  and  the  Internet.  As  a result  of  this  experience,  I am  keenly  aware  of  issues  surrounding  distance 
education,  both  political  as  well  as  pedagogical,  and  understand  the  concerns  that  many  faculty  members  have  on 
campus  regarding  its  use  in  higher  education.  However,  I feel  such  experience  has  only  made  me  more  willing  to 
explore  research  into  this  area  of  education.  I believe  that  my  experience  with  distance  education  will  not  inhibit  the 
execution  of  my  thesis,  rather,  I hope  it  will  provide  a knowledgeable  context  to  record  faculty  concerns  and  a 
dispassionate  vehicle  for  applying  educational  research  to  these  concerns. 

How  this  project  was  conducted 

The  intent  of  this  project  was  not  to  engage  in  rigorous  scientific  study,  but  rather,  to  explore  general 
perceptions  regarding  distance  education  and  to  provide  a cross-sectional  representation  of  faculty  concerns  at 
Canton  College.  1 divided  the  academic  schools  at  Canton  College  into  two  fields  of  interest:  general  studies 
(sociology,  psychology  and  English)  and  science,  health  and  technology.  Three  representatives  from  each  respective 
school  were  then  asked  to  participate  in  personal  interviews.  Faculty  members  with  varying  degrees  and  experience 
in  distance  education  were  selected.  An  honest  effort  was  made  to  objectively  choose  faculty  without  a reference  to 
their  beliefs;  however,  I must  acknowledge  that  I did  choose  those  faculty  I knew  to  be  abundant  in  their  opinions  so 
as  to  obtain  as  much  data  as  possible.  Each  faculty  member  during  the  interview  was  asked  to  discuss  particular 
issues  or  concerns  that  they  had  related  to  distance  education  and  to  prioritize  such  issues.  Educational  research, 
including  published  formal  studies  and  journal  reports,  was  then  applied  to  specific  faculty  concerns  regarding 
distance  education. 
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PART  C:  FACULTY  OPINIONS  ON  DISTANCE  EDUCATION 
Overview  of  Faculty  Outlook 

Currently,  one  in  10  higher  education  National  Education  Association  (NEA)  members  teaches  a distance 
education  course  (NEA,  2000,  4).  The  Association’s  June  2000  report,  “A  Survey  of  Traditional  and  Distance 
Learning  Higher  Education  Members”  provides  statistical  information  regarding  faculty  outlook  towards  distance 
education.  Among  the  findings: 

H 51%  of  traditional  faculty  hold  positive  feelings  toward  distance  education  courses,  compared 

to  22%  who  hold  negative  feelings.  A significant  proportion,  (28%),  of  traditional  faculty 
remain  undecided. 

H Among  distance  learning  faculty,  72%  hold  positive  feelings  compared  to  only  14  % who  hold 

negative  feelings. 

United  University  Professions  (UUP),  a union  comprising  24,000  academic  and  professional  faculty  from 
29  campuses,  conducted  a similar  poll.  Members  were  asked  about  the  issue  of  distance  learning  within  the  State 
University  of  New  York  (SUNY)  system.  Published  results  from  a poll  of  500  UUP  members  indicated  that 
members  were  overwhelmingly  receptive  to  the  use  of  distance  learning  but  expressed  a strong  skepticism  about  its 
quality,  effectiveness  and  impact  on  their  profession  (Scheurman  2000,1 1): 

H 72  percent  of  respondents  believe  distance  learning  will  mean  more  work  for  the  same 

pay. 

H 73  percent  of  respondents  worry  about  their  intellectual  property  rights. 


ilio  BEST  COPY  AVAILABLE 


621 


H 68  percent  of  respondents  do  not  believe  that  distance  learning  courses  provide  the 

same  quality  as  traditional  courses. 

Faculty  Outlook  at  Canton  College 


The  debate  between  the  new  and  the  old,  technology  and  traditional,  has  been  going  on  since  the 
days  of  Plato  and  Socrates,  Socrates,  versed  in  the  traditional  art  of  oral  education,  refuted  the 
written  expression,  the  new  educational  format  of  his  student,  Plato,  Such  an  analogy  parallels  the 
resistance  of  traditional  pedagogy  to  new  educational  delivery,  distance  education  (Horn  2000). 


Ultimately,  the  success  of  any  distance  education  program  rests  with  the  faculty.  In  general,  Canton 
College  faculty  expressed  concerns  ranging  from  pedagogical  and  student  achievement  issues,  to  those  of 
assessment,  compensation  and  workload.  Faculty  seemed  most  concerned  with  issues  of  quality  learning  and  less 
concerned  with  issues  of  copyright  and  compensation.  Although  issues  of  workload,  compensation  etc.  were 
important  to  Canton  faculty,  they  seemed  less  important  than  the  issues  of  quality  instruction  and  student  learning. 
Interestingly,  this  observation  somewhat  parallels  the  1999  NEA  findings  in  which  polled  faculty  reported  that  they 
evaluated  distance  education  primarily  on  quality  of  education  and  secondarily  on  more  traditional  union 
considerations  (NEA,  2000,  35). 

Interview  Results 

Interview  results  reflecting  individual  faculty  concerns  regarding  distance  education  and  comments  related 
to  those  concerns,  can  be  found  in  Appendix  B. 

PART  D:  RESEARCH  APPLIED  TO  FACULTY  CONCERNS 
Concern:  Student  Achievement 

Of  primary  concern  to  the  faculty  at  Canton  College  is  the  question  of  student  learning.  Do  distant  students 
learn  as  much  as  students  receiving  traditional  face-to-face  instruction?  Faculty  members  at  Canton  College  were 
passionate  in  their  belief  that  student  achievement  is  significantly  lower  in  a distance  education  than  it  is  in 
traditional  instruction.  Many  faculty  members  argued  that  students  do  not  learn  as  much,  as  well,  or  as  effectively  in 
a distance  learning  format  as  they  do  in  a traditional  learning  format. 

However,  hundreds  of  general  media  comparison  studies  conducted  over  the  last  forty  years  have 
demonstrated  no  significant  difference  in  achievement  levels  between  distant  and  traditional  learners  (Cyrs  1997,4). 
Most  of  this  research  has  generally  been  in  the  area  of  instructional  television.  Studies  by  Crow  (1977)  “found  that 
there  was  no  significant  difference  in  student  achievement  regardless  of  the  proximity  of  the  instructor.’’  Dubin  and 
Travaggia’s  (1968)  longitudinal  study  demonstrated  “clearly  and  unequivocally  that  there  is  no  measurable 
difference  among  truly  distinctive  methods  of  college  instruction  when  evaluated  by  student  performance.” 
Whittington  (1989),  in  his  review  of  research  literature  on  distance  education  concluded  that  “ findings  of  equivalent 
student  achievement  hold  even  when  rigorous  methodological  research  standards  are  applied.”  According  to 
DeLoughry  (1998),  students  learn  as  much  and  as  well  in  computer-mediated  instruction  as  they  do  in  traditional 
educational  settings.  DeLoughry  cites  a study  conducted  by  researchers  at  the  New  Jersey  Institute  of  Technology 
to  support  his  claim: 


Researchers  tested  the  effectiveness  of  online  instruction  by  studying  five  courses  in  which  a 
total  of  98  students  were  enrolled  for  classroom-based  instruction  and  80  others  were  instructed 
online.  A comparison  of  the  average  test  and  course  grades  for  the  two  groups  in  each  course 
turned  up  no  statistically  significant  differences  between  the  experimental  and  control  groups. 

Research  has  also  demonstrated  in  general  that  students  learn  at  a distance  as  well  and  as  effectively  as 
students  in  traditional  face-to-face  classrooms  (Cyrs  1997,4).  The  following  excerpt  from  Clark  (1983)  summarizes 
the  influences  of  media  on  student  learning: 


Media  comparison  smdies  clearly  suggest  that  media  do  not  influence  learning  under  any 
conditions.  The  best  current  evidence  is  that  media  are  mere  vehicles  that  deliver  instruction  but 
do  not  influence  student  achievement  any  more  than  the  truck  that  delivers  our  groceries  causes 
changes  in  our  nutrition.  Basically,  the  choice  of  vehicle  might  influence  the  cost  or  extent  of 
distributine  instruction,  but  onlv  the  content  of  thj^  vehmlo-can  influence  achievement  (Clark 
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While  comparison  studies  may  document  education  outcomes,  they  omit  several  key  concepts  regarding 
student  learning.  First,  comparison  studies  do  little  to  address  learner  experience,  i.e.,  student  satisfaction  levels  and 
student  attitudes.  In  essence,  they  ignore  the  role  of  social  presence  in  student  learning.  Recent  educational  research 
on  distance  learning  has  recognized  this  deficiency  and  is  now  shifting  the  focus  from  comparative  studies  to  theory 
-based  studies.  Theory-based  studies  explore  the  social  aspects  of  student  learning.  Fahab  (2000)  outlines  the 
following  research  findings: 


^ Fulford  and  Zhang  (1993)  evaluated  learner  perception  of  interaction  in  instruction.  Their  research 
findings  concluded  that  overall  interaction  dynamics  may  have  a stronger  impact  on  learners’  satisfaction 
than  strictly  personal  participation. 

^ McDonald  and  Gibson  (1998)  explored  interpersonal  interaction  and  group  development  in  an 
asynchronous  distance  education  environment.  Their  studies  concluded  that  students  are  capable  of 
resolving  interpersonal  issues  and  form  organized,  cohesive  working  groups. 

^ Gunawardena  (1995,  p.  164)  studied  the  social  presence  theory  for  community  building  in  computer 
mediated  conferencing.  She  concluded  “in  spite  of  the  low  social  context  cues  of  the  medium,  student 
perceptions  of  the  social  and  human  qualities  of  the  medium  depends  on  the  social  presence  created  by  the 
instructors  and  the  online  community.” 

^ Tsui  and  Ki  (1996)  studied  school  factors  affecting  computer  mediated  instruction.  Their  study  revealed 
that  communication  among  students  was  bilateral,  as  students  often  hesitated  to  enter  a dialog  started  by 
two  other  students. 


Recent  studies  by  Kanuka  and  Anderson  (1998)  also  address  the  social  process  of  learning  in  a distance 
education  environment.  After  studying  the  interaction  among  participants  in  a distance  education  forum,  both 
researchers  determined  that  a “significant  time  was  engaged  in  social  interchange  followed  occasionally  by  social 
discord.  Social  discord  served  as  a catalyst  to  the  knowledge  construction  process.” 

Secondly,  although  learning  outcomes  and  student  achievement  levels  are  documented  in  media 
comparison  studies,  “quality  outcomes”  are  not.  What  are  quality  outcomes?  Do  they  simply  imply  satisfied  learners 
or  are  they  outcomes  that  demonstrate  that  students  have  achieved  cognitive  skills  not  previously  possessed  prior  to 
their  distance  learning  experience?  Comparison  studies  would  do  better  to  qualify  outcomes  in  their  analysis. 

Thirdly,  although  these  comparison  studies  have  addressed  distance  education  as  a whole,  they  have  yet  to 
explore  the  intricacies  and  particular  features  of  each  of  the  various  distance  learning  modalities.  Little  attention  has 
been  given  to  specific  media  features  and  how  such  features  contribute  to  learner  outcomes.  For  example,  how  does 
a particular  computer-mediated  medium  that  uses  audiographics  contribute  to  student  interaction?  Michael  G. 

Moore,  in  his  editorial  on  the  1995  Distance  Education  Research  Symposium,  brings  this  issue  to  focus: 


We  must  begin  to  look  at  the  distinguishing  characteristics  of  these  different  settings.  A further  step  is  to 
ask  how  these  contexts  affect  the  learners,  and  their  learning,  the  learning  experience,  process  and 
outcomes  (Moore  1995). 

It  would  be  a judicious  for  comparison  studies  to  explore  learner  characteristics  as  well.  As  distance 
education  moves  from  a marginal  to  integral  role  in  higher  education,  learner  characteristics  and  their  role  in  the 
process  of  learning  will  become  increasingly  important 
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(Powell,  Conway,  Ross,  1990).  Traditionally,  successful  distance  learning  students  have  been  identified  as 
older,  working  students  possessing  time  management  skills,  high  self-motivation,  a positive  attitude,  and  risk  taking 
personalities.  Demographic  changes,  however,  will  eventually  alter  learner  characteristics.  Characteristics  of  success 
that  have  previously  defined  successful  distance  learning  students  may  not  be  characteristic  of  a new  distance 
education  population. 


Concern:  Distance  Education  as  Big  Business 


Several  faculty  members  at  Canton  College  expressed  their  concern  that  distance  education  has 
increasingly  become  a consumer  commodity,  rather  than  a vehicle  of  education.  They  believe  profit  is  the  primary 
motivating  factor  in  the  implementation  of  distance  education  in  colleges  and  universities  across  the  country.  A 
recent  editorial  in  AFT  ON  CAMPUS,  November  2000  supports  this  conclusion:  “Wall  Street  is  waging  huge  sums 
on  the  convergence  of  education  and  the  Internet.  There’s  e-commerce,  now  there’s  e-leaming”  (Gladieux  2000,10). 
Wall  Street’s  interest  in  online  higher  education  has  spurred  college  administers  interest  in  online  education.  Green 
(2000)  cites  a recent  article 'm  Forbes:  “[Business-to-consumers}  is  yesterday’s  story.  Education-to-business  and 
education-to-consumers  is  tomorrow’s.” 

Traditionally  private,  for-profit  companies  occupied  the  technical  training  or  blue-collar  aspect  of  the 
distance  education  market.This  is  changing.  For-profit  education  companies  such  as  HungryMinds,  SmartPlanet, 
eHigherEduation,  SmartForce  and  eLearning  are  now  competing  with  colleges  and  universities  for  the  market  share 
of  distance  education  programs  in  continuing  higher  education.  SmartForce  reported  sales  worth  of  $1 89  million  in 
1999;  eLearning  generated  revenues  of  nearly  $100  billion  (AFT  ON  CAMPUS  2000). 

Increasingly,  these  same  for-profit  institutions  are  also  competing  with  colleges  that  offer  credit  courses 
and  degrees.  The  response  of  public  institutions  to  the  incursion  of  these  companies  has  been  twofold:  colleges  are 
now  forming  partnerships  with  such  companies,  or  they  entering  the  for-profit  field  themselves.  (Moore  2000).  The 
University  of  Colorado,  for  instance,  has  formed  a partnership  with  eCollege,  the  large  for-profit  education  industry 
(see  Appendix  C).  New  York  University  has  created  its  own  for-profit  educational  company,  NYU  Online.  The 
University  of  Phoenix,  with  a student  population  of  at  least  56,000,  is  the  largest  for-profit  university  in  the  US. 
Other  corporate  alliances  and  educational  institutions  include  California  State  University  and  the  publishers  Simon 
& Schuster  likewise,  the  University  of  Washington  has  entered  a partnership  with  Prentice  Hall. 

Consider  the  following  Internet  headings: 


going  tfi 
based  fo 


► REPRESENTATIVES  OF  NEW  YORK  UNIVERSITY’S  ONLINE  DEGREE 
PROGRAM  are  talking  to  agencies  about  NYU  Online’s  estimated  $10  million  budget 
{Brandweek  Online  2000). 

► COLUMBIA  UNIVERSITY  ESTABLISHES  NEW  COMMERCIAL  VENTURE  IN 
ONLINE  EDUCATIONAL  RESOURCES  Commercial  business  will  be  used  to  develop 
online  courses  and  build  strategic  alliances  with  the  most  competitive  Internet  related  businesses 
{Chronicle  of  Higher  Education  2000). 

» T-LEARNING  EXPERIENCE* 

Universities  2 1 , a network  of  1 8 prominent  universities  in  1 0 countries,  announced  plans 
Monday  to  develop  online-learning  materials  with  Thomson  Learning  — a division  of  the 
Thomson  Corporation,  an  international  company  focusing  on  electronic  delivery  of  information 
{Chronicle  of  Higher  Education  2000). 

» WHAT  MAKES  A ^DOT>EDU7 

Community  colleges  have  stepped  up  their  fight  for  the  right  to  use  World  Wide  Web  addresses 
ending  in  ”.edu"  — an  increasingly  coveted  distinction  as  colleges  and  companies  compete  for 
distance-education  students  online.  {Chronicle  of  Higher  Education  2000). 

hey  are 
i web- 
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spheres  of  influence  (Moore  1999).  Consequently,  external  competition  has  forced  many  colleges  and  universities  to 
enter  the  global  educational  market.  Institutions  are  adopting  strategies  to  ensure  competitiveness;  these  strategies 
include  distance  education. 

Education  on  demand  will  dominate  the  marketplace  and  those  institutions  that  can  adapt  to 
those  changes  will  thrive.  Those  that  do  not  may  find  themselves  out  of  business  (Olcott 
1996). 


Just  what  are  the  implications  of  merging  distance  education  with  big  business? 

Michael  Moore,  in  his  editorial  on  distance  education  and  big  business,  expressed  the  following  concerns: 


1.  Distance  education  will  contribute  to  widening  the  issue  of  inequity.  The  traditional  spirit  of  distance 
education  was  to  provide  access  to  those  people  denied  opportunity  to  conventional  education.  Access 
today  to  the  new  distance  education  requires  access  to  the  new  technology.  “Americans  living  in  rural 
areas  are  still  behind  those  in  urban  areas  in  Internet  access. ’’(Moore  2000). 

2.  Distance  education  will  align  itself  to  the  principles  of  business.  Investments  will  be  targeted  in  the  rich 
markets  of  the  health  industry,  business  and  information  technology.  Continuing  education  needs  of  the 
rest  of  society  may  not  be  addressed. 

3.  Price  wars,  common  in  the  private  sector  and  industry,  will  infiltrate  education.  Will  institutions  that 
undercut  each  other’s  price,  undercut  quality  as  well? 


Some  question  whether  school  itself  has  been  transformed  into  a lucrative  market  (Apple).  In  “Digital 
Diploma  Mills”  David  Noble  condemns  distance  learning  as  the  “degradation  of  the  quality  of  education  in  pursuit 
of  the  dollar”  and  maintains  that  dollars  not  pedagogical  interest,  are  behind  these  partnerships  (Scheurman 
2000,1 1).  Noble  views  distance  education  as  “the  biggest  threat  ever”  to  institutions  and  faculty. 

Many  schools  are  unaware  of  the  risks  of  associated  with  moving  online.  Schools  rush  online  to  increase 
revenues  and  for  fear  of  being  left  behind  in  the  competitive  market.  Today,  75  percent  of  two -and  four-year 
colleges  offer  some  form  of  online,  and  the  number  is  expected  to  increase  to  90  percent  by  next  year  (Green  2000). 
This  represents  a dilemma  for  smaller  colleges  and  universities.  Smaller  schools  offering  online  courses  and  degrees 
will  have  to  compete  with  the  larger,  more  prestigious  schools  like  Duke  and  Harvard  University  for  students. 
Colleges  and  universities  will  need  to  hire  sales  and  marketing  teams  just  to  attract  students  to  programs.  “Marketing 
will  become  more  costly  as  the  field  grows  and  name  recognition  becomes  ever-more  difficult  to  establish  “ (Green 
2000).  Software  development  companies  can  spend  up  to  $ 1 million  developing  a single  course.  As  more  schools 
adopt  the  gold-rush  mentality,  questions  exist  who  will  survive  in  a crowded  market  have  fallen  by  the 
wayside.”(Green  2000). 

In  Joshua  Green’s  essay  “The  Online  Education  Bubble”  Green  cautions  college  administrators  to  be  wary 
of  rushing  online  in  a capricious  and  fickle  Internet  market: 


Many  schools  seem  blissfully  unaware  of  the  risk  associated  with  moving  online.  Online 
education  could  be  the  latest  in  a string  of  over  hyped  Internet  concepts  in  which  an  excess  of 
giddy  supply  overestimates  the  demand  (Green  2000). 


College  administrators,  like  businessmen,  view  technology  a s a vehicle  for  cutting  costs  and  generating 
revenues.  Technology  means  productivity.  David  Noble,  an  outspoken  critic  of  distance  education,  believes  online 
courses  serve  as  “a  potential  means  of  generating  revenue  for  universities  while  cutting  labor  costs  to  the  core  {ON 
CAMPUS  2000).  However  the  data  suggests  that  many  industries  experience  performance  enhancement  with 
technology  rather  than  an  increase  in  productivity  or  profitability  with  technology  (Fahey  1998).  Colleges  and 
universities  are  not  immune  to  this  productivity  paradox.  According  to  William  Scheuerman,  president  of  UUP  and 
American  Federation  of  Teachers  (AFT)  vice  president:  “You  can’t  use  the  technology  as  an  end  to  save  a buck. 
Once  education  is  driven  by  savings,  savings  become  the  goal  and  education  slips  to  a subordinate  position 
(Scheuerman  2000,  1 1). 
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Concern:  Appropriate  Pedagogy 


Canton  College  faculty  expressed  considerable  concern  regarding  to  pedagogical  issues  and  distance 
education.  Many  of  the  traditional  faculty  that  were  interviewed  for  this  paper  characterize  their  teacher  role  as  one 
of  “content  provider”  and  “skilled  facilitator”  whose  primary  teaching  methodology  relies  heavily  upon  personal 
interaction  with  students.  Faculty  members  believe  that  distance  education,  in  either  its  synchronous  or 
asynchronous  form,  is  impersonal,  lacking  the  impact  of  direct  contact  with  students.  Many  view  distance  learning 
as  a poor  substitute  for  “face-to-face”  instruction. 

At  the  heart  of  this  pedagogical  debate  is  the  issue  of  student-to -student,  student-to-content  and  student-to- 
instructor  communication.  Interaction  is  vital  to  the  learning  process  (Bruffee  1982  & Flanders  1970).  It  provides  a 
dialogue  and  a framework  for  confronting  ideas  and  negotiating  meaning.  Pedagogy  that  enhances  communication 
and  social  presence  ultimately  promotes  effective  instruction.  Therefore,  strategies  that  encourage  student 
involvement,  provide  individual  feedback,  and  promote  interpersonal  relationships  do  best  to  foster  interaction. 
Qualities  such  as  voice  variation,  self-confidence,  stage  presence  and  mastery  of  information  are  equally  as 
important.  While  all  of  these  are  essential  characteristics  of  an  effective  and  engaging  instructor,  they  become  even 
more  critical  in  a distance  environment.  Faculty  question  whether  such  qualities  and  methodology  can  effectively  be 
transferred  to  an  environment  in  which  student  and  instructor  are  separated  by  distance. 

It  has  been  argued  that  it  is  the  essential  role  of  the  teacher  to  guide  and  monitor  students  through  the 
learning  process.  Garris  on  (1990)  maintains  that  students  need  interaction  with  the  teacher  in  order  to  “question  and 
challenge  pre-existing  viewpoints  and  validate  the  knowledge  gained.”  Streibel  (1998)  believes  student  require  an 
instructor’s  help  in  formulating,  understanding  and  solving  problems.  If  interaction  is  so  critical  to  the  educational 
process,  how  is  this  achieved  in  distance  learning  settings? 

Much  of  the  educational  research  surrounding  distance  education  has  been  devoted  to  documenting 
learning  outcomes,  and  less  effort  applied  to  understanding  the  essential  nature  of  an  educational  learning 
experience.  According  to  Wong  (1987,p.9),  the  “focus  of  much  institutional  effort  in  distance  education  has  been 
directed  toward  the  packaging  and  delivery  of  knowledge  for  the  independent  adult  learner...  [and]  little  attention 
has  been  paid  to  the  nature  of  the  human-to-human  and  human-to-machine  interactions  in  the  learning  process. 
Institutions  emphasize  putting  courses  online  rather  than  the  choice  of  appropriate  pedagogy  (Winograd  2000).  As 
Eastmond  (cite)  points  out,  “process  fall  short  of  theory.” 

Furthermore,  educational  research  has  not  evaluated  the  effectiveness  of  interactive  strategies  that  are 
currently  being  promoted  and  used  by  distance  learning  instructors.  Chat  rooms,  discussion  forums  for  students,  are 
commonly  employed  as  an  interactive  tool  promoting  communication  among  classmates.  While  chat  rooms  do 
provide  students  with  the  vehicle  to  discuss  ideas  among  themselves,  they  may  also  lead  to  the  reinforcement  of 
mistakes  and  misunderstandings  of  course  materials.  Interactive  strategies  would  do  best  if  framed  by  educational 
research. 

Other  questions  persist  regarding  appropriate  and  effective  teaching  strategies  in  a distance  learning 
environment.  How  do  instructors  achieve  elaborate  encoding  (i.e.,  the  conscription  of  learning  in  a meaningful  way 
in  their  distance  courses)?  Some  educators  recommended  that  frequent,  short  quizzes  with  scores  and  comments 
displayed  assist  in  elaborate  encoding  and  reinforcement  {Syllabus  2000).  Once  again  these  may  be  worthwhile 
teaching  strategies,  but  studies  validating  these  findings  are  lacking. 

Concern:  Student  dropout  rate 

Many  Canton  faculty  associate  distance  education  with  high  attrition  and  believed  that  student  profiles  are 
the  most  likely  factors  influencing  course  completion.  Research,  however,  indicates  multivariate  reasons  for  student 
drop  out.  While  descriptive  analysis  points  to  learner  characteristics  as  a pivotal  factor  in  student  attrition  (Rekkedal 
(1993,  19),  learner  characteristics  are  just  one  part  of  the  puzzle.  Kennedy  and  Powell  (1976,  61)  reported  that 
students  may  drop  out  of  distance  education  courses  because  of  academic  intimidation  and  the  fear  the  lack  of  ability 
“to  learn  to  debate  and  communicate  in  a manner  which  is  acceptable  to  the  academic  community”  (Cookson  24). 
Recent  research  using  a micro -sociological  approach  to  analysis  suggests  a combination  of  student  characteristics 
and  life  circumstances  preclude  students  from  completing  a course: 


The  individual  part-time  student  has  a difficult  time  in  maintaining  an  equilibrium  of  pressures  within 
his  life,  pressures  arising  from  his  job,  from  his  domestic  situation,  from  his  academic  work  and  also 
from  possible  variations  in  his  own  personality  (Kennedy  and  Powell  1976  ,61). 
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Ostman  and  Wagner  (1987,  47)  studied  of  the  influences  of  demographic,  social  interaction,  psychological, 
and  institutional  variables  on  course  withdrawal  and  found  that  lack  of  time  constituted  the  most  influential  single 
predictor  of  discontinuance.” 

Concerns:  Workload,  compensation 

NEA  (2000)  findings  support  faculty  concerns  of  increased  workload  and  inadequate  compensation: 

• Over  half  (53%)  of  distance  learning  faculty  spend  more  hours  per  week  preparing  and  delivering 
their  distance  learning  courses  than  they  do  for  a comparable  traditional  course. 

• In  spite  of  spending  more  hours  on  their  distance  learning  course  most  (84%)  of  faculty  receive  no 
course  reduction.  63%  are  compensated  for  their  course  as  if  it  were  part  of  their  normal  course 
load. 

Yet,  such  data  does  not  address  the  role  providing  institutions  play  in  the  support  and  design  of  distance 
learning  courses.  Access  to  instructional  designers  and  other  support  personnel  could  greatly  reduce  time  spent 
delivering  instruction.  Likewise,  time  and  effort  is  dependent  on  the  degree  of  interaction  between  instructor  and 
student.  Courses  that  encourage  interaction  require  more  time  investment.  Workload  can  be  reflective  of  institutional 
support  as  well  as  individual  instructor  strategies. 

CONCLUSION 

Faculty  members  at  Canton  College  acknowledged  the  value  of  distance  education  in  providing  access  and 
time  flexibility  to  non-traditional  working  students.  They  were,  however,  doubtful  about  the  quality  of  distance 
education  and  concerned  about  the  impact  distance  education  may  have  on  their  profession.  Both  academic  areas  at 
Canton  College  expressed  similar  viewpoints  regarding  distance  education.  While  members  of  the  science,  medical 
and  technology  areas  of  the  college  were  the  ones  most  like  ly  to  utilize  some  form  of  technology  in  their  instruction, 
they  nonetheless  expressed  similar  concerns  regarding  distance  education  as  did  the  humanities  and  liberal  arts 
curricula.  Analysis  of  faculty  concerns  at  the  college  did  not  portray  any  one  faction  as  vehemently  for  or  against 
distance  education.  This  was  quite  the  contrary  to  the  dialogue  expressed  at  faculty  meetings.  Perhaps  the  context  of 
one-on-one  interviews  influenced  the  presentation  of  opinions. 

Many  of  the  concerns  that  Canton  College  feculty  expressed  during  their  interviews  are  shared  by  other 
institutions  of  higher  education  as  well.  A recent  transcript  in  the 

Chronicle  of  Higher  Education  (2000)  detailed  a discussion  between  regarding  the  impact  of  distance 
education  (specifically  online  education)  on  community  colleges.  Much  of  the  dialogue  echoed  the  issues  Canton 
College  faculty  raised  during  their  interviews.  Community  college  faculty  expressed  concerns  over  standards  for 
distance  education,  attrition  rates,  competition  choking  or  reducing  offerings,  quality  of  instruction,  workload 
release  time,  support,  faculty  rights  and  compensation. 

Faculty  members  at  the  college  believed  that  little  educational  research  has  been  applied  to  distance 
education.  This  is  not  true.  Much  inquiry  has  been  dedicated  to  quantifying  learning  outcomes.  Invariably, 
comparative  studies  of  distance  education  and  classroom  instruction  have  proven  time  and  again  no  significant 
difference  between  distance  education  learning  outcomes  and  those  of  traditional  instruction.  However,  such 
research  fails  to  address  the  complexities  of  distance  education,  doing  little  to  explore  the  theoretical  foundations  of 
the  field.  Recent  studies,  however,  have  begun  to  investigate  the  social  experience  of  distance  education, 
documenting  learner  perception,  experience  and  attitude. 

Faculty  members  question  the  pedagogical  soundness  of  distance  education.  Many  view  distance  education 
as  an  assembly -line  approach  to  education,  a medium  for  delivering  more  data  and  less  knowledge.  They  argue  that 
distance  education  hinders  interaction,  precluding  the  exchange  of  social  and  verbal  cues  between  instructor  and 
student.  Pedagogical  approaches  that  increase  interaction  and  successfully  engage  students  in  the  process  of  learning 
would  most  likely  augment  student  scholarship.  Yet  such  approaches  have  yet  to  be  informed  by  educational 
research. 

The  evidence  supports  faculty  concerns  that  distance  education  is  a vehicle  for  big  business.  Partnerships 
between  corporations  and  major  universities  are  increasingly  common.  Institutions  are  forced  to  compete  for 
students  as  never  before.  The  effect  of  such  competition  will  have  on  smaller  schools  that  have  limited  resources  and 
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visibility  remains  unclear.  Administrators  view  distance  education  as  a cost-saving  and  revenue-generating  measure. 
The  following  excerpt  from  Michael  Moore  (2000)  is  a sobering  reflection  on  the  relationship  between  distance 
education  and  the  business  model  of  productivity: 


Plans  at  the  University  of  Georgia  Board  of  Reagents  call  for  the  entire  first  two  years  of  the 
university  curriculum  to  be  available  over  the  Internet,  and  for  complete  degree  programs  in  all 
traditional  disciplines  to  be  available  by  2002.  One  curriculuniwill  be  offered  to  every  student  in  the 
state.  The  board  will  select  faculty  members  (eight  per  course)  from  throughout  the  state  university 
system  to  construct  each  core  course.  There  will  be  a designated  instructor/facilitator  for  evaluation. 
Facilitator  faculty  will  have  no  academic  freedom  regarding  the  courses  they  teach  (Moore  2000). 


PERSONAL  REFLECTIONS 

As  I listened  to  my  colleagues  discuss  distance  education  I could  not  help  but  sense  their  apprehension  over 
the  rapid  changes  in  education.  They  seemed,  however,  resigned  to  the  fact  that  higher  education,  despite  our  best 
protests  and  ideological  concerns,  was  on  a path  of  change  that  none  of  us  had  expected  or  anticipated  twenty  years 
ago. 

Distance  education  is  a reflection  of  the  changes  occurring,  not  only  in  high  education,  but  in  society  as 
well.  Technological  advancements  such  as  the  Internet  have  shaped  businesses  and  influenced  policies,  both 
governmental  and  educational.  While  distance  education  has  made  it  easier  for  people  to  access  higher  education, 
the  technology  that  drives  its  delivery  is  unavailable  to  many.  Ironically,  the  same  technology  that  seeks  to  avoid 
inequity,  may  be  the  cause  of  it. 

Distance  education  also  forces  us  to  re-evaluate  our  roles  as  faculty,  students  and  institutional  members. 
Many  in  the  educational  community  are  reluctant  to  embrace  its  concept.  They  cite  issues  such  intellectual  property, 
fair  compensation,  impact  on  quality  of  education,  increased  workloads,  adequate  compensation  for  development 
and  implementation  of  courses  and  decreased  student -learner  interaction  as  mainstays  of  their  resistance.  These 
issues  are  real,  and  yet  to  be  resolved. 

The  lack  of  sound  pedagogy  is  perhaps  the  most  critical  issue  for  me.  Suggestions  from  experienced 
distance  learning  instructors  regarding  effective  methodology  and  teaching  strategies  do  not  bear  the  weight  of 
sound  educational  research.  I believe  there  is  much  to  learn  about  the  social  learning  experience  of  students  in  a 
distance  learning  environment. 

I also  believe  that  colleges  and  universities  ignore  the  realities  of  market  demand.  Distance  education  is 
not  a substitute  for  the  budget  woes  or  fiscal  dilemmas  that  plague  our  colleges,  nor  is  it  a substitute  for  the  day-  to  - 
day  interactions  that  define  the  brick  and  mortar  experience  of  traditional  education. 

General  Studies 


Interview  #i 

Profile:  Sociology  professor  12  years 
Experience  with  distance  education:  little 


Primary  Concern 

Comments 

Loss  of  student/teacher  relationship 

Primary  role  as  a teacher  is  “sage  on  stage”. 
Teacher-centered  pedagogy.  Paid  to  interpret 
information  and  giving  information  is  “contingent 
upon  reading  my  cues  from  my  audience.” 

Other  concerns 

Comments 

Student  characteristics 

Traditional  students  seem  to  lack  self-discipline. 
How  would  such  a student  fit  into  or  succeed  in  a 
distance  learning  environment?  A campus  setting 
allows  physical  presence  of  the  instructor  at  a set 
time,  not  possible  in  distance  learning  settings. 
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Education  as  big  business 

See  institutions  as  public  service  trying  to  emulate  a 
business  model  and  make  faculty  entrepreneurs  (this 
affects  teaching). 

Time  investment,  compensation  and  intellectual 
property  rights 

Certainly  an  issue,  but  not  primary 

Interview  #2 

Profile:  Psychology  professor  17  years 

Experience  with  distance  education:  little;  experienced  with  multimedia 


Primary  Concern 

Comments 

Impersonal  process 

Little  interaction  outside  the  classroom 
Can  not  extract  the  social  process 

Allows  image  to  be  the  role  model 

Focus  not  on  student-teacher  interaction,  rather,  on 

“presentation  material” 

Virtual  skills -only  apply  in  a virtual  world 

Other  concerns 

Comments 

Quality  of  education 

Distance  education  diminishes  quality 

Emergence  of  “two  tiers  of  education” 

Virtually  educated  vs.  Traditionally 

Time  investment,  compensation  and  intellectual 
property  rights 

Certainly  an  issue,  but  not  primary 

Interview  #5 

Profile:  English  professor  20  years 
Experience  with  distance  education:  none 

Primary  Concern 

Comments 

Lack  of  student  interaction 

Face-to-face  a priority 

One-on-one  interaction-  office  hours 

Lectures  more  effective  in  delivering  instruction 

Other  concerns 

Comments 

Lack  of  training 

Would  need  a lot 

Time  investment,  compensation  and  intellectual 
property  rights 

Certainly  an  issue,  but  not  primary. 

Science,  medical  and  technology 


Interview 

Profile:  Wireless  Communication  12  years 

Experience  with  distance  education:  none,  extensive  multimedia 

Primary  Concern 

Comments 

Student  motivation  (learner  characteristics) 

Students  are  not  motivated  in  class,  less  so  in  an 
online  environment 

Other  concerns 

Comments 

Lack  of  interaction 

“Interface  tends  to  alienate” 

People  go  to  college  for  interaction-  the  college 
experience  (You  can’t  have  bear  blast  over  the 
Internet!) 
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Time  investment,  compensation  and  intellectual 
property  rights 

Certainly  an  issue,  but  not  primary. 

Interview  #5 

Profile:  Chemistry  Professor  28 years 

Experience  with  distance  education:  none,  extensive  multimedia 


Primary  Concern 

Comments 

Student  achievement 

Would  be  poor  for  traditional  18-20  year  olds. 

Other  concerns 

Comments 

Lack  of  personal  interaction,  although  this  is  course 
dependent 

Face-to-face  interaction  provides  motivation. 
Acknowledges  that  discussion  can  occur  in  distance 
education  environment 

Support  issues 

Need  adequate  communication  between  instructor 
and  (infrastructure)computing  center 

Time  investment,  compensation  and  intellectual 
property  rights 

Certainly  an  issue,  but  not  primary. 

Interview  #6 

Profile:  Biology  Professor  4 years 

Experience  with  distance  education:  none,  extensive  multimedia 

Primary  Concern 

Comments 

Lack  of  student  interaction 

Role  of  the  professor  is  to  provide  motivation, 
create  interest,  explain  concepts,  provide  direction. 
Would  be  difficult  to  transfer  this  approach  to  a 
distance  education  format 

Other  concerns 

Comments 

Student  achievement 

Believe  significantly  lowered 

Student  motivation 

‘Traditional  students  can’t  pay  attention” 

Distance  education  is  “edu-tainment” 

Traditional  education  emphasizes  content;  distance 
education  emphasizes  delivery 

Marketing 

“Who  wants  100  Introduction  to  Biology  courses?” 

Time  investment,  compensation  and  intellectual 
property  rights 

Certainly  an  issue,  but  not  primary. 

Additional  comments: 

• Lack  of  educational  research  frames  distance  education 

• Pedagogical  approach  would  be  difficult  to  transfer  to  a distance  learning  environment 

• Administration  encourages  faculty  to  produce  online  courses.  Appropriate  pedagogy  not 
addressed. 

• Validity  of  distance  education  noted:  expanded  opportunities  for  commuters  and  working  non- 
traditional  students 
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Abstract: 

In  the  fall  of  2000,  the  University  of  the  Incarnate  Word,  a small,  Catholic,  liberal  arts  university  located  in 
San  Antonio,  Texas  became  the  largest  IBM  ThinkPad  University  in  the  South.  At  present,  2000  laptops  have  been 
distributed  to  students  and  faculty.  This  paper  will  explain  the  implementation  process  and  the  components  that 
made  this  a successful  program.  Commitment  from  the  University  leadership,  involved  corporate  partners,  a broad- 
based  planning  team,  effective  communication,  faculty  training  and  a supportive  infrastructure  lead  to  a positive  first 
year  experience  for  students.  The  paper  discusses  problems  that  were  encountered  and  makes  recommendations  to 
institutions  that  might  be  considering  such  an  initiative. 

Introduction: 

In  the  fall  of  2000,  the  University  of  the  Incarnate  Word  became  the  largest  IBM  ThinkPad  University  in 
the  South.  At  present,  2000  laptops  have  been  distributed  to  students  and  faculty.  This  paper  will  explain  the 
implementation  process  and  the  components  that  made  this  a successful  program.  Commitment  from  the  University 
leadership,  involved  corporate  partners,  a broad-based  planning  team,  effective  communication,  faculty  training  and 
a supportive  infrastructure  lead  to  a positive  first  year  experience  for  students. 

The  University  of  the  Incarnate  Word  (UIW)  is  a small.  Catholic,  liberal  arts  university  located  in  San 
Antonio,  Texas.  The  University  enrolls  4,283  students,  of  which  3,519  are  undergraduates.  Seventy -one  percent  of 
UlW’s  student  population  can  be  designated  as  minority.  The  student  population  reflects  that  of  the  city,  with  the 
majority  coming  from  Hispanic  (54%)  households.  Most  of  the  students  are  first-generation  college  attendees  who 
use  financial  aid  to  pay  for  their  education  (77%).  The  University  qualifies  as  a Hispanic  Serving  Institution. 

Based  on  our  student  demographics,  the  UIW  leadership  has  had  a great  concern  about  the  increasing 
“digital  divide”  between  the  Anglo  and  the  minority  populations.  In  August  2000,  the  U.S.  Commerce  Department 
reported  in  Falling  Through  the  Net:  Toward  Digital  Inclusion,  that  the  national  average  for  computer  ownership 
was  51%,  but  the  penetration  rate  was  only  33.7%  for  Hispanics  households.  The  report  indicated  that  the  most 
Hispanics  access  Internet  services  from  schools,  libraries  and  work  rather  than  from  home.  (Only  16.1  % of 
Hispanics  access  the  Internet  from  home.)  With  technology  use  patterns  such  as  this,  if  UIW  did  not  provide  the 
students  it  serves  with  access  to  technology,  who  would? 

At  the  University  of  the  Incarnate  Word,  technology  literacy  is  seen  an  important  part  of  the  liberal  arts 
core  curriculum.  Students  are  required  to  take  a course  in  computer  literacy  and  the  integration  of  technology  into 
courses  in  the  major  is  standard  instructional  practice.  For  many  years,  there  has  been  budgetary  support  for 
infrastructure  improvements,  the  creation  of  computer  labs  and  for  technology  skill  development  among  faculty. 

Still,  it  had  been  difficult  to  keep  up  with  the  increasing  demand  for  access  to  technology.  The  student  to  computer 
ratio  was  high,  there  were  not  enough  staff  to  keep  labs  open  24/7  and  there  was  the  expense  of  refreshing  the 
technology  every  3 years.  Requiring  students  to  have  laptops  seemed  like  a good  solution  to  these  problems.  The 
question  raised,  however,  was  how  could  the  program  be  affordable  for  our  students  who  already  have  trouble 
paying  tuition? 

Initial  Planning 

In  the  fall  of  1999,  a Laptop  Planning  Team  made  up  of  faculty,  administrators,  staff  and  students  was 
charged  with  investigating  the  potential  of  a pilot  program  that  mandated  laptop  purchases  for  all  junior  and  senior 
business  majors  (500  students).  The  Dean  of  the  Business  School  was  trying  to  respond  to  a recent  employer 
survey,  which  indicated  that  employers  considered  technology  skills  a major  factor  when  hiring  business  school 
graduates.  The  Laptop  Planning  Team  identified  three  goals  for  the  program:  (1)  put  standardized  technology  into 
the  hands  of  the  students;  (2)  embed  technology  into  the  learning  experience;  and  (3)  give  students  the  required 
technological  skills  to  make  them  successful  after  graduation. 
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Initially,  the  team  used  the  Internet  and  telephone  interviews  to  survey  other  universities  that  had  laptop 
programs.  The  Dean  and  the  Chief  Information  Officer  visited  the  small,  Catholic  St.  Gregory’s  University  in 
Shawnee,  Oklahoma,  which  has  had  laptop  program  since  1997.  Major  laptop  vendors  (IBM,  Toshiba,  Dell, 
Gateway,  Compaq)  were  asked  to  do  presentations.  The  Laptop  Planning  Team  prepared  a report  for  UIW’s 
President.  After  reviewing  the  recommendations,  he  decided  to  expand  the  pilot  beyond  the  business  students  to 
ALL  fulltime  sophomores  and  juniors,  an  estimated  800  students  in  the  Fall  2000. 

Through  a competitive  procurement  process,  in  which  a formal  request  for  proposal  was  prepared  and  sent 
to  each  of  the  five  major  vendors,  IBM  was  selected  as  UIW’s  corporate  partner.  IBM’s  proposal  included  leasing 
the  laptop  computers  over  a 3 -year  period,  technical  support  for  broken  computers,  and  insurance  in  case  of  theft. 
The  laptop  chosen  was  sufficiently  robust  to  connect  to  the  campus  network  and  to  run  standardized  productivity 
software.  IBM  offered  additional  support  in  sharing  their  experience  with  other  ThinkPad  universities.  They 
provided  such  assistance  as  sample  letters  to  students  and  parents,  examples  of  laptop  orientation  materials  and 
information  on  policies  and  strategies  used  by  other  schools.  In  addition,  IBM  representatives  became  a part  of  our 
Laptop  Planning  Team.  Our  IBM  representative  drove  down  from  Austin  every  two  weeks  to  attend  the  team 
meetings. 

The  team  determined  that  each  student  would  receive  a laptop,  a printer  and  a backpack.  This  would 
eliminate  the  need  to  install  networked  printing  stations  around  campus.  In  addition,  each  laptop  would  have  a 
standard  software  installation.  A Microsoft  Campus  site  license  was  purchased  so  that  faculty  could  expect  all 
students  to  have  the  basic  productivity  tools.  Standardizing  hardware  and  software  would  allow  the  technical 
support  staff  to  limit  the  problems  that  typically  occur  when  dealing  with  differing  configurations.  The  need  for 
more  technical  staff  was  factored  into  the  plan.  The  team  estimated  the  cost  of  the  lease  and  the  necessary  support 
to  be  about  $1,100  per  year.  The  University  leadership  made  a major  commitment  to  finance  the  program  by 
providing  a $300  technology  grant  each  semester  so  that  students  had  to  only  pay  $500  peryear  for  their  laptop.  At 
the  end  of  3 years,  students  would  be  able  to  purchase  the  laptop  for  $1.  IBM  has  reported  that  UIW  has  the  lowest 
priced  ThinkPad  University  program  in  the  United  States. 

The  University  Board  of  Trustees  approved  the  plan  in  the  spring  of  2000  for  implementation  that  fall. 
Getting  the  word  out  to  the  students  about  the  program  was  a challenge.  Students  were  informed  through  various 
means,  including  direct  mail,  student  forums  and  via  a web  site  (http://www.uiwtx.edu/'-'Laptop).  A Laptop  Appeals 
committee  was  established  to  deal  with  students  who  felt  that  they  should  be  exempt  from  the  program.  Currently, 
all  recruitment  literature  has  been  revised  to  include  the  laptop  program  and  the  University  extensively  advertises 
itself  as  an  IBM  ThinkPad  University. 


Laptop  Deployment 

During  the  summer  months  of  the  first  year,  the  laptop  orientation  session  was  planned.  It  was  planned  as  a 
2-hour  session  where  students  would  be  given  their  laptops,  check  inventory,  sign  the  leasing  agreement,  plug  in  and 
connect  to  the  network,  set  up  their  machine  for  networking  and  e-mail  and  informed  of  computer  policies  and 
services.  The  sessions  would  be  held  in  a wired  lecture  hall  that  held  65  students.  Our  intent  was  to  keep  the 
distribution  session  small  so  that  there  would  be  enough  technical  support.  Students  had  to  register  for  the  sessions 
at  the  laptop  web  site.  The  orientations  were  scheduled  for  the  week  before  school  started.  Unfortunately,  due  to  a 
delay  in  manufacturing  the  new  ThinkPad  model,  these  sessions  had  to  be  delayed  until  mid-September.  Students 
were  contacted  by  mail  and  by  phone  concerning  the  delay.  To  make  up  for  the  situation,  IBM  agreed  to  provide  a 
wireless  card  for  free  with  each  of  the  laptops. 

The  laptops  were  distributed  in  18  orientation  sessions,  which  spread  over  a 4-week  period. 

Representatives  from  the  technology  services,  financial  aid,  business  office,  student  body  and  IBM  manned  the 
distribution  stations  and  served  as  technical  support.  One  lessoned  the  team  learned  was  that  not  all  students  are 
willing  to  pick  up  their  laptops,  even  though  they  are  being  charged  for  them.  Many  missed  the  sessions  for  which 
they  had  registered.  Therefore,  additional  small  orientations  had  to  be  scheduled.  There  were  also  those  who 
delayed  until  their  appeal  to  the  Laptop  Appeals  Committee  was  heard. 

Preparing  the  Faculty 

The  student  laptop  distribution  was  only  part  of  the  University  plan.  During  the  first  year  of  the  program, 
all  138- fulltime  faculty  received  laptops.  A Hispanic  Serving  Institution  Title  V grant  allowed  UIW  to  offer 
extensive  technology  training  to  faculty  during  the  summers  of  2000  and  2001 . More  than  50%  of  the  faculty  have 
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participated  in  the  Title  V training.  On-going  training  during  the  regular  school  year,  a technology  fellowship 
program  and  instructional  technology  support  staff  have  helped  faculty  to  integrate  the  laptops  into  their  teaching. 

Another  means  of  pushing  integration  has  been  the  implementation  of  Blackboard  5 (Level  One),  a course 
management  software  package.  The  product  has  proven  to  be  easy  to  learn;  and  has  been  adopted  by  an 
overwhelming  number  of  faculty  (62%).  Three  2 -hour  workshops  introduced  60  faculty  to  Blackboard  before  the 
Spring  2001  semester.  Last  summer  40  faculty  participated  in  a weeklong  seminar,  which  focused  on  online 
pedagogy  as  well  as  provided  hands-on  experience  with  Blackboard.  At  present,  there  are  2,000  students  taking  120 
Blackboard  enhanced  courses  (an  82%  increase  from  the  spring  semester).  The  University  intends  to  migrate  to  the 
portal  solution.  Blackboard  5.5  (Level  2),  during  the  Spring  2002  semester.  Blackboard  is  changing  the 
instructional  environment,  and  the  way  in  which  instructors  and  students  communicate.  With  the  portal  product,  the 
University  expects  to  draw  the  entire  community  to  one  place  where  students,  faculty  and  staff  go  to  find 
information,  to  communicate  and  to  get  support. 

Infrastructure  & Support 

Having  2000  additional  computers  on  campus  has  forced  UIW  to  enhance  the  infrastructure  - not  only  in 
terms  of  the  network,  but  also  in  terms  of  software  and  technical  personnel.  In  the  summer  of  2000,  the  campus  was 
48%  wired;  now  it  is  100%  wired.  A decision  was  also  made  to  implement  wireless  networking  in  the  library,  in 
classroom  buildings,  and  in  student  gathering  areas.  The  University  expects  to  spend  $350,000  over  a two-year 
period  to  make  the  entire  campus  wireless.  Recently,  the  University  was  awarded  a Texas  Infrastructure  Fund  grant 
of  $100,000  to  offset  some  of  the  cost.  Shortly  after  the  laptops  were  distributed,  the  network  was  working  at  100% 
capacity.  Students  were  taking  up  all  the  bandwidth  by  downloading  music,  videos  and  making  international  phone 
calls  over  the  Internet.  The  University  has  since  had  to  add  a packet  shaper  to  prioritize  the  amount  of  bandwidth 
various  applications  use  on  the  network.  A virtual  private  network  has  been  installed  to  increase  security.  The 
additional  strain  on  the  network  has  caused  the  University  to  increase  the  number  of  T-1  lines  from  1 to  3.  Finally, 
plans  are  being  made  to  purchase  a network  sniffer  to  help  the  infrastructure  staff  keep  track  of  how  the  network  is 
being  used 

A newly  established  Technical  Support  Services  now  employs  3 fulltime  staff  and  10  student  workers  to 
support  the  laptops.  Clearly  the  number  of  staff  pales  in  comparison  with  other  institutions,  but  at  this  point,  the 
workload  is  manageable.  When  the  staff  cannot  fix  the  equipment,  it  is  sent  to  IBM  for  repair.  However,  help  desk 
hours  are  limited  to  daytime  hours  during  the  standard  workweek.  To  augment  support,  Technology  Services  is 
developing  online  resources  and  tutorials  so  that  laptop  users  can  get  help  24/7  via  the  Internet.  Building 
renovations  were  completed  to  create  a space  for  the  technicians  and  for  storage  of  laptops.  A walk  up  help  desk 
area  was  created  and  a gated  room  was  outfitted  with  wire  mesh  on  the  walls  and  ceiling  so  that  large  quantities  of 
laptops  could  be  securely  stored. 

In  addition  to  office  renovations,  changes  are  planned  for  the  classrooms.  A laptop  classroom  is  being 
created  with  a counter  that  surrounds  the  perimeter  of  the  room  with  data  jacks  and  plugs  for  25  laptops.  With 
laptops  in  the  hands  of  students  and  faculty,  the  requests  for  data  projectors  in  the  classrooms  have  increased.  This 
year  $50,000  from  the  Title  V grant  will  provide  the  University  with  projectors  for  our  main  classroom  building.  As 
new  facilities  are  planned,  projectors  and  wiring  are  included  in  the  construction  budget.  Although  the  laptops  have 
batteries,  providing  electricity  to  old  classrooms  has  become  a major  concern  from  faculty  who  want  students  to 
bring  their  laptops  to  class. 

Problems 

Of  course,  there  have  been  problems  with  the  implementation  that  were  not  anticipated.  During  the  first 
year,  student  attitudes  toward  the  program  were  not  as  positive  as  expected.  Some  students  refused  to  pick  up  their 
laptops.  Many  complained  openly  in  student  forums  and  in  the  school  newspaper.  However,  there  has  been  a 
change  in  attitude  during  the  second  year.  Students  were  much  more  excited  when  receiving  their  laptops.  As  an 
indication,  this  year  71%  of  the  freshmen  opted  to  get  a laptop  rather  than  wait  until  they  are  sophomores.  Other 
complaints  have  come  from  graduate  students  and  adult  degree  completion  program  students  who  were  not  eligible 
for  the  program  but  who  wanted  laptops.  IBM  has  now  established  a direct  purchase  program  for  these  students  as 
well  as  for  faculty,  staff  and  alumni.  Students  who  received  laptops  last  year  have  complained  because  this  year’s 
model  is  faster  and  has  more  capacity.  They  want  a way  to  return  their  model  and  get  the  newest  one.  A two-year 
refresh  program  would  help  with  this  problem,  but  the  University  is  not  prepared  to  commit  to  this  change  at  this 
time. 
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Students,  who  graduate  or  withdraw  from  UI W,  can  purchase  their  laptop.  Unfortunately,  some  students 
drop  out  without  notification  and  without  returning  the  laptop.  In  the  orientation,  students  are  warned  that  felony 
charges  will  be  filed  against  them  if  they  fail  to  return  their  laptops  upon  leaving  school.  The  practice  is  to  send 
these  students  two  letters  requesting  the  return  of  the  laptop  and  a final  certified  letter  demanding  the  laptop  back. 

The  first  year  there  were  12  such  incidents.  Our  response  was  to  file  charges  with  the  police  department  and  the 
district  attorney’s  office.  Regrettably,  the  local  district  attorney  has  decided  that  criminal  intent  was  not  evident  in 
these  cases  and  that  the  matter  should  be  handled  in  civil  court.  Complicating  matters  further,  is  the  fact  that  laptops 
are  being  leased  from  IBM,  who  actually  owns  the  laptops.  Admittedly,  the  University  does  not  yet  have  a 
successful  and  easy  solution  to  this  problem. 

The  University  has  found  a solution  for  what  to  do  with  the  returned  laptops.  The  University  has  offered 
the  leasing  option  for  the  “used”  laptops  to  departments  within  the  institution.  This  allows  them  to  spread  the  cost  of 
the  laptops  over  two  budgetary  years,  making  it  more  affordable.  The  laptops  are  insured  for  theft  and  have  the  IBM 
asset  protection  plan,  which  makes  the  offer  more  attractive.  The  adult  degree  completion  program  and  the  high 
schools  that  are  associated  with  UIW  are  planning  to  outfit  several  laptop  labs  using  these  machines. 

Lessons  Learned 

The  UIW  experience  has  much  to  offer  any  institution  that  is  interested  in  implementing  a laptop  program. 
The  UIW  Laptop  Planning  Team  could  offer  this  advice. 

1.  Form  a planning  team  made  up  of  representatives  from  all  of  the  university  constituencies  who  are 
impacted  by  the  program.  Expect  to  meet  regularly  through  out  the  year. 

2.  Research  what  other  schools  have  done  and  do  site  visits  to  schools  that  are  similar  in  size  and  budget. 

3.  Carefully  prepare  a request  for  proposals  for  laptop  vendors  that  allow  the  team  to  make  comparisons. 

Look  for  vendors  that  are  willing  to  do  more  than  just  sell  laptops  at  a good  price.  Consider  experience, 
support  and  commitment.  Look  for  a service  plan  and  a way  to  insure  the  equipment. 

4.  Standardize  the  hardware  and  software  selection  so  that  it  is  easier  to  integrate  into  the  classroom 
experience  and  so  that  it  easier  to  service. 

5.  Use  a variety  of  methods  to  inform  students  and  parents  about  the  project.  Expect  complaints  about  the 
program  and  develop  mechanisms  for  appeals. 

6.  Prepare  the  faculty  by  giving  them  the  laptops  first  and  by  providing  training  to  support  the  integration. 

7.  Expect  to  upgrade  the  infrastructure.  The  students  will  be  using  the  network  in  ways  that  will  eat  up 
bandwidth. 

8.  Expand  the  technical  support  services  on  campus.  Do  not  put  together  a plan  without  making  this  a part  of 
the  budget. 

9.  Spend  time  on  the  orientation  plans.  More  time  spent  with  the  student  during  orientation  means  less  time 
spent  on  service  problems  later. 

10.  Develop  accessible  instructional  materials  to  help  students  use  the  technology.  Do  not  expect  them  to  pick 
it  up  on  their  own  or  to  get  it  in  class. 

1 1.  Find  a killer  application  such  as  Blackboard  to  push  the  use  of  technology  into  the  classroom  and  into  the 
lives  of  the  university  community. 

12.  Plan  for  classroom  arrangements  and  technology  to  support  students  and  faculty  taking  laptops  into  the 
classrooms. 

13.  Develop  policies  to  handle  the  problems  such  as  non-retumed  laptops  and  acceptable  computer  use. 

14.  Be  patient  and  flexible.  Changes  do  not  occur  overnight  and  some  unanticipated  problem  will  always  come 
up. 

The  University  is  now  in  its  second  year  of  the  laptop  program.  Despite  the  problems  that  have  occurred, 
the  University  leadership  is  committed  to  keeping  the  program.  The  mission  of  the  institution  states  that  “The 
University  is  committed  to  educational  excellence...”  and  is  “open  to  thoughtful  innovation  that  serves  ever  more 
effectively  the  spiritual  and  material  needs  of  people.”  The  laptop  program  is  supportive  of  this  mission.  It  puts 
technology  in  the  hands  of  students,  who  otherwise  would  not  have  access.  Laptop  ownership  gives  students  the 
tools  they  need  to  complete  their  academic  programs  and  allows  them  to  develop  the  skills  needed  to  compete  when 
they  leave  the  institution. 
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Community  of  Practice:  What  is  it,  and  how  can  we  use  this  metaphor 
for  teacher  professional  development? 
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Abstract 

In  this  paper  / provide  the  reader  with  the  definitions  and  characteristics  that  are  often  associated  with  the 
community  of  practice  metaphor.  Then  / propose  that  community  of  practice  is  an  appropriate  tool  for 
descriptive  research  or  improving  the  interactions  that  take  place  in  a community,  but  not  necessarily  an 
appropriate  instructional  tool  for  designing  and  developing  learning  communities  in  schools.  / demonstrate 
this  by  reporting  on  findings  from  a case  study  of  a teacher  professional  development  program  for 
technology  curriculum  integration 

Introduction 

In  the  past  decade  community  of  practice  has  become  a popular  term  in  both  educational  and  business 
settings.  Many  of  its  implications  are  in  alignment  with  situated  learning  theories,  sociocultural  learning  theories, 
and  organizational  learning  theories.  Community  of  practice  as  a metaphor  provides  researchers  with  a theoretical 
lens  for  explaining  why  members  of  a community  do  what  they  do  in  everyday  settings,  and  how  community 
members  define  practices  and  engage  in  identity  formation.  The  notion  of  community  of  practice  provided  the 
educational  research  community  a method  for  explicating  cultural  knowledge  that  is  often  times  a tacit  set  of  rules 
and  rituals  that  function  as  a lens  through  which  members  of  a community  view  and  interpret  the  world  and  give 
order,  significance  and  meaning  to  their  experiences  (Maynard,  2001).  The  endorsement  that  community  of  practice 
has  won  in  the  field  of  educational  research  represents  the  ideological  shift  that  occurred  during  the  late  1980s  where 
researchers  began  to  express  their  interest  in  examining  learning  in  everyday  settings  rather  than  school  settings  (see 
Brown,  Collins,  & Duguid,  1989;  Lave,  1988;  Resnick,  1987;  Rogoff,  1990). 

Consequently,  there  has  been  a plethora  of  research  surrounding  community  of  practice  in  both  educational 
and  business  settings.  Many  of  these  efforts  have  been  based  on  research  using  ethnographic  methods.  The  outcomes 
of  these  ethnographic  studies  have  helped  identify  the  characteristics  and  mechanisms  of  community  of  practice 
especially  how  it  influences  organizational  learning  and  individual  learning.  Thus  within  educational  research 
community  of  practice  has  become  an  interesting  and  useful  metaphor  for  describing  and  understanding 
organizational  learning.  However,  for  the  community  of  practice  metaphor  to  mature  as  a research  and  development 
framework  in  educational  settings,  we  need  more  discussions  surrounding  how  it  can  be  used  as  a metaphor  for 
improving  preservice  and  inservice  teacher  education.  This  manuscript  will  specifically  focus  on  inservice  teacher 
professional  development. 

The  charge  of  this  manuscript  is  to:  (a)  reexamine  the  definitions  and  characteristics  that  are  often 
associated  with  the  community  of  practice  metaphor;  (b)  clarify  that  this  metaphor  is  a theoretical  tool  for 
educational  researchers  make  sense  of  interactions  that  take  place  in  educational  settings;  and  (c)  use  the  community 
of  practice  metaphor  as  a theoretical  lens  to  reflect  on  interactions  that  took  place  at  a rural  Midwestern  school 
district  that  was  involved  in  a teacher  professional  development  program.  I will  conclude  with  suggestions  on  how 
to  use  community  of  practice  as  an  instructional  deign  tool  for  supporting  and  enhancing  organizational  learning. 

The  data  presented  here  has  been  extracted  as  a portion  of  a larger  multiple  case  study  that  took  place  in  two  rural 
Midwestern  school  districts  during  August  1998  to  May  2000. 

Literature  Review 

Community  of  practice  is  a term  popularized  by  Lave  and  Wenger  (1991)  when  they  examined  the 
legitimate  peripheral  participation  of  apprentices  in  professional  communities.  They  claimed  that  apprentices  of  a 
community  of  practice  are  given  legitimate  roles  within  the  community,  and  their  actions  have  a direct  consequence 
for  the  entire  community.  As  an  apprentice  gradually  appropriates  the  skills  that  are  necessary  for  her  to  become  a 
more  skillful  member  of  the  community,  she  is  assigned  legitimate  tasks  that  have  greater  consequences.  Within  this 
theoretical  framework,  communities  of  practice  consist  of  groups  of  individuals  with  common  goals  who  are 
engaged  in  joint  activities  in  a common  setting.  The  way  that  work  related  activities  are  exercised  in  a community  of 


practice  affects  the  way  that  its  members  view  the  world,  and  it  also  defines  the  legitimacy  of  a task  practiced  in  the 
community.  Lave  and  Wenger  (1991)  define  the  term  as  follows: 

A community  of  practice  is  a set  of  relations  among  persons,  activity,  and  world,  over  time  and  in  relation 
with  other  tangential  and  overlapping  communities  of  practice,  A community  of  practice  is  an  intrinsic 
condition  for  the  existence  of  knowledge,  not  least  because  it  provides  the  interpretive  support  necessary 
for  making  sense  of  its  heritage,  Thus,  participation  in  the  cultural  practice  in  which  any  knowledge  exits  is 
an  epistemological  principle  of  learning,  (p,  98) 

Additionally,  the  history  and  experiences  shared  in  a community  of  practice  defines  what  constitutes 
competence  within  that  community  (Wenger,  2000),  In  other  words,  the  shared  experiences  within  a community 
define  the  legitimacy  of  practices  that  are  carried  out  on  an  everyday  basis. 

Interestingly,  the  focus  of  Lave  and  Wenger’s  work  was  not  to  identify  and  define  the  notion  of  a 
community  of  practice.  They  focused  their  work  on  illuminating  the  setting,  processes,  and  individuals  involved  in 
professional  communities,  such  as  the  Yucatec  midwives  and  naval  quartermasters.  Lave  and  Wenger  introduced  the 
notion  that  community  members  consist  of  newcomers  and  old-timers,  and  the  learning  activities  involved  for  a 
newcomer  to  transition  to  an  old-timer  is  extraordinarily  situated  in  the  everyday  practices  of  the  community 
contrary  to  learning  experiences  that  K-16  learners  face  in  schools.  Lave  and  Wenger  used  community  of  practice  as 
a descriptive  metaphor  in  their  anthropological  studies  involving  legitimate  peripheral  participation  to  portray  how  a 
community  makes  meaning  of  the  world  and  how  the  world  and  individuals’  identities  are  defined  by  the  history  and 
practices  shared  by  the  community  members.  Lave  and  Wenger  never  made  any  claims  from  the  implications  of 
their  study  about  how  to  constitute  commu  nity  of  practices  in  educational  settings  (Palincsar,  Magnusson,  Marano, 
Ford,  & Brown,  1998), 

Wenger  (1998)  further  examined  the  concept  of  community  of  practice.  In  his  1998  work  he  provided 
definitions  of  community,  practice,  and  identity  formation  within  a community.  He  reported  that  community  practice 
is  an  integral  part  of  our  daily  lives,  and  it  is  everywhere.  Communities  of  practice  do  not  have  a name  or  a 
membership  card,  but  members  of  a community  of  practice  identify  themselves  as  a “we”  defined  by  the  joint 
participation  for  attaining  a common  goal.  This  notion  of  “we”  draws  boundaries  between  multiple  communities  of 
practices,  and  these  boundaries  can  act  as  an  important  agent  for  learning  systems  because  they  connect 
communities  and  offer  opportunities  for  learning  (Wenger,  2000),  Community  members  that  cross  over  to  another 
community  of  practice  are  exposed  to  practices  and  artifacts  that  are  new  to  them.  These  new  experiences  can  allow 
individuals  to  engage  in  activities  that  they  never  did  prior  to  the  boundary  crossing.  Then  these  individuals  can 
bring  back  their  newly  negotiated  understandings  to  their  original  community  by  becoming  the  brokers  of  the  new 
practices  (Wenger,  1998), 

Wenger  (1998)  cautions  that  community  of  practice  is  not  a new  solution  to  existing  organizational 
problem,  and  “they  are  not  a design  fad,  a new  kind  of  organizational  unit  or  pedagogical  device  to  be  implemented” 
(p.  228),  Wenger  further  explains  that: 

Communities  of  practice  are  about  content — about  learning  as  a living  experience  of  negotiating 
meaning — not  about  form.  In  this  sense,  they  cannot  be  legislated  into  existence  or  defined  by  decree.  They  can  be 
recognized,  supported,  encouraged,  and  nurtured,  but  they  are  not  reified,  designable  units.  (p,229) 

Finally,  Wenger  (2000)  cautions  researchers  not  to  romanticize  the  notion  of  community  of  practice.  At  the 
same  time  that  a community  of  practice  supports  learning,  it  can  also  be  the  mechanism  that  enables  individuals  to 
learn  the  practice  of  not  learning.  Therefore,  a community  of  practice  is  not  necessarily  always  advantageous  for 
human  everyday  practices. 

The  field  of  educational  research  quickly  adopted  community  of  practice  as  a conceptual  tool  for  both 
research  and  instructional  design.  As  a research  tool  community  of  practice  has  been  used  to  identify  individual 
identity  formations  and  interindividual  interactions  that  take  place  within  a classroom  setting  or  a broader 
educational  setting  (see  Matusov,  1999,  2001;  Maynard,  2001).  As  an  instructional  design  tool  community  of 
practice  has  been  introduced  as  an  architectural  guideline  of  instructional  environments  as  used  in  Barab  and  Duffy 
(2000)  and  in  Palincsar  et  al,  (1998).  However,  in  these  instances  the  authors  stumbled  into  situations  where  they 
realized  that  community  of  practice  could  not  be  used  as  a pedagogical  device  for  designing  communities  in  K-16 
education  or  in  teacher  professional  development.  For  example,  Barab  and  Duffy  used  community  of  practice  as  the 
overarching  metaphorical  notion  for  providing  K-16  students  a highly  situated  and  authentic  learning  environment. 
Thus,  they  point  out  the  shortcomings  of  using  community  of  practice  as  an  architectural  tool  of  learning 
environments  because  the  experiences  learners  encounter  in  school  settings  are  alienated  from  the  real  problems  that 
community  of  practices  encounter.  To  alleviate  this  shortcoming  Barab  and  Duffy  suggest  instructional  designers  to 
use  the  notion  of  practice  fields  (Senge,  1994)  in  K-16  educational  settings. 
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Palincsar  et  al.  (1998)  used  community  of  practice  as  a design  tool  for  tracking  the  birth  and  growth  of  a 
community  within  a teacher  professional  development  program  that  focused  on  improving  classroom  practices  of  K- 
5 teachers  specifically  in  the  area  of  science  education.  However,  the  above  authors  point  out  themselves  that  it  is 
difficult  to  use  community  of  practice  as  an  instructional  design  tool  for  professional  development  because  within 
the  everyday  work  life  of  teachers  there  is  a lack  of  a commonly  shared  mission  or  a joint  enterprise  (Hargreaves, 
1993),  and  teachers  lead  individualistic  and  isolated  work  lives  (Little,  1990),  where  they  do  not  necessarily  feel  as 
they  are  a member  of  a community  of  practice.  Ironically  the  attempts  for  using  community  of  practice  as  an 
instructional  design  tool  for  architecting  educational  environments  have  reinforced  Wenger’s  (1998)  position  that 
community  of  practice  is  not  a designable  unit.  Instead,  community  of  practice  is  a useful  tool  for  identifying  and 
supporting  community  efforts  in  order  to  improve  and  assist  the  educational  efforts  of  a group  of  individuals. 

This  Study 
TICKIT 


This  study  took  place  in  the  context  of  a teacher  professional  development  program  about  technology 
curriculum  integration.  The  program  that  the  study  participants  were  involved  in  was  the  Teacher  Institute  for 
Curriculum  Knowledge  about  Integration  of  Technology  (TICKIT).  TICKIT  is  a university  school  partnership 
between  five  rural  Indiana  schools  and  Indiana  University.  Each  year,  in  this  program  there  are  at  least  25  teachers 
enrolled,  and  they  are  responsible  of  developing,  implementing,  and  evaluating  at  least  two  lessons  that  integrate 
technology  in  their  subject  area.  During  their  participation,  TICKIT  participants  are  provided  support  from  Indiana 
University  staff  and  their  local  schools  for  integrating  technology  into  their  everyday  teaching  activities  and  for 
becoming  technology  leaders  in  their  schools.  In  exchange,  upon  completion  of  all  projects  required  in  TICKIT, 
teachers  receive  six  graduate  credit  hours  that  can  be  used  toward  either  a master’s  degree  or  for  revalidating  their 
Indiana  teaching  license. 

Methods 

Participants 

The  initial  participants  of  this  study  were  selected  from  (a)  school  districts  that  enrolled  in  TICKIT  for  two 
consecutive  years  during  the  1998-1999  and  1999-2000  school  year,  and  (b)  school  districts  that  provided  support 
for  participant  teachers.  The  primary  participants  were  individuals  who  were:  (a)  TICKIT  1998-1999  participants 
who  were  classroom  teachers,  and  (b)  TICKIT  1998-1999  participants  who  participated  in  the  entire  yearlong 
program.  The  secondary  participants  were  non-TICKlT  teachers,  administrators,  technology  coordinators,  or 
technology  support  staff.  Anonymity  of  the  study  participants  was  maintained  by  the  use  of  pseudonyms  for 
individuals  and  school  district  names.  For  the  purpose  of  this  manuscript,  I will  present  a portion  of  the  entire  data 
from  the  Hillsdale-Berkley  School  District.  At  the  Hillsdale-Berkley  School  District  there  were  four  primary 
participants,  including  two  eighth  grade  teachers , one  sixth  grade  teacher,  and  one  third  grade  teacher,  and  five 
secondary  participants,  including  two  teachers,  one  media  specialist,  one  technology  coordinator,  and  one 
technology  support  person. 

Researcher  Role 

The  above  participants  and  I were  well  acquainted  with  each  other  during  this  study  because  I was  the 
TICKIT  graduate  assistant  that  maintained  the  website,  conducted  technology  related  workshops  for  participants, 
and  provided  any  other  assistance  requested  by  the  program  coordinators  or  TICKIT  participants.  Therefore,  there 
were  moments  during  this  study  when  I took  the  role  of  more  than  an  outside  observer.  For  example,  when  there 
were  inservice  technology  sessions  at  the  Hillsdale-Berkley  school  district,  the  teachers  were  very  eager  for  me  to 
participate,  partly  for  my  data  collection  purposes  and  partly  so  that  I could  help  them  with  teacher  training. 

Data  Collection  Methodology 

In  this  research,  naturalistic  inquiry  methodology  (Lincoln  & Cuba,  1985)  was  used,  where  the  data 
collection  took  place  in  a natural  setting  and  there  were  no  variables  manipulated  in  anticipation  of  confirming  a 
hypothesis.  More  specifically,  I conducted  a case  study  (Merriam,  1998;  Stake,  1995).  The  data  collection  methods 
included  document  analysis,  interviews,  and  classroom  observations  with  1998-1999  TICKIT  participants,  non- 
participating teachers,  and  school  technology  support  staff 
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Trustworthiness 


I attempted  to  maintain  the  trustworthiness  of  this  qualitative  study  by:  (a)  prolonged  engagement  with  the 
research  site,  (b)  persistent  observation,  (c)  triangulation,  and  (d)  member  checking.  Although  this  research  began 
after  the  TICKIT  1998-1999  program,  I had  close  ties  with  the  participants  during  TICKIT  1999-2000.  Therefore,  I 
had  a total  of  two  years  of  engagement  with  the  research  site. 

Data  Presentation 
School  District  Background 

Hillsdale-Berkley  School  District  is  a rural  Midwestern  school  system  with  minimal  industry  in  the 
community.  The  district  serves  six  schools  (four  elementary  schools,  one  middle  school,  and  one  high  school).  Due 
to  the  rural  environment  in  which  the  school  is  situated,  it  is  isolated  Irom  many  educational  resources,  such  as 
public  libraries  and  museums  that  could  be  available  in  more  urban  settings.  The  district’s  mission  is  to  “encourage 
and  direct  the  physical,  mental  and  social  growth  of  each  student...  so  he  can,  to  the  best  of  his  abilities,  become  a 
well  adjusted,  contributing  member  of  our  society”  (Hillsdale-Berkley  School  District  Technology  Implementation 
Plan  1997-2001). 

For  the  past  five  years  the  Technology  Planning  Committee,  which  is  a district-wide  committee,  has  been 
planning  for  the  technology  implementation  plan.  The  mission  of  this  committee  is  to  “create,  implement,  evaluate 
and  revise  a long-range  technology  plan  for  learning  that  will  move  the  students  of  the  schools  well  into  the  21st 
century”  (Hillsdale-Berkley  School  District  Technology  Implementation  Plan  1997-2001).  The  district 
acknowledges  that  electronic  technology  has  become  an  integral  part  of  the  function  of  the  community.  Therefore, 
their  goals  state  “learners  and  teachers  will  develop  the  skills  necessary  to  effectively  utilize  technology  ethically 
and  creatively  in  serving  both  the  individual  and  society”  (Hillsdale-Berkley  School  District  Technology 
Implementation  Plan  1997-2001). 

However,  many  of  the  technology  related  initiatives,  such  as  applying  to  various  state  grants  and 
implementing  technology  in  the  curriculum,  have  been  efforts  pursued  by  pockets  of  enthusiastic  teachers  rather 
than  a movement  led  by  district  administrators  and  staff  The  common  practice  for  gaining  grant  money,  at  the 
Hillsdale-Berkley  School  District,  works  such  that  the  school  library  media  specialist  finds  grant  applications  that  fit 
a single  or  several  teachers’  needs;  then  the  teacher  will  apply  for  the  grant  individually  or  with  the  media 
specialist’s  help.  Therefore,  grant  application  preparations  are  considered  as  extra  workload  for  teachers,  and  they 
usually  spend  time  after  school  hours  to  write  grants.  When  the  teacher  or  groups  of  teachers  gain  the  grant  money, 
it  is  distributed  for  their  classroom  use  and  not  necessarily  for  school-wide  initiatives 

Hillsdale-Berkley  School  District  and  TICKIT 

At  the  Hillsdale-Berkley  School  District,  prior  to  enrolling  in  TICKIT,  there  were  several  self- motivated 
teachers  experimenting  with  the  use  of  technology  in  their  classrooms.  These  teachers  were  interested  in  using 
technology,  because  when  the  integration  was  successful,  it  oftentimes  provided  a rich  learning  environment  for 
students,  and  students  were  motivated  to  learn  in  this  technology  rich  environment.  The  school  district  did  not  have 
funds  for  providing  teachers  with  technology  in  their  classrooms;  therefore,  the  technology  enthusiastic  teachers 
applied  for  small  grants  that  allowed  them  to  provide  the  hardware  and  software  in  their  classrooms. 

Hillsdale-Berkley  teachers  worked  in  interdisciplinary  teams,  but  technology  related  activities  were 
contained  within  the  boundaries  of  the  classroom.  Consequently,  teachers  did  not  develop  the  practice  of  sharing 
physical  resources  such  as  equipment  and  software  or  sharing  teaching  related  ideas  and  stories.  Furthermore, 
teachers  that  were  not  comfortable  in  using  technology  in  their  classroom  did  not  consider  the  enthusiastic  teachers’ 
efforts  as  curriculum  development.  They  perceived  activities  related  to  technology  curriculum  integration  to  be  a 
hobby-like  activity. 

For  some  teachers  at  the  Hillsdale-Berkley  School  District,  participating  in  TICKIT  gave  justifications  for 
non-TICKIT  participating  teachers  to  acknowledge  the  TICKIT  participants’  efforts.  By  participating  in  TICKIT, 
the  teachers’  efforts  in  integrating  technology  into  the  curriculum  and  prioritizing  them  over  other  teaching 
responsibilities  were  well  accepted  by  non-TICKIT  teachers.  Becoming  participants  of  a university  school 
partnership  program  gave  renewed  sense  of  credibility  to  the  activities  surrounding  technology  that  the  Hillsdale- 
Berkley  TICKIT  participants  were  engaging  in.  For  example,  during  her  interview  Naomi,  an  eighth  grade  Language 
Arts  teacher,  commented  that  when  she  shared  the  projects  she  completed  in  TICKIT  with  her  non-TICKIT 
participating  colleagues  on  her  team,  the  project  was  acknowledged  as  an  important  curriculum  unit  and  not 
“Naomi’s  computer  project:” 
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it's  accepted  better.  It's  not  "oh,.. it’s  Naomi’s  computer  project"  now  it’s  like  "oh,  their  doing  the  WebQuest 
project."  Which  is  a big  difference,,,  if  it’s  my  little  project  on  the  computer  it's  not  valid.  It's  just  that  thing 
she  does,  but  if  it's  a WebQuest,  suddenly  it's  like  part  of  the  curriculum  and  they  see  educational  value  that 
they  did  not  before.  I think  that  changed  the  attitude  a little  bit,  (Naomi,  Primary  Participant  Interview, 
February  29,  2000) 

This  acknowledgement  by  non-TICKIT  teachers,  that  the  TICKIT  participating  teacher  projects  were  part 
of  the  curriculum  and  not  just  a computer  project,  encouraged  a change  in  attitude  of  the  non-TICKIT  teachers,  and 
some  became  excited  to  participate  in  their  schools’  curriculum  technology  integration  efforts. 

At  the  end  of  the  program,  the  Hillsdale-Berkley  teachers  felt  that  they  gained  new  technology  skills  that 
made  them  feel  more  comfortable  and  confident  in  using  various  technologies  in  their  classroom.  With  their  newly 
gained  confidence  and  technological  skills,  teachers  at  the  Hillsdale-Berkley  School  District  became  eager  to 
incorporate  technology  into  their  classroom.  Deborah,  an  eighth  grad  Math  teacher,  commented  on  this  in  the 
following  excerpt:  “being  comfortable  of  using  the  Internet ...  being  able  to  create  Web  pages... did  a lot  to  me  in 
making  me  feel  comfortable... and  try  to  do  things  for  my  students,.,”  (Deborah,  Primary  Participant  Interview, 
March  8,  2000).  Furthermore,  Alice,  a sixth  grade  Language  Arts  and  Social  Studies  teacher,  felt  that  her  newly 
gained  confidence  energized  her  and  made  her  eager  to  continue  to  work  on  integrating  technology  in  her 
classrooms.  She  mentioned  during  her  interview  that:  “I  am  always  thinking  about  ways  [to  integrate  technology  in 
the  curriculum],  what  can  I do,  what  can  I do”  (Alice,  Primary  Participant  Interview  February  2000).  However, 
Hillsdale-Berkley  teachers  realized  that  there  was  not  enough  equipment  for  them  to  use  in  their  day-to-day 
teaching,  especially  in  the  Middle  School. 

With  the  new  sense  of  confidence,  new  university  connections,  and  camaraderie  shared  among  TICKIT 
teachers  at  Hillsdale-Berkley,  teachers  became  eager  and  excited  to  apply  for  new  technology-related  grants.  As  the 
non-TlCKlT  teachers  at  the  district  witnessed  several  of  the  TICKIT  projects  and  found  that  the  school  was 
purchasing  more  equipment  they  became  interested  in  using  technology  in  their  own  classrooms.  Some  non-TICKIT 
teachers  even  chose  to  enlist  themselves  in  TICKIT  for  the  1999-2000  year. 

The  TICKIT  1998-1999  teachers  continued  to  influence  their  non-TICKIT  participating  colleagues  by 
taking  a leadership  role  during  inservice  teacher  training  sessions  and  by  making  themselves  available  for  helping 
other  teachers.  Emma,  the  media  specialist,  commented  how  the  five  TICKIT  participants  from  her  school  have  been 
helpful  to  other  teachers. 

They've  been  helpful  in  helping  people  design  other  things  too,  and  getting  their  Web  page  going  or  you 
know,  designing  a WebQuest  too,  so  they've  been  good  resources  people.  (Emma,  Secondary  Participant 
Interview,  April  16,  2000) 

For  example,  during  February  2000,  Emma  coordinated  a two  session  series  of  after-school  inservice 
programs  for  teachers  at  Hillsdale-Berkley  Middle  School  on  Web  publishing.  Henry,  a third  grade  teacher  who 
participated  in  TICKIT,  was  the  instructor  for  the  inservice  session,  and  many  of  the  participants  of  the  session  were 
TICKIT  1998-1999  teachers,  I observed  the  second  day  of  one  of  these  sessions,  where  teachers  brought  in  the 
curriculum  they  wanted  to  publish  on  the  Web  and  worked  on  it  while  Henry  and  Emma  assisted  in  answering 
questions. 

The  first  day  of  the  in-service,  according  to  Emma,  was  a typical  training  session  where  Henry 
demonstrated  how  to  use  Netscape  Communicator.  The  second  session  was  designed  for  teachers  to  bring  their 
recent  Web  publishing  projects  and  use  the  inservice  time  as  work  time.  The  TICKIT  participants  other  than  Henry 
who  were  present  at  the  inservice  were  helpful  in  answering  other  teachers’  questions  while  they  were  working  on 
their  own  projects  as  well. 

Discussion 

When  examining  the  historical  development  at  the  Hillsdale-Berkley  School  District  prior  to  TICKIT, 
teachers  enthusiastic  about  using  technology  in  the  classroom  had  to  isolate  themselves  from  their  day-to-day 
teaching  colleagues  if  they  wanted  to  develop  and  implement  technology  infused  curriculum.  Within  the  everyday 
practice  of  teaching  at  Hillsdale -Berkeley,  curriculum  technology  integration  was  not  acknowledged  as  a legitimate 
teaching  related  practice. 

Teaching  efforts  surrounding  technology  use  in  the  classrooms  were  perceived  by  other  teachers  as 
indulging  in  hobby-like  activities  that  satisfied  individual  interests  rather  than  meeting  curricubm  needs  with 
pedagogical  values.  Additionally,  indulging  in  such  “illegitimate”  practices  were  potentially  threatening  to  a 
teacher’s  career  because  it  could  have  discredited  their  teaching  as  a whole  by  other  local  teachers.  Inevitably,  the 
self-motivated  teachers  that  were  enthusiastic  about  integrating  curriculum  and  technology  had  to  isolate  themselves 
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from  the  local  boundaries  of  their  teaching  colleagues  and  choose  not  to  share  their  ideas  and  stories  regarding 
technology  implementation  with  local  colleagues. 

While  participating  in  TICKIT  Hillsdale-Berkley  teachers  found  other  teachers  interested  in  sharing  and 
legitimizing  the  common  goal  of  designing,  developing,  implementing,  and  evaluating  new  methods  for  using 
technology  in  the  classroom  that  addressed  curriculum  and  student  needs.  Based  on  the  partnership  build  by 
becoming  a TICKIT  participant,  these  colleagues  included  teachers  from  their  school  district  and  outside  of  their 
school  district,  and  university  faculty.  Here,  the  teachers  from  Hills  dale-Berkley  found  a new  group  of  individuals  to 
associate  with  while  they  excused  themselves  from  their  local  colleagues.  In  this  way,  TICKIT  introduced  to  them 
new  peers  for  Hillsdale-Berkley  TICKIT  teachers  to  work  together  to  develop  technology -integrated  lessons. 

After  TICKIT,  once  the  Hillsdale-Berkley  teachers  attained  their  goal  of  a successful  curriculum 
technology  integration  project,  they  became  boundary  brokers  (Wenger,  1998)  between  the  TICKIT  community  and 
their  local  community  of  teachers.  After  completing  the  program,  TICKIT  teachers  from  Hillsdale-Berkley  brought 
to  their  local  community  new  artifacts  (i.e.  successful  curriculum  technology  projects)  and  stories  surrounding  those 
artifacts.  These  new  artifacts  and  stories  of  TICKIT  teachers  acted  as  catalysts  that  encouraged  other  teachers  to 
become  involved  in  TICKIT  for  the  following  two  school  years. 

Additionally  after  TICKIT,  Hillsdale-Berkley  teachers  found  a group  of  local  teachers  and  technology  staff 
who  shared  their  vision  for  school-wide  curriculum  technology  integration.  These  teachers  wrote  grants  together  to 
acquire  funding  that  would  support  purchasing  technology  equipment  and  professional  development  activities  for 
local  teachers.  Ordinarily,  the  community  has  great  influence  over  shaping  the  everyday  practices  that  teachers 
choose  to  engage  in  because  the  community  defines  the  rules  and  division  of  labor  associated  with  practices. 
However,  as  seen  in  the  above  example,  community  endorsement  is  not  the  sole  predictor  of  success. 

Innovations  that  are  introduced  to  everyday  practices  of  teachers,  such  as  a professional  development 
program,  can  change  the  perception  of  local  practices,  and  legitimatize  new  teaching  related  practices  that  were 
formally  illegitimate.  For  example,  after  TICKIT  technology  enthusiastic  teachers  in  Hillsdale-Berkley  no  longer 
needed  to  isolate  themselves  from  their  local  colleagues  because  the  activities  surrounding  the  use  of  technology  in 
classrooms  became  legitimate.  Therefore,  TICKIT  teachers  did  not  need  to  belong  to  an  outside  group  to  attain  their 
technology  curriculum  integration  goals.  They  became  a self-sustained  group  of  teachers  that  shared  a common  goal 
and  were  acknowledged  by  many  of  their  local  colleagues  as  technology  leaders  in  the  district. 

Through  the  above  examination  there  are  several  transitions  that  can  be  identified.  These  transitions  include 
the  following:  (a)  the  non-TICKIT  teachers  perception  of  TICKIT  teachers*  everyday  efforts  for  integrating 
technology  into  the  curriculum  changed  from  a hobby-like  illegitimate  teaching  practice  to  a legitimate  curriculum 
development;  (b)  TICKIT  teachers*  perception  of  their  own  skills  and  confidence  regarding  technology  use  in  the 
classroom  changed  from  being  mediocre  and  timid  users  of  technology  to  skillful  and  bold  users,  and  (c)  the 
TICKIT  teachers*  identity  changed  from  a closet  technology  user  in  the  classroom  to  a technology  leader  in  the 
district.  The  key  artifact  that  became  the  catalyst  for  the  above  transitions  was  the  TICKIT  participating  teachers* 
curriculum  technology  integration  projects.  These  projects  served  the  purpose  of  reified  objects  that  helped 
institutionalizing  and  legitimizing  new  practices  at  the  Hillsdale-Berkley  school  district. 

Conclusion 

In  this  manuscript  I presented  community  of  practice  as  a useful  tool  for  examining  interactions  that  take 
place  in  professional  development  settings  in  order  to  understand  everyday  practices  within  a teacher  community 
and  to  identify  what  practices  are  perceived  to  be  legitimate  and  what  practices  are  not.  Teacher  professional 
development  involves  introducing  and  legitimizing  new  practices  within  the  teaching  community.  In  some  cases, 
such  as  in  this  study,  professional  development  programs  can  assist  in  nurturing  and  enhancing  practices  that  are 
already  practiced  within  a small  group  of  teachers  in  a school  district  by  introducing  the  non -practicing  teachers  to 
new  artifacts  that  are  reifications  of  legitimate  pedagogical  practices. 

Using  community  of  practice  as  a tool  for  examining  local  teaching  practices  within  a school  district  can 
help  universities,  school  districts,  and  teachers  involved  in  a professional  development  program  identify  what  are  the 
accelerators  and  barriers  of  success  for  introducing  new  pedagogical  practices.  Once  this  is  identified  it  is  the  task  of 
the  professional  development  program  and  its  participants  to  leverage  the  local  practices  to  identify  methods  for 
legitimizing  new  teaching  related  activities.  Therefore,  community  of  practice  can  be  used  as  a theoretical  lens  for 
identifying,  supporting,  and  strengthening  practices  that  already  exist  in  school  districts.  This  will  then  assist  school 
districts  to  develop  robust  and  pedagogically  sound  educational  practices. 

In  conclusion  I suggest  that  in  professional  development  settings  universities,  school  districts,  and  teachers 
use  community  of  practice  as  a metaphor  for  analyzing  current  practices  and  developing  strategies  for  infusing  new 
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practices  to  a school  district.  As  the  works  of  Barab  and  Duffy  (2000)  and  Palincsar  et  al  (1998)  suggest,  there  are 
limitations  for  using  community  of  practice  as  guidelines  for  designing  and  developing  new  learning  environments 
in  educational  settings.  Therefore,  in  future  research  there  is  a need  for  examining  the  advantages  and  shortcomings 
of  using  community  of  practice  as  a tool  for  designing  and  developing  educational  environments. 
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Family  Characteristics  of  Authentic  Materials  and  Activities  in  Constructivist  Learning 

Environments 


Abstract 


Betul  Ozkan 
Iowa  State  University 


In  the  constructivist  literature,  the  use  of  authentic  materials  and  authentic  activities  as  tools  of  meaningful 
learning  is  repeatedly  emphasized.  Using  Wittgenstein 's  concept  of  family  resemblance  as  a framework,  I 
conducted  an  extensive  study  of  the  authentic  activities  and  materials  used  in  three  elementary  classrooms 
in  Des  Moines,  Iowa.  The  presentation  will  be  based  on  analysis  of  interviews,  observations,  field  notes, 
and  student  portfolios  using  N*VIVO,  a qualitative  software  package.  In  recent  years,  educators  all 
around  the  world  have  been  searching  for  new  approaches  to  teaching  and  learning.  In  both  developed  and 
developing  nations,  many  policy  makers  as  well  as  educators  are  emphasizing  more  than  ever  the 
importance  of  education.  The  constructivist  perspective  is  one  valuable  response  to  these  concerns  about 
finding  alternatives  to  traditional  teaching  methods.  It  is  also  an  area  of  cutting-edge  applied  research. 

Introduction 


Authenticity,  in  fact,  is  one  of  the  main  characteristics  of  constructivist  learning  environments  (CLE). 
There  are  many  theoretical  discussions  of  authentic  learning,  and  many  lists  of  the  theoretical  and  conceptual 
characteristics  of  authentic  learning  materials  and  activities.  However,  there  is  very  little  research  on  the  actual 
characteristics  of ‘authentic’’  materials  and  activities  as  they  are  used  in  real  classrooms.  Are  the  actual 
characteristics  similar  to  or  different  from  the  theoretical  characteristics  described  in  the  literature?  This  paper 
addresses  this  question  and  compares  what  I found  in  the  classrooms  studied  to  the  theoretical  characteristics  of 
authentic  learning  described  in  literature.  The  concept  of  family  resemblance,  which  was  developed  by  the 
philosopher  Ludwig  Wittgenstein  (1953,  1958),  has  been  used  as  a framework  for  understanding  the  concept  of 
authentic  materials  and  activities. 


The  Notion  of  “Family  Resemblance’’ 

This  study  researches  the  family  characteristics  of  CLE  using  the  notion  of ‘family  resemblance’.  The 
“family  resemblance”  concept  was  developed  by  Austrian  philosopher  Ludwig  Wittgenstein  (1889-1951).  In  his 
book  Principal  Investigations  published  posthumously  in  1953,  he  discusses  language  philosophy  in  detail.  In  his 
early  studies  “he  tended  to  take  an  essentialist  view  of  common  nouns:  the  view  that  there  must  be  some  common 
element  in  all  cases  in  which  we  apply  such  a word,  something  that  regulates  our  use  of  it  for  all  those  cases” 
‘(Brenner  1999,  p.23).  However,  the  later  Wittgenstein  challenged  this  requirement  by  describing  counterexamples, 
starting  with  the  noun  ‘game’  (1953): 

“Consider  . . .board  games,  card  games,  ball  games,  Olympic  games,  and  so  on . . . If  you  look  at  them  you 
will  not  see  something  that  is  common  to  all,  but  similarities,  relationships,  and  a whole  series  of  them  at 
that  (PI,  #66).” 

Here  Wittgenstein  (1953)  makes  a straightforward  statement  of  fact:  “if  we  examine  those  things  we  call 
games,  we  will  not  find  any  single  property  in  virtue  of  which  they  are  called  games;  instead  we  find  that  they  are 
grouped  together  by  a whole  series  of  overlapping  similarities  akin  to  family  resemblances”. 

The  family  resemblance  concept  of  Wittgenstein  fits  perfectly  with  constructivist  theory  and  helps  orient 
studies  like  this  one.  In  this  study  I was  looking  for  family  resemblances,  not  universal  traits.  So  the  features 
mentioned  below  are  not  the  common  features  of  all  three  classrooms  I observed  all  the  time.  Rather,  they  are  often 
found,  and  taken  as  a group  they  constitute  a set  of  family  resemblances. 


Setting  of  the  Study 

For  this  study  data  were  collected  in  three  different  elementary  school  classes.  The  classes  were  drawn 
from  two  different  school  districts  in  the  Des  Moines,  Iowa  area.  These  classes  were  selected  through  an  extensive 


process  that  began  with  seeking  recommendations  from  professors  and  school  administrators.  These  sources 
recommended  a large  pool  of  classrooms  that  were  considered  “constructivist.”  I interviewed  15  teachers  and 
observed  in  their  classrooms.  Many  classrooms  were  not  actually  using  constructivist  teaching  strategies  and  were 
rejected.  Eventually,  three  were  found  that  regularly  used  constructivist  teaching  strategies  including  authentic 
instruction.  The  teachers  in  these  three  classrooms  were  all  well  known  for  their  regular  use  of  authentic  materials 
and  activities,  and  preliminary  observations  confirmed  this. 

There  were  two  teachers  from  the  same  school  district  who  participated.  Their  school  uses  the  Multi-Age 
Classrooms  (MAC)  concept.  The  first  teacher  (#T1)  taught  a class  with  grades  3,  4 and  5 in  the  room  whereas  the 
second  teacher  (#T2)  taught  first  and  second  graders  in  her  room.  They  were  both  very  experienced  teachers,  and 
had  masters*  degrees.  The  third  teacher  (#T3)  was  from  another  school  district  and  she  taught  only  third  graders. 

During  a seven  week  period  in  October,  November,  and  December  2000  I visited  these  classes  each  school 
day.  1 videotaped  all  the  class  time  with  single  digital  camera.  When  conducting  the  interviews,  I also  used  the  video 
camera  with  the  permission  of  teachers  and  parents.  After  my  visits  to  classes  I took  field  notes  on  a daily  basis  in 
order  to  clarify  the  videotapes*  content. 

Once  the  data  were  collected  I used  NVivo,  a popular  qualitative  data  analysis  program  to  code  and  analyze 

the  data. 

Results 


Family  Characteristics:  There  are  several  characteristics  of  constructivist  learning  environments  in  which 
authentic  activities  and  materials  are  used  regularly.  First,  teachers  use  real  life  materials  in  order  to  stimulate  deeper 
understandings.  Second,  the  meaningful  learning  experiences  are  the  basis  of  most  but  not  all  activities  in  the  three 
classrooms  studied.  Third,  there  are  many  learning  centers  in  which  students  practiced  hands-on  activities.  The  focus 
of  instruction  is  on  learning  in  contexts  instead  of  learning  in  “out  of  context.”  All  the  activities  were 
developmentally  appropriate  to  the  cognitive  level  of  students  and  the  teachers  helped  students  adjust  activities  to 
their  personal  levels  of  development.  Thus,  these  classrooms  tended  to  be  what  some  have  termed  “child-centered” 
as  well.  Technology  use  and  integration  in  these  classrooms  was  also  a very  distinguished  feature.  There  were  also 
other  characteristics,  such  as  differences  in  the  physical  environment  or  and  differences  in  the  teachers’  roles  which 
made  these  classrooms  innovative. 

Constructivist  Theory  Versus  Constructivist  Practice:  My  observations  in  classrooms,  plus  interviews  with 
teachers  and  students,  indicates  there  is  a strong  relationship  between  constructivist  theory  and  practice.  Teachers 
rarely  use  direct  instruction  methods  which  are  often  criticized  by  constructivist  theorists.  Instead,  they  use  a variety 
of  student  centered  instructional  approaches.  However,  while  theorists  tend  to  present  the  image  of  constructivist 
teaching  and  learning  as  exclusive,  my  study  clearly  shows  that  the  teachers  do  occasionally  use  traditional  methods 
such  as  drills  and  objective  tests.  These  were  used  infrequently,  but  they  were  used.  The  teachers,  who  were  aware 
of  constructivist  theory  and  had  extensive  experience  in  “teaching  constructively,”  seemed  comfortable  using 
traditional  methods  occasionally.  However,  these  methods  did  not  characterize  or  define  the  classroom  experiences 
of  students. 

Constructivist  Methods  Versus  Traditional  Methods  to  Learning:  The  classrooms  studied  are  considered 
good  examples  of  constructivism.  However,  I regularly  observed  traditional  methods  in  these  learning  environments. 
There  may  be  many  reasons  for  this.  Perhaps  it  is  very  difficult  use  constructivist  methods  all  the  time.  Perhaps 
teachers  are  concerned  that  the  traditional  expectations  of  what  schools  should  accomplish,  such  as  high  scores  on 
standardized  tests,  calls  for  traditional  methods.  Perhaps  it  is  difficult  to  collect  the  resources  needed  for 
constructivist  teaching.  Or,  perhaps  constructivist  teachers  still  see  a place  for  traditional  methods.  They  have  not 
replaced  traditional  methods  with  constructivist  methods;  they  have,  instead,  made  constructivist  methods  their 
primary  approach  and  they  continue  to  use  traditional  methods  because  they  see  a place  for  them  in  the  classroom. 

Another  reason  for  limited  use  of  constructivist  teaching  is  the  school  environment.  The  third  teacher  I 
observed,  for  example,  wanted  to  do  more  than  she  was  doing.  However,  she  felt  other  teachers  and  her  principal 
had  concerns  about  constructivist  approaches.  This  teacher  used  the  most  traditional  methods  and  the  fewest 
constructivist  approaches.  The  other  two  teachers,  who  were  in  schools  where  innovative  teaching  was  the  norm, 
and  where  they  were  encouraged  and  supported  by  other  teachers  and  administrators,  were  much  more  likely  to 
experiment. 

Authentic  Materials  Versus  Instructional  Materials:  Even  though  the  classrooms  studied  might  be 
considered  as  constructivist  learning  environments  which  involve  the  use  of  authentic  activities  and  materials,  I also 
regularly  observed  the  use  of  traditional  instructional  materials.  For  example,  in  language  arts  classes  students 
regularly  read  storybooks,  some  of  them  assigned  by  the  teacher  and  some  selected  by  students.  The  activities 


D 


646 


students  engaged  in  (e.g.,  reading  to  other  students,  finding  the  meaning  of  words  they  do  not  understand,  discussing 
the  story  with  others)  fall  within  the  broad  constructivist  framework  but  the  storybooks  themselves  are  not 
necessarily  “authentic:  materials.  However,  as  stated  by  Cronin  (1993)  assuring  that  all  learning  materials  are 
“authentic”  is  not  the  purpose  of  authentic  learning.  Instead  it  is  important  to  use  authentic  materials  and  activities 
as  much  as  makes  educational  sense.  Thus,  the  use  of  authentic  materials  is  as  regular  but  not  continuous 
characteristic  of  the  classrooms  studied.  Authentic  materials  are  a family  characteristic  that  will  be  seen  some  of  the 
time,  but  not  all  the  time. 
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Abstract 

In  order  to  understand  the  cognitive  and  affective  roots  underpinning  the  differences  between  teachers ' and 
parents  ’ teaching  approaches  for  hearing-impaired  children,  this  study  proposes  a four^level  analysis 
structure  (global-level  schema,  middle -level  schema,  local-level  schema,  and  prepositional  reasoning)  to 
construct  and  then  compare  teachers'  and  parents'  mental  models  for  teaching.  The  outcomes  of  this 
comparison  reveal  many  fundamentally  different  perceptions  and  attitudes  about  teaching  between 
teachers  and  parents,  which  often  result  in  ineffective  communication  between  the  two.  Therefore,  it  is 
suggested  that  to  improve  the  effectiveness  of  instruction  between  teachers  and  parents  in  special 
education,  one  needs  to  understand  how  teaching  exceptional  children  is  mentally  represented  before  any 
intervention  is  designed. 

Introduction 


In  special  education,  while  teachers  have  the  expertise  of  teaching  children  with  special  needs, 
parents,  usually  unprepared  for  such  a daunting  task,  are  the  ones  who  execute  the  teaching  tasks  and  help  children 
to  learn  everyday.  Therefore,  it  is  a common  practice  for  teachers  to  train  parents  to  teach  their  children.  However, 
the  training’s  ineffectiveness  often  puzzles  the  teachers.  It  is  a commonly  shared  feeling  among  teachers  that 
teaching  parents  is  more  difficult  than  teaching  children  because  it  requires  a lot  more  persuasion,  reminding,  and 
rectification  to  help  parents  to  accept  and  to  internalize  the  teaching  approaches.  Parents,  on  the  other  hand,  often 
find  it  difficult  to  completely  accept  teachers’  approaches.  An  important  reason  is  that  the  disparity  between  being  a 
parent  and  being  a teacher  to  a child  is  often  concomitant  with  different  perceptions  of  teaching,  which  often  results 
in  different  teaching  styles.  For  example,  parents  have  been  observed  to  incline  to  provide  more  intense  instruction 
(Rogoff,  Ellis,  & Gardner,  1984),  to  use  more  controlling  and  straightforward  styles  to  manage  children’s  activities, 
and  to  be  more  performance  oriented  (Wertsch,  Miick,  & Arns,  1984).  Moreover,  this  disparity  also  exerts  a 
significant  impact  on  the  extent  to  which  parents  identify  themselves  with  teachers’  teaching  approaches. 

Apparently,  when  factors  like  parental  expectations,  responsibilities,  relationship  with  the  child,  etc.  are 
amalgamated  with  teaching,  parents  often  develop  a different  view  from  teachers.  Especially  when  the  child  requires 
special  care,  the  effects  of  all  these  factors  seem  to  be  intensified;  therefore,  more  divergent  teaching  styles  are 
expectable.  However,  surprisingly  few  studies  have  been  conducted  to  understand  how  differently  teachers  and 
parents  think  about  teaching  children,  in  particular,  with  special  needs.  As  a result,  questions  such  as  “Why  parents 
ignore  teachers’  advice?”  and  “Why  teachers  have  trouble  communicating  a pedagogical  notion  to  parents?”  are  left 
answered.  In  order  to  answer  these  questions,  this  study  intends  to  delve  into  teachers’  and  parents’  mental  models 
to  present  and  then  to  compare  how  the  notions  of  teaching  are  internally  represented  in  teachers’  and  parents’ 
minds. 

Mental  Models 

Although  it  has  been  widely  adopted  and  studied  by  many  researchers,  there  is  no  consensus  of  the 
definition  of  the  term  “mental  models”  (Hong  & O’Neil,  1992).  In  general,  the  following  features  of  mental  models 
have  been  described  in  the  literature: 

First,  mental  model  are  internal  representations  that  we  construct  in  mind  for  the  message  or  event  that  we 
encounter  (Norman,  1983;  Johnson-Laird,  1983).  Such  representations  should  be  “essentially  problem-oriented,  not 
encyclopedic  in  nature,  and  are  bound  to  be  incomplete  or  underspecified”  (Sanford  & Moxey,  1999,  p.74).  That  is, 
they  present  an  analogous  structure  to  the  represented  entities  (Johnson-Laird,  1983;  1989)  instead  of  an  identical 
mirror  image  in  our  mind. 

Second,  mental  models  are  explanatory  constructs  (Johnson-Laird,  1983).  Once  the  internal  representation 
has  been  created  in  our  mind,  it  begins  to  influence  the  way  we  see,  interact  with,  and  reason  about  the  world 
(Greeno,  1989;  Dutke,  1996),  and  to  make  us  experience  events  by  proxy  (Johnson-Laird,  1983,  p.  397). 
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Third,  mental  models  have  computational  capabilities  (Greeno,  1989;  Wilson  and  Rutherford,  1989). 
Especially  for  discourse  comprehension,  such  computational  processes  are  often  represented  by  propositional 
reasoning  (Anderson,  1993;  Johnson-Laird,  1983). 

Finally,  mental  models  are  schema -based  (Driscoll,  1994;  Dutke,  1996;  Rouse  & Morris,  1986;  Wilson  & 
Rutherford,  1989;  Johnson-Laird,  1983).  Sanford  and  Moxey  (1999)  further  argue  that  the  notion  of  mental  model 
would  be  void  if  it  is  not  rooted  in  background  knowledge. 

What  these  features  imply  is  that  by  mapping  the  perceived  message  or  entities  to  our  background 
knowledge,  our  mind  constructs  a prior-knowledge  interpreted  representation  of  the  given  message,  which  not  only 
presents  itself  as  the  surrogate  reality  for  the  message  in  our  mind,  it  also  formulates  the  premises  under  which  we 
reason  and  make  inferences.  Given  the  close  link  between  mental  models  and  schema,  to  complete  the  theoretical 
discussion  of  mental  models,  it  is  important  to  understand  the  construct  of  schema  first. 

Schema 


Originally  defined  by  Barlett  (1932)  as  “an  active  organization  of  past  reactions,  or  of  past  experiences, 
which  must  always  be  supposed  to  be  operating  in  any  well-adapted  organic  response”  (p.  201),  schemas  have  been 
described  as  unconscious  mental  structures  (Brewer,  1987)  that  contain  networks  of  “interrelations...  hold  among 
the  constituents  of  concepts  in  question”  (Rumelhart,  1980,  p.  34).  Essentially,  according  to  Minsky  (1975),  it  is  a 
construct  that  organizes  our  prior  knowledge  in  a hierarchical  structure  that  contains  networks  of  nodes  and 
relations.  While  the  top-level  nodes  are  “fixed,  and  represent  things  that  are  always  true  about  the  supposed 
situation”  (p.  212),  the  bottom-level  nodes  contain  automatically  assigned  default  values  that  stipulate  the  conditions 
that  the  incoming  data  have  to  meet  in  order  to  be  accepted.  These  default  values— the  declarative  knowledge 
represented  by  propositions  -function  as  premises  that  rule  our  reasoning  processes  (Johnson-Laird,  1983).  Based 
on  these  propositions,  people  reason  in  the  format  of  if-then  productions,  which  serve  as  the  building  block  of 
human  cognitive  operations  (Anderson,  1983;  1993).  Those  proposition-based  declarative  knowledge  thus  become 
the  joints  where  the  mental  model  theory  dovetail  with  the  schema  theory  and  thus  formulate  the  integrated  internal 
representation  through  which  we  perceive  and  understand  the  world. 

In  this  study,  the  integration  of  schema  and  mental  models  is  presented  by  a four-level  structure,  i.e., 
global-level  schema,  middle-level  schema,  local-level  schema,  and  proposition  reasoning,  as  will  be  discussed 
below. 

Methodology 

All  the  data  were  collected  in  the  Children’s  Hearing  Foundation  in  Taiwan,  a non-profit  organization  that 
provides  free  Auditory -Verbal  therapy  (Auditory -Verbal  International,  1991)  and  audiology  services  for  hearing- 
impaired  children.  To  answer  the  question  “Given  the  same  aspiration  of  helping  children  to  learn  as  effectively  and 
efficiently  as  possible,  and  the  same  teaching  strategy  (Auditory- Verbal  approach),  why  and  how  teachers  and 
parents  still  teach  differently?”  purposive  sampling  was  used  to  select  and  recruit  parents  on  the  basis  of  the 
following  three  criteria:  (a)  they  must  have  attended  the  CHF’s  lessons  for  at  least  one  year;  (b)  they  must  be  the 
primary  care-takers  and  committed  to  the  task  of  teaching  their  children  to  speak;  and  (c)  they  accept  and  value  the 
importance  of  the  Auditory -Verbal  approach.  Three  mothers  and  their  corresponding  teachers  were  invited  to 
participate  in  the  study.  Information  about  these  teachers,  parents,  and  children  is  tabulated  in  Table  1. 


Children  & mothers 

Joey  & his  mother 

Kyle  & his  mother 

Tony  & his  mother 

Age  (child  /mother) 

5 / mid  30 

5.5  / mid  30 

5 / mid  30 

Siblings 

Only  child 

Has  a younger  brother 
(also  hearing  impaired) 

Has  an  elder  sister  (with 
normal  hearing) 

Mother’s  education 

College 

College 

College 

Years  in  the  CHF 

2 years  and  9 months 

1 year 

1 years  and  6 months 

Child’s  hearing  loss 

Severe -to-profound  / 
Cochlear  Implant 

Moderate-to-severe  / 
Wearing  hearing  aids 

Profound  / 
Cochlear  Implant 

Corresponding  teachers 

JiU 

Karen 

Tina 

Age 

Late  20 

Late  20 

Late  30 

Marriage 

Single 

Single 

Mother  to  2 boys 

Years  in  the  CHF 

3 years 

1 year  and  6 months 

1 years  and  6 months 

Time  with  the  child 

2 years  and  9 months 

1 years 

6 months 
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Table  1 . Background  information  of  the  three  teachers,  three  mothers,  and  three  children. 

Data  collection 

After  participants  were  recruited  and  before  any  data  collection  process  began,  the  researcher  spent  two 
weeks  in  the  foundation  to  get  acquainted  with  participants  in  order  to  build  a mutual  trust.  Over  the  span  of  three 
months,  interviews,  observations,  and  stimulated  recalls  (Bryan,  Bay,  Shelden,  & Simon,  1990;  Anthony,  1994; 

Clark  & Peterson,  1986;  Van  Noord  & Kagan,  1976)  were  conducted.  All  of  the  interviews  and  recalls  were 
audiotaped,  transcribed,  and  translated  from  Chinese  to  English.  An  external  audit  (Lincoln  & Cuba,  1985)  was 
invited  to  ensure  the  accuracy  of  the  transcriptions  and  translations. 

Interviews:  Two  interviews  for  each  participant  were  conducted:  one  at  the  beginning  and  one  at  the  end  of 
the  data  collection  processes.  The  interviews  were  open-ended  in  nature  to  elicit  a holistic  picture  of  participants’ 
perceptions  of  teaching. 

Observations:  For  each  teacher-parent  pair,  two  therapy  sessions  at  the  CHF  which  contain  representative 
episodes  or  interactions  that  evidenced  certain  thinking  processes  were  videotaped  and  fully  transcribed. 

For  each  mother,  two  teaching  sessions  at  home  were  videotaped  and  fully  transcribed.  To  reduce 
the  intrusion,  the  researcher  and  the  video  camera  were  far  away  from  the  teaching  site  as  much  as  possible. 

Stimulated  Recall  on  Videotapes 

After  each  videotaped  session  ended,  the  teachers  (for  the  CHF  sessions)  and  mothers  (for  CHF  and  home 
sessions)  were  asked  to  reflect  on  the  videotape  right  after  the  sessions  ended.  The  purpose  of  the  stimulated  recall 
was  to  (a)  reconfirm  the  researcher’s  observations,  (b)  allow  participants  a chance  to  organize  and  reexamine  what 
they  thought  and  felt  in  the  sessions,  and  (c)  expose  hidden  messages  that  were  otherwise  not  observable  in  the 
interactions. 

Data  analysis 

In  this  study,  two  types  of  data  were  collected:  (a)  what  people  did,  and  (b)  what  people  said.  The 
fieldnoted  videotape  transcriptions  were  first  broken  down  into  different  activity  groups,  e.g.,  reading  a story.  The 
constituent  unit  behaviors  and  their  relations  within  each  exchange  of  verbal  or  bodily  language  were  first  identified, 
coded,  and  represented  by  boxes  and  arrows.  These  intra -activity  models  are  called  micro -level  interaction  models. 
After  all  micro -level  interaction  models  are  completed,  patterns  of  people’s  launching  and  reacting  to  an  interaction 
across  different  teaching  activities  emerged.  These  emerged  patterns  are  thus  constructed  into  inter-activities 
models,  called  macro-level  interaction  models,  in  the  format  of  flowcharts  to  represent  people’s  knowledge  of 
conducting  a certain  teaching  procedure.  Because  this  model  concerns  knowledge  of  procedures  that  are  directly 
related  to  conducting  a teaching  activity,  it  constitutes  the  local-level  of  the  schema  of  teaching. 

After  all  the  interview  and  recall  data  were  transcribed  and  verified  by  the  participants,  they  were  first 
distilled  into  propositions.  As  the  gradation  of  their  relations  to  a specific  teaching  behavior  emerged,  these 
propositions  were  divided  into  three-level  schema  to  represent  the  hierarchy  of  thoughts. 

Global-level  schema:  composed  of  essential  attributes  (Wilson  & Rutherford,  1989,  p.  623)  that  represent 
prototype  (Hampton,  1991;  Medin,  1978)  conceptualization  of  teaching  that  emerged  from  data  as  the  quintessential 
notions  of  a typical  thinking  mode  that  formulates  the  central  tendency  of  a person’s  conceptualizing  a teaching  task 
in  a genuine  yet  context-free  teaching  scenario.  In  reference  to  Krathwohl’s  affective  taxonomy  (Krathwohl,  Bloom, 
& Masia,  1964),  this  is  analogous  to  the  “generalized  set”. 

Middle -level  schema:  composed  of  themes  or  beliefs  that  were  repetitively  voiced  by  the  participants  as  the 
bottom-line  principles  that  they  hold  on  to  as  a guidance  for  reasoning  or  behaving  toward  a certain  situation.  This  is 
corresponding  to  “commitment”  in  Krathwohl’ s affective  taxonomy. 

Local-level  schema:  composed  of  the  network  of  context -bound  and  teaching-behavior-related  propositions 
that  represent  the  declarative  knowledge  (Anderson,  1983)  of  conducting  a certain  teaching  procedure.  This  local- 
level  schema  from  verbal  data  will  be  matched  with  the  one  from  observation  data  (macro -level  interaction  models) 
on  the  basis  of  corresponding  constituent  behaviors  so  that  the  actions  and  the  knowledge  embedded  in  each  action 
are  integrated. 

Propositional  reasoning  productions:  as  every  teaching  task-related  proposition  implies  a goal  to  achieve, 
which  is  commonly  derived  from  the  global-  and  middle-level  schema,  propositions  with  the  same  goal  are  first 
grouped.  The  goal  serves  as  the  condition,  the  “if’  part,  of  an  if-then  production  (Anderson,  1993),  and  these 
propositions  make  the  action,  the  “then”  part  (or  counter-examples  as  pointed  out  by  Johnson-Laird,  1983).  These 
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constituent  behavior  based  if-then  productions  therefore  represent  the  underlying  cognitive  operations  of  each 
teaching  behavior. 

After  analyzed  with  this  framework,  the  teachers’  and  parents’  mental  models  for  teaching  are  compared 
and  the  most  significant  differences  are  presented  as  follow. 

Results 


Due  to  similar  training  and  education  backgrounds,  a striking  consistency  was  found  among  the  three 
teachers’  data.  Therefore,  these  data  were  combined  as  one  and  compared  to  the  three  mothers’  individual  data  on 
the  basis  of  the  four-level  mental  model  framework.  However,  to  avoid  redundancy,  at  the  three-level  schemas, 
teachers’  and  the  three  mothers’  models  are  compared  at  the  same  time;  but  to  reveal  the  individual  differences,  the 
teachers’  propositional  reasoning  models  are  compared  with  each  of  the  three  mothers’  individually.  The  following 
are  the  outcomes  of  the  comparison. 

Comparison  of  Global-Level  Schema 

The  teachers’  and  the  three  mothers’  global-level  schemas— the  prototype  conceptualization  of  teaching-are 
characterized  as  language-oriented  (the  teachers),  performance-oriented  (Joey’s  mother),  activity-oriented  (Kyle’s 
mother),  and  child -oriented  (Tony’s  mother).  For  the  teachers,  what  all  the  professional  knowledge  eventually  boils 
down  to  is  a two -in-one  concern:  how  to  improve  children’s  auditory  and  verbal  abilities.  While  interacting  with  a 
child,  resorting  to  all  the  techniques  and  strategies  at  the  back  of  their  mind,  the  teachers’  thoughts  were  entirely 
anchored  on  applying  the  Auditory -Verbal  approach  to  the  ongoing  interactions  to  help  the  child  learn.  In  other 
words,  the  prototype  conceptualization  of  the  teachers’  mental  processes  was  centered  on  the  notion  of  developing 
children’s  auditory  and  verbal  abilities  surrounded  by  the  techniques  and  strategies  that  realize  it.  In  contrast,  the 
notions  that  the  mothers  gravitated  toward  while  interacting  with  their  children  were  related  to  what  they  think  they 
need  to  do  to  help  their  children  learn  the  most  effectively  and  efficiently.  For  Joey’s  mother,  the  thought  that 
dominated  her  teaching  mind  was  “how  to  let  Joey  learn  as  much  and  as  effectively  as  possible”;  therefore, 
imitation,  correction,  and  full  concentration  were  the  core  features  of  the  prototype.  For  Kyle ’s  mother,  the 
prototype  of  the  teaching  mind  centered  on  “how  to  stimulate  Kyle’s  reasoning  ability  in  natural  interactions”.  Q & 

A interactions,  modeling  correct  answers,  and  conducting  various  activities  were  the  core  features.  Finally,  for 
Tony’s  mother,  the  prototype  of  the  teaching  mind  centered  on  “how  to  seize  the  moment  to  input  language 
expressions  for  Tony  to  listen  to  and  learn”.  Around  it,  the  core  features  consist  of  observing  Tony’s  behavior, 
incessantly  input  language  expressions,  and  pleasant  interactions.  These  prototypical  schemas  not  only  identify  the 
core  of  different  people’s  reasoning  processes,  more  importantly,  they  point  out  the  driving  forces  that  shape 
people’s  efforts  and  set  the  tone  for  perceiving  interactions. 

Comparison  of  Middle-Level  Schema 

The  comparison  of  the  teachers’  and  mothers’  middle-level  schemas,  i.e.,  attitudes  and  values,  reveals  the 
following  differences. 

1.  Meaningful  interaction:  There  was  a consensus  belief  among  participants  that  language  teaching 
is  meaningful  only  when  children  pay  attention  to  it.  However,  the  approaches  of  getting  their 
attention  onto  the  target  language  objects  varied  significantly.  The  teachers  and  Tony’s  mother 
used  rewards  to  trade  for  children’s  attention  and  cooperation,  whereas  Joey’s  mother  and 
Kyle’s  mother  usually  resorted  to  demanding  or  warning.  Such  a difference  is  derived  from  two 
assumptions  of  teaching:  (a)  teaching  is  to  capitalize  on  any  learning  opportunity  that  the  child’s 
attention  allows;  and  (b)  teaching  is  to  dictate  the  desirable  learning  behaviors  that  the  child 
should  enact. 

2.  Expectations:  The  teachers  based  their  expectations  for  children  on  their  current  abilities  and 
took  other  developmental  factors  into  account.  Kyle’s  mother  based  her  expectations  on  what 
she  thought  to  be  reasonable  for  children  of  Kyle’s  age  to  accomplish.  Joey’s  mother  based  her 
expectations  on  what  she  thought  to  be  necessary  for  Joey  to  achieve.  Tony’s  mother  closely 
follows  the  teacher’s  lead  and  based  her  expectations  on  the  progress  that  Tony  made  in  learning 
each  individual  language  target. 


«>  ^ BEST  COPY  AVAILABLE 

1140 


3. 


4. 


5. 


6. 


7. 


8. 


Pleasant  learning  interactions:  The  teachers  and  Tony’s  mother  believe  that  fun  and  learning  can 
and  should  go  hand  in  hand,  whereas  Joey’s  mother  and  Kyle’s  mother  would  rather  separate 
the  two  and  concentrate  on  the  learning  outcomes  while  in  lessons.  For  Joey’s  mother,  nothing 
was  more  important  than  getting  Joey  ready  for  school.  She  would  do  anything  to  push  and  help 
Joey  improve  his  language  ability,  having  fun  excluded. 

Productivity  in  contrast  to  effectiveness  of  learning:  The  mothers  shared  the  same  goal  of 
making  every  lesson  as  productive  as  possible;  the  difference  was  how  to  accomplish  it.  For 
Joey’s  mother,  being  productive  means  getting  more  accurate  responses  from  Joey,  so  she 
would  keep  correcting  and  pushing  him  until  he  gave  a correct  response;  for  Tony’s  mother  it 
means  striving  for  more  opportunities  for  conducting  verbal  interactions  with  Tony,  so  she 
would  never  stop  talking  while  she  was  with  Tony;  and  for  Kyle’s  mother,  it  means  more 
cognitive  stimulations  for  Kyle,  so  she  would  constantly  ask  Kyle  questions.  The  teachers,  on 
the  other  hand,  look  for  effectiveness  besides  productivity.  The  effectiveness  that  they  look  for 
is  a good-quality  learning  process  that  naturally  brings  out  good  learning  outcomes  in  a pleasant 
and  natural  learning  atmosphere. 

Time  pressure:  All  of  the  mothers  shared  the  same  eagerness  to  help  their  children  to  learn  as 
fast  as  possible.  In  one  way  or  another,  they  expressed  their  fear  that  time  with  their  children 
was  too  precious  to  be  wasted  on  things  that  do  not  have  direct  impact  on  their  children’s 
learning.  The  teachers  empathized  with  such  stress  and  anxiety,  but  those  emotions  were  not 
evidenced  in  the  way  they  taught  in  the  therapy  sessions  or  interviews.  Most  of  the  time,  what 
concerned  the  teachers  was  whether  or  not  the  child  had  made  progress  over  time.  In  contrast, 
what  concerned  the  mothers  was  whether  or  not  their  children  were  going  to  be  fine  in  a 
mainstreamed  school. 

Begin  with  approaches  in  contrast  to  begin  with  children:  The  teachers  rooted  their  thinking 
about  teaching  on  the  teaching  approaches  and  tailored  them  to  meet  individual  child’s  learning 
needs,  whereas  the  mothers  began  with  their  assumptions  of  their  children’s  learning 
propensities  and  customized  the  teaching  approaches  for  their  children.  The  difference  is  that 
the  former  may  lose  sight  on  the  mismatch  between  teaching  approaches  and  children’s  learning 
behaviors,  whereas  the  later  may  preclude  the  children  from  new  effective  learning  experiences. 
Professional  knowledge  in  contrast  to  experience:  When  the  mothers’  experiences  were  at  odds 
with  the  teachers’  professional  knowledge,  the  teachers’  advice  was  often  disregarded.  It  was 
observed  that  the  mothers  were  more  open  to  advice  regarding  technical  aspects  of  the  approach 
which  may  generate  more  observable  effects  such  as  strategies  for  stimulating  children’s 
speaking  and  listening  abilities  than  conceptual  aspects,  such  as  principles  for  shaping  children’s 
behaviors.  Especially  when  the  mothers  felt  their  approaches  had  led  to  a certain  reasonable 
success  in  the  past,  it  was  hard  for  them  to  take  the  teachers’  advice  to  their  hearts. 

Natural  interactions:  While  teachers  believed  that  learning  naturally  accumulates  in  normal-yet- 
language-targets -focused  interactions,  the  mothers  did  not  entirely  trust  the  processes,  or  at  least 
did  not  regard  it  as  enough.  The  didactic  nature  derived  from  their  own  educational  experiences 
often  led  the  mothers  to  address  the  learning  duties  that  the  children  had  to  fulfill  during  casual 
interactions.  That  is,  in  addition  to  normal  exchanges  of  language,  the  mothers  expect  more 
specific  learning  outcomes  and  emphasize  learning  efforts  such  as  memorization  from  the 
children. 


Comparison  of  Local-Level  Schemas 

The  local-level  schemas,  represented  by  networks  of  teaching  actions,  were  broken  down  to  sub-networks 
that  represent  different  parts  of  the  entire  teaching  procedure.  Teaching  behaviors  and  the  patterns  of  their  relations 
are  compared  between  the  teachers’  and  the  mothers’  schemas. 

1.  Before  lessons  begin:  The  atmosphere  and  settings  of  the  CHF  environment  are  designed  to  elicit 

the  best  learning  performance  from  children.  Having  come  to  the  therapy  sessions  for  at  least  a 
year,  the  children  knew  what  was  going  to  happen  and  how  they  were  expected  to  behave  in 
sessions;  therefore  they  were  usually  more  psychologically  prepared  for  the  teachers’  activities.  In 
contrast,  it  was  more  difficult  for  the  mothers  to  wait  and  seize  the  children’s  motivated  moments 
and  squeezed  in  language-focused  interactions  on  a twenty-four  hours  basis.  Therefore,  sometimes 
the  mothers  had  to  acquire  the  children’s  attention  by  more  forceful  way. 
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2.  Provide  verbal  stimuli:  Once  an  activity  was  chosen,  the  interactions  usually  began  with  a verbal 
stimulus.  Two  differences  were  observed  in  the  way  that  the  teachers  and  mothers  presented  the 
verbal  stimuli  by  the  Auditory- Verbal  approach.  First,  given  their  trust  and  acceptance  of  the 
approach,  the  mothers  did  not  exercise  it  as  proficient  as  the  teachers  did.  The  reason  may  be  due 
to  negligence  or  implicit  resistance  to  certain  techniques  (e.g.,  a mother  said  “I  didn’t  provide 
reinforcers  because  children  need  to  learn  to  study  independently  whether  having  a reinforcer  or 
not).  Second,  the  teachers  have  a lot  more  flexibility  in  changing  materials  or  activities  than  the 
mothers.  This  is  because  that  the  teachers  have  more  teaching  resources  than  the  mothers  while  the 
mothers  have  more  predetermined  teaching  objectives  for  the  lessons. 

3.  Problem-solving  processes:  Believing  that  children  need  to  learn  to  solve  problems  by  themselves, 
the  teachers  often  intentionally  brought  children  into  a problem-bound  situation,  in  which  the 
teachers  facilitated  the  thinking  process  and  carefully  associated  children’s  problem-solving 
behaviors  with  language.  This  effort  was  not  observed  in  the  mothers’  teaching  schema.  The 
mothers  either  predicted  the  consequence  of  children’s  actions  to  keep  children  from  making 
mistakes  or  told  them  what  to  do  right  after  the  problem  occurred. 

4.  Incorrect  response:  The  three  mothers  demonstrated  three  styles  of  handling  their  children’s 
inabilities  to  respond  to  stimuli.  Joey’s  mother  accepted  nothing  but  accurate  responses;  therefore, 
regardless  of  Joey’s  motivation  level,  she  would  keep  asking  Joey  to  try  until  he  could  respond 
correctly.  Kyle’s  mother  preferred  natural  interactions,  so  she  would  add  only  one  or  two  extra 
clues  and  then  disclosed  the  correct  response  quickly.  Tony’s  mother  primarily  concerned  about 
inputting  language  expressions  for  Tony  to  listen  to  and  learn;  therefore,  instead  of  correcting  his 
mistake,  she  would  follow  his  reaction  and  verbally  describe  his  response  for  him.  On  the  other 
hand,  the  teachers  would  break  down  the  stimulus,  add  extra  clues,  or  repeat  the  stimulus  to  help 
the  child  understand.  Meanwhile,  they  always  checked  on  the  children’s  motivation  levels  to  avoid 
building  any  association  between  adverse  emotions  and  learning. 

5.  Non-verbal  response:  When  the  children  gave  non-verbal  responses,  the  teachers  and  Joey’s 
mother  would  try  to  help  the  children  respond  verbally.  Kyle’s  mother  usually  did  not  notice  the 
difference.  She  would  accept  non-verbal  responses  and  comment  on  the  accuracy  of  the  responses. 
Tony’s  mother  would  help  Tony  to  respond  verbally  only  when  he  had  demonstrated  the  ability  of 
saying  it  correctly  in  the  past. 

6.  Intelligibility  check  and  imitation:  Upon  receiving  a verbal  response  from  a child,  the  teachers  and 
mothers  had  to  check  its  intelligibility  to  decide  if  an  imitation  was  needed.  Joey’s  mother  always 
asked  Joey  to  imitate  the  accurate  words  or  phrases  repetitively  until  he  did  it  right.  Tony’s  mother 
only  asked  Tony  to  imitate  when  she  knew  that  he  had  done  it  before.  Kyle’s  mother  did  not  ask 
for  imitation  very  often,  but  when  she  did,  she  would  correct  her  children’s  tongue  positions  in 
addition  to  having  the  children  listen  and  imitate.  The  teachers  would  ask  children  to  imitate  only 
when  children  were  ready  to  do  it,  and  only  do  a few  rounds  of  practice  to  avoid  stressing  children 
out.  In  addition,  visual  cues  were  used  when  nothing  else  worked  and  would  be  taken  away 
immediately  after  the  pronunciation  was  correctly  made. 

7.  Reinforcement,  expansion,  feedback,  and  turn -taking:  All  of  the  participants  believed  in  the 
effectiveness  of  reinforcement,  so  they  more  or  less  would  give  children  rewards  or  compliments 
when  they  performed  well.  The  feedback  that  the  teachers  and  Tony’s  mother  provided  in  reaction 
to  the  child’s  response  included  repeating  the  child’s  responses  and  providing  evaluation  on  its 
accuracy  in  a positive  way.  Joey’s  mother  and  Kyle’s  mother  usually  provided  only  judgment. 

With  the  possible  exception  of  Kyle’s  mother,  all  participants  wuold  try  to  expand  the  children’s 
language  whenever  possible.  Turn -taking  was  a technique  that  the  mothers  seldom  apply  at  home. 

Comparison  of  Propositional  Reasoning  Processes 

In  order  to  reveal  the  differences  between  the  teachers’  and  the  mothers’  mental  models  in  detail,  each  of 
the  three  mothers’  propositional  reasoning  processes  is  compared  individually  to  the  teachers’. 

Joey’s  Mother  and  the  Teacher: 

1)  Academic  oriented  in  contrast  to  language  oriented:  If  Joey’s  mother/teachers  want  to  help  Joey  to  keep  up  with 
other  children  in  school, 

a)  Joey’s  mother  will  teach  him  the  academic  materials  in  advance. 

b)  The  teachers  will  focus  on  helping  him  to  establish  good  language  ability  first. 
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2)  Push  for  maximum  performance  in  contrast  to  leading  into  best  performance:  If  Joey’s  mother/teachers  want  to 
help  Joey  learn  as  much  as  possible, 

a)  Joey’s  mother  will  keep  talking  to  him,  correct  every  mistake  he  makes,  and  keep  pushing  for  a good 
imitation  until  he  gets  it  right, 

b)  The  teachers  will  provide  motivators  to  stimulate  his  learning  interests,  and  conduct  language-focused 
interactions  in  designed  activities  or  normal  daily  communications, 

3)  Demand  him  to  do  it  right  in  contrast  to  help  him  to  do  it  right:  If  Joey  gives  an  incorrect  response  but  Joey’s 
mother/teachers  believe  Joey  can  and  want  to  help  him  respond  to  the  question  correctly, 

a)  Joey’s  mother  will  feel  unacceptable,  raise  her  voice  to  demand  his  full  attention,  repeat  the  same  question, 
and  be  reluctant  to  provide  more  clues  because  she  does  not  like  to  lower  the  standard, 

b)  The  teachers  will  reexamine  his  standing  in  the  course  of  language  development  and  rephrase  or  provide 
more  clues  to  help  him  respond  correctly, 

4)  Make  the  child  to  learn  effectively  in  contrast  to  look  for  effective  teaching  for  the  child:  If  Joey’s 
mother/teachers  want  to  help  Joey  to  learn  effectively, 

a)  The  mother  will  create  a serious  learning  atmosphere  and  demand  complete  concentration  to  make  him  take 
learning  seriously  to  learn  effectively, 

b)  The  teachers  will  identify  where  his  attention  is  at  and  present  prioritized  language-focused  interactions  in  a 
meaningful  way  to  either  attract  the  child’s  attention  or  to  follow  it, 

5)  Ignore  distraction  in  contrast  to  capitalize  on  distraction:  If  Joey’s  mother/teachers  want  to  help  Joey  learn  as 
much  as  possible  but  are  disrupted  by  an  unexpected  incident  in  a lesson, 

a)  Joey’s  mother  will  consider  the  incident  as  a distraction,  and  will  ignore  it  and  bring  Joey’s  attention  back 
to  the  planned  lesson  as  soon  as  possible, 

b)  The  teachers  will  capitalize  on  the  incident  and  associate  it  with  corresponding  language  expressions, 

6)  Eliminate  the  negatives  in  contrast  to  capitalize  on  the  positives:  If  Joey’s  mother/teachers  want  to  help  Joey 
learn  from  the  mistake  he  made, 

a)  The  mother  will  incessantly  correct  any  mistake  he  made  and  resort  to  punishment  to  keep  him  from 
making  the  same  mistake  again, 

b)  The  teachers  will  emphasize  the  correct  part  of  the  response,  associate  it  with  the  desired  response,  or 
directly  model  the  correct  response  when  such  association  does  not  work, 

7)  Model  solution  in  contrast  to  guide  thinking  processes:  If  Joey’s  mother/teachers  want  to  help  Joey  to  learn  how 
to  solve  a problem, 

a)  Joey’s  mother  will  verbally  or  physically  model  the  solution  for  him  to  imitate  and  learn,  and  expect  him  to 
memorize  it, 

b)  The  teachers  will  provide  more  visual  or  verbal  cues  to  help  him  understand  the  problem  and  facilitate  the 
thinking  process  to  guide  him  to  solve  the  problem  successfully, 

Kyle’s  Mother  and  the  Teacher 

1)  Activity-oriented  in  contrast  to  language-oriented:  If  Kyle’s  mother/teachers  want  to  help  Kyle  learn  through  an 
activity, 

a)  Kyle’s  mother  will  primarily  focus  on  the  activity  itself  and  try  to  generate  successful  outcomes  out  of  it, 

b)  The  teachers  will  try  to  create  relationships  among  people,  objects,  or  the  surroundings  in  the  activity  to 
demonstrate  different  language  features, 

2)  Reading  in  contrast  to  meaningful  interactions:  If  Kyle’s  mother/teachers  want  to  help  Kyle  learn  different 
language  expressions, 

a)  Kyle’s  mother  will  teach  by  reading  books, 

b)  The  teachers  will  associate  the  language  expressbns  with  Kyle’s  personal  experiences 

3)  Quantity  of  interactions  in  contrast  to  quality  of  interactions:  If  Kyle’s  mother/teachers  want  to  improve  Kyle’s 
general  language  abilities, 

a)  Kyle’s  mother  will  conduct  more  language  interactions  with  him  to  compensate  for  her  relatively 
unfocused  language  interactions, 

b)  The  teachers  will  conduct  language-targets -focused  language  interactions  with  him  and  expose  him  to  an 
environment  abundant  with  reasonably  advanced  language  features, 

4)  Make  transition  to  the  future  in  contrast  to  do  what  is  working  now:  If  Kyle’s  mother/teachers  want  to  get  Kyle 
to  learn, 

a)  Kyle’s  mother  will  provide  no  motivators  other  than  verbal  encouragement  because  that  is  what  happens  in 
schools. 
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b)  The  teachers  will  either  present  a motivator  in  advance  or  provide  reinforcers  afterward  to  elicit  good 
learning  behaviors. 

5)  Memorization  in  contrast  to  comprehension:  If  Kyle’s  mother/teachers  want  to  teach  Kyle  a new  concept, 

a)  Kyle’s  mother  will  describe  it  for  Kyle  to  listen  to  and  have  him  memorize  it. 

b)  The  teachers  will  provide  more  visual  or  verbal  cues  to  bridge  the  gap  between  the  stimulus  and  the  child’s 
cognitive  level,  and  facilitate  the  thinking  process  to  guide  the  child  to  comprehend  successfully. 

6)  Memorization  in  contrast  to  role  playing:  If  Kyle’s  mother/teachers  want  to  help  Kyle  learn  different  language 
expressions  related  to  different  roles, 

a)  Kyle’s  mother  will  model  the  expressions  for  him  to  memorize  and  hope  that  he  practices  on  his  brother. 

b)  The  teachers  will  have  the  child  play  different  roles  and  help  him  learn  the  language  expressions  associated 
with  those  roles. 

7)  Warning  and  punishment  in  contrast  to  enticing  motivation:  If  Kyle’s  mother/teachers  want  to  get  Kyle  to 
concentrate  on  the  lesson, 

a)  Kyle’s  mother  will  warn  him  and  let  him  know  that  she  will  be  mad  if  he  does  not  behave. 

b)  The  teachers  will  provide  motivators  or  change  to  a more  interesting  activity. 

8)  Combination  of  approaches  in  contrast  to  Auditory -Verbal  approach  only:  If  Kyle’s  mother  wants  to  help  her 
children  make  an  accurate  pronunciation, 

a)  Kyle’s  mother  will  model  the  correct  pronunciation  and  its  basic  elements  for  him  to  imitate  first,  and  then 
correct  his  tongue  position. 

b)  The  teachers  will  help  him  learn  to  make  accurate  pronunciation  by  associating  the  sound  they  modeled  for 
himwith  the  sound  he  made. 

Tony’s  Mother  and  the  Teacher 

1)  Need  to  know  the  child’s  progress  in  contrast  to  trust  the  processes:  If  Tony’s  mother/teachers  want  to  design 
activities  for  Tony  to  learn  something, 

a)  Tony’s  mother  will  design  activities  that  begin  with  commands  or  questions  for  him  to  respond  to  test  how 
well  he  understands  her. 

b)  The  teachers  will  present  abundant  language  expressions  in  accordance  with  his  language  capabilities  to 
establish  his  habit  of  listening. 

2)  Language  learning  with  top  priority  in  contrast  to  language  learning  grounded  in  integrated  child  education:  If 
Tony’s  mother/teachers  want  to  keep  Tony  staying  in  an  activities  so  he  can  learn, 

a)  Tony’s  mother  will  keep  him  from  feeling  frustrated,  tolerate  poor  manners  of  learning,  and  allow  him  to 
have  more  rewards  than  he  deserves  so  he  will  enjoy  the  activities. 

b)  The  teachers  will  teach  him  how  to  deal  with  frustration,  fairness,  and  accept  no  inappropriate  behaviors. 

3)  Ride  on  his  inappropriate  response  in  contrast  to  insist  the  right  way  to  respond:  If  Tony’s  mother/teachers 
want  to  help  Tony  learn  when  he  intentionally  ignore  the  accurate  response  and  responding  to  stimuli  the  way 
he  wants, 

a)  Tony’s  mother  will  forget  about  the  stimuli  and  tries  to  associate  his  incorrect  response  with  corresponding 
language  expressions  instead. 

b)  The  teachers  will  first  make  sure  the  stimulus  has  been  clearly  presented,  and  second,  make  sure  he  learn 
and  demonstrate  the  association  between  the  stimulus  and  its  appropriate  response  correctly  at  least  once. 

Discussion 

While  the  global-level  schema  describes  the  context-free  prototype  conceptualization  of  teaching,  middle- 
level  schema  represents  participants’  general  values  about  teaching,  local-level  schema  describes  the  context - 
specific  teaching  procedures,  and  propositional  reasoning  represents  the  cognitive  computations  behind  each 
teaching  behavior.  Based  on  this  four-level  framework,  the  comparison  of  three  teachers’  and  three  mothers’mental 
models  reveals  striking  differences.  Given  the  fact  that  the  teachers  and  the  mothers  shared  the  same  goal,  i.e.,  to 
help  their  children  to  learn  language  as  effectively  and  efficiently  as  possible,  and  that  they  all  highly  value  the 
importance  of  the  Auditory- Verbal  approach  for  their  children,  what  these  differences  show  is  that  training  is  not 
about  delivering  knowledge  but  about  transforming  a cognitive  system.  If  such  transformation  takes  place  only  at 
local-level  schema,  e.g.,  learning  the  techniques  of  the  Auditory -Verbal  approach  but  not  the  attitudes  toward  the 
nature  of  children’s  learning,  the  newly  learned  will  be  interpreted  in  accordance  with  the  existing  value  systems, 
and  the  outcomes  of  such  training  will  often  be  different  than  the  trainers  would  have  expected.  Oftentimes,  what 
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matters  to  a child’s  learning  experiences  is  not  how  accurate  a teacher  or  mother  can  execute  a teaching  technique, 
but  the  subtle  facial  expressions,  pace,  tone  of  speech,  bodily  language  that  reflect  a person’s  fundamental  attitudes 
that  are  conveyed  through  executing  these  techniques.  While  training  a subject  matter  whose  application  is 
inseparable  of  personal  attitudes  and  subjective-decision  making,  understanding  how  the  subject  matter  is  mentally 
represented  is  crucial  for  the  success  of  the  training, 

Furthermore,  all  the  descriptions  of  the  teachers’  and  the  mothers’  lower  level  schemas  should  be  viewed  in 
the  perspectives  of  higher  level  schemas.  On  the  other  hand,  propositional  reasoning,  although  demonstrated  as 
closely  attached  to  a particular  unit  behavior,  is  actually  operated  on  the  basis  of  the  whole  schema  system. 
Therefore,  this  four-level  framework  represent  four  aspects  of  the  internal  representations  that  we  construct  for 
comprehending  the  encountered  entites  in  an  integrated  yet  dynamic  format. 

Finally,  the  comparison  outcomes  show  that,  comparing  to  the  mothers,  the  teachers  are  more  process- 
oriented,  flexible  yet  prepared,  and  fun-driven.  They  believe  in  enhancing  the  positive  performance  and  adjusting 
their  approaches  to  deal  with  children’s  lack  of  progress  or  learing  motivations.  With  these  teaching  inclinations  as 
the  platform  for  the  langauge-oriented  prototypical  teaching  mind,  which  sets  the  direction  for  cognitivly  processing 
interactions,  all  the  thinking  and  teaching  behaviors  gravitate  toward  building  children’s  auditory  and  verbal  abilities 
in  the  way  in  accordance  with  Auditory -Verbal  approach.  On  the  other  hand,  the  mothers  are  more 
outcome/performance-driven,  and  goal-minded,  Joey’s  mother’s  performance-oriented  teaching  mind  results  in  a 
error-eliminating,  negativly  reinforcing,  highly  controlling  type  of  teaching  style,  which  concerns  mainly  about  how 
accurate  Joey’s  responses  are  and  how  serious  that  Joey’s  learning  attitude  is.  Kyle’s  mother’s  activity-oriented 
teaching  mind  result  in  an  activity-abundant  yet  language-unfocused  teaching  style,  which  compensate  for  less 
skillfully  conducted  interactions  by  increasing  the  quantity  of  interactions.  On  the  other  hand,  Tony’s  mother’s 
child -oriented  teaching  mind  generates  a fun-loving,  encouraging,  everything-for-langauge-interactions  style,  which 
weighs  eliciliting  Tony’s  learning  motivation  over  many  other  equally  important  developmental  concerns.  What 
these  differences  indicate  is  that  people  conceptualize  the  task  of  teaching  in  different  ways.  They  value  different 
aspects  or  outcomes  of  teaching,  and  thus  are  more  open  to  information  that  is  based  on  the  same  predispositions  of 
teaching.  Developed  from  their  personal  learning  experiences,  expectations,  or  maternally  instincts,  these  teaching 
predispositions  constitute  the  personally -defined  optimal  teaching  package  that  they  believe  is  for  their  children.  It 
not  only  takes  a lot  of  mutual  understanding  and  empathy  to  make  clear  what  the  differences  really  are  about,  it  also 
takes  a lot  of  acclimatization  to  do  something  different  from  what  they  have  accustomed  to  do. 

Revealing  these  background  knowledeg-rooted  internal  representations  of  teaching  is  to  improve  the 
mutual  understanding  between  teachers  and  parents.  Especially  in  special  education  where  parents’  teaching  efforts 
are  crucial  for  children’s  development,  when  the  mothers’  perceptions  of  teaching  are  intricately  intertwined  with 
parental  expectations,  responsibilities,  feelings  of  guilt,  and  attitudes  toward  the  child’s  handicap,  teachers’ 
unawareness  of  these  background  issues  may  result  in  totally  off-the-mark  interactions  between  teachers  and 
parents,  A study  of  three  teachers  and  three  mothers  does  not  bear  too  much  generalization  contribution.  However, 
the  intricacy  and  the  integrity  of  mental  models  could  not  be  demonstrated  with  reasonable  power  of 
representativeness  if  no  lengthy  and  in-depth  qualitative  data  collection  and  analysis  procedures  are  conducted.  This 
study  intends  to  propose  a framework  for  representing  mental  models,  and  to  build  three  teachers’  and  three 
mothers’  teaching  models  as  a window  for  understanding  how  teaching  is  conceived  during  teaching.  The  next  step 
is  to  take  these  information  as  a given  and  figure  out  the  way  of  promoting  the  effectiveness  of  interactions  between 
teachers’  and  parents’  models. 
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Evaluation  of  instruction  (training):  It  is  NOT  optional  for  professionals  (or,  who  screened 

the  baggage  on  your  flight?) 
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Abstract 


Training  is  a technical  term,  applicable  to  interventions  that  result  in  a performance  outcome,  however,  the 
term  is  often  used  inappropriately  to  elevate  conglomerations  of  content  to  training  status  even  though  no 
performance  improvement  results  or  is  ever  likely  to  result.  Myths  about  evaluation,  that  it  is  optional,  or 
too  expensive,  or  somehow  unfair  to  learners  (all  false  statements)  persist  because  of  a lack  of  competence 
in  evaluation  skills  on  the  part  of  a majority  of  practitioners.  Actually  evaluation  is  a critical  component  in 
effective  and  efficient  (and  therefore  cheap  in  total  cost)  training,  and  it  is  unfair  to  learners  to  expect 
performance  improvement  on  the  job  unless  the  training  has  been  proven  through  rigorous  evaluation.  The 
current  rates  of  change  in  both  technical  and  non-technical  arenas,  as  well  as  a slowing  economy  and 
tightening  budgets  contribute  to  a reality  where  fewer  and  fewer  people  can  afford  to  throw  away  hours  of 
instruction  or  training  on  nonfunctional  conglomerations  of  content.  Rigorous,  full  evaluation  is  simply 
not  optional  for  Instructional  Technology  professionals. 

Introduction 

Although  calls  for  data  in  the  field  of  Instructional  / Performance  technology  continue  from  every  direction 
(e.g.  Anglin,  et  al,  2000;  IBSTPI,  1998;  Gery,  1999;  Merrill,  et.  al,  1996;  Shrock,  1999),  the  number  of  publications 
which  include  meaningful  evaluation  is  not  increasing,  and  may  be  decreasing  (Werner  & Klein,  2000),  Evaluation 
is  often  considered  to  be  optional  or  too  expensive,  or  is  simply  not  a part  of  the  project  from  the  beginning.  Some 
customers  and  clients  have  been  misinformed  in  the  past  that  evaluation  is  expensive,  or  unnecessary,  or  somehow 
unfair  to  the  learners  (all  these  assumptions  are  false).  In  fact,  evaluation  is  not  only  not  expensive,  it  is  the  cheapest 
step  in  the  process  and  the  best  practice  for  all  involved.  It’s  also  not  at  all  time-consuming  if  the  intervention 
involves  observable,  measurable  performance  outcomes  and  the  practice  is  sufficient  and  aligned  to  those  outcomes 
(inherent  in  the  definition  of  training,  so  an  intervention  without  observable,  measurable  outcomes  is  simply  not 
training).  In  fact,  the  most  expensive  choice  anyone  could  make  is  to  forego  evaluation  and  release  dysfunctional  or 
non-functional  interventions  that  then  consume  vast  amounts  of  resources  in  terms  of  organizational  resources, 
material  costs,  and  learner  time  and  effort.  In  addition,  most  unmeasured  “training”  sets  learners  up  to  fail  - thus 
impacting  their  subsequent  learning  efforts  far  into  the  future, 

A case  study,  utilizing  a technology-based  intervention  to  address  a need  for  skill  development  by  new 
hires  in  a semiconductor  fabrication  facility  (Fab),  is  provided  to  illustrate  the  evaluation  process,  including 
summative  evaluation,  and  demonstrates  the  necessity  for  evaluation  as  well  as  the  total  value  (costs  and 
benefits)  of  routine  and  systematic  evaluation  of  interventions.  The  application  of  Level  1 (trainee 
response)  data.  Level  2 (posttest  performance)  data,  and  Level  3 (on-the-job  performance)  data  to  revise  the 
intervention  until  it  functions  at  the  required  level  to  meet  the  business  need  are  described. 

Foundations 

Instructional  technology  is  the  application  of  expertise  (in  the  areas  of  human  information  processing, 
learning,  performance,  cognition,  etc.)  to  solve  real-world  gaps  in  skills  and  knowledge  through  specially 
designed  interventions.  It  is  this  expertise,  along  with  effectiveness  and  efficiency  data  from  previous 
interventions,  that  enables  us  to  develop  instruction  that  works  for  the  target  audience.  Although  it  seems  to 
be  lost  on  a large  number  of  practitioners,  the  profession  of  instructional  technology  is  rooted  in 
observable,  measurable  performance  outcomes,  under  conditions  where  the  consequences  of  failure  are 
serious;  after  all,  the  application  of  learning  theory  to  solve  real-world  problems  effectively  and  efficiently 
really  took  off  as  experts  worked  with  the  military  to  prepare  soldiers  to  fight  and  survive  in  many  types  of 
warfare  (see  Gagne,  1989;  Dick  & Carey,  1985;  Anderson,  1995;  National  Research  Council,  1991), 

As  we  work  to  improve  the  performance  of  others,  we  must  role  model  performance  to  business  indicators 
ourselves,  and  demonstrate  that  discipline  around  assessment  and  accountability  are  fundamental  to  continuous 
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improvement.  Using  our  own  tools  while  demonstrating  efficiency  and  effectiveness  and  performing  at  least  to  the 
levels  we  demand  of  others  are  both  critical  components  of  our  credibility.  Our  customers  are  nearly  always  held  to 
some  level  of  minimum  performance;  can  you  imagine  a factory  manager  who  is  only  measured  according  to  the 
amount  of  raw  materials  going  into  the  factory?  For  this  scenario,  actual  production  of  working  units  is  not 
measured,  ever.  Training  which  is  not  measured  according  to  actual  performance  outcomes  of  trainees  after  training 
(i.e.,  training  tracked  by  a body  count  or  attendance,  and  with  opinion  surveys)  is  very  analogous  to  the  Factory 
Manager  who  is  measured  only  by  how  much  raw  material  goes  into  the  factory.  Our  customers  and  clients  deserve 
better;  the  standards  of  our  profession  demand  better  (see  standards  at  AECT  site,  http://www.aect.org  and  at 
IBSTPI  at  httn://www.ibstDi.org:  consider  reviewing  the  new  ISO  requirements  for  training  also).  It*s  impossible 
for  measurement  to  be  superfluous,  to  be  expensive  in  terms  of  total  cost,  or  be  time  consuming  in  terms  of  total 
resources.  One  basic  premise  of  both  Total  Quality  and  Continuous  Improvement  is  that,  if  you  can’t  measure  what 
you’re  doing  then  you  don’t  know  what  you’re  doing  (measurement  is  critical  to  improving  both  process  and 
product;  remember,  it’s  possible  to  change  for  the  worse).  This  is  so  very  critical  because  an  intervention  isn’t 
training  because  the  word  training  appears  in  the  title.  Training  is  only  training  if  it  functions  to  enable  a permanent 
(or  near  permanent)  performance  change  that  is  observable  and  measurable. 

The  Premise 

There  are  clearly  issues  with  the  present  status  of  the  profession  of  performance  or  instructional 
technology.  Training  is  treated  as  a joke,  as  a necessary  evil,  as  too  expensive  or  too  difficult  to  do  well,  and  the  low 
expectations  of  end  users  (trainees)  and  performance  owners  (managers)  enable  the  practitioners  to  perpetuate 
content  masquerading  as  training.  The  same  problems  have  been  discussed  over  and  over  in  the  past  twenty  years  of 
Training  & Development  (an  American  Society  of  Training  & Development  publication),  as  well  as  in  articles 
published  inside  and  outside  the  field.  The  complaints  are  consistent  across  the  years,  and  are  detailed  in  articles 
like:  Fumham’s  Fire  the  training  department,  (1997),  Kruger,  & Dunning’s  Unskilled  and  unaware  of  it:  How 
difficulties  in  recognizing  one's  own  incompetence  lead  to  inflated  self-assessments  (1999),  and  Armour’s 
lesson:  Billions  wasted  on  job-skills  training  (1998),  just  to  name  a few.  For  example,  in  a USA  Today  article. 
Armour  (1998)  explains  provides  some  insights,  “Faced  with  crippling  skill  shortages,  employers  are  spending 
skyrocketing  amounts  of  money  training  workers.  The  problem?  Many  programs  just  don’t  work.  Billions  of 
dollars  are  spent  on  wasteful  training  courses,  experts  say  . . . $5.6  billion  to  $16.8  billion  is  wasted  annually  on 
ineffective  training  programs,”  and  even  provides  a solution,  “ . . . experts  say  businesses  should  check  for  results  to 
separate  effective  programs  from  costly  gimmicks.  ‘American  industry  is  spending  billions  and  billions  on  training 
programs  and  doing  no  evaluation  of  their  effectiveness,’  says  Cary  Chemiss,  a professor  at  Rutgers.  ‘You  have  to 
measure  it.’  ” 

Case  Study 

Semiconductor  Equipment  Operation  Training 

The  intervention  was  designed  to  meet  a business  need;  new  hires  needed  skills  training  to  work  in  a 
semiconductor  wafer  Fab.  SortSoft  was  a new  station  controller  software  interface  for  the  Schlumberger  9000 
(S9K)  tester.  The  SortSoft  Computer  Based  Training  (CBT)  package  is  based  on  a full,  reakime  simulator  of  the 
SortSoft  interface,  and  therefore  was  expected  to  reduce  certification  time  for  level  1 Technicians  in  Sort  areas 
across  the  company,  as  well  as  reducing  the  trainer  time  (for  technician  trainers  or  engineers/equipment  owners)  and 
S9K  tester  (production)  time  required  for  training  functions. 

The  project  was  unique  because  it  was  the  company’s  first  full,  real-time  simulator  built  to  run  on  a PC  and 
supported  by  a fully -aligned  CBT.  In  addition,  concurrent  development  was  required  as  the  training  was  to  be 
available  to  support  the  implementation  of  the  product  at  each  Fab  site.  The  objectives  of  the  training  package 
accurately  reflect  the  certification  checklist  from  the  factories.  In  fact,  the  team  nearly  gained  approval  for  the 
course  completion  to  stand  for  certification.  The  equipment  owner  engineers  (those  responsible  for  the  operation, 
up-time,  maintenance,  training  and  certification  for  the  equipment  they  “own”)  acknowledged  the  alignment  of  the 
training  and  felt  that  the  posttest  did  reflect  the  actual  operations  skills  required.  Although  several  additional  alarm 
sequences  were  added  to  the  wafer  sorting  segment  of  the  posttest,  the  engineers  did  not  accept  it  as  full  certification 
because  they  wanted  to  check  for  themselves  that  each  operator  could  demonstrate  all  the  skills  on  an  actual  tester. 

At  the  time  the  team  began  initial  design,  a checklist  was  created  covering  all  the  skills  required  for 
certification  on  the  SortSoft  interface.  The  checklist  was  used  for  updating  the  skills  of  technicians  certified  to 
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operate  the  existing  interface  at  the  development  facility.  That  checklist  was  also  used  as  a basis  for  another 
recording  instrument;  equipment  owner  engineers  were  asked  to  carefully  track  the  time  it  took  to  re -train  the 
technicians  on  the  new  interface  and  to  train  new  technicians  as  well.  Seven  new  technicians  were  trained  at  the 
development  Fab,  and  the  recorded  times  for  training  new  technicians  were  used  as  a baseline  for  comparison  in 
measuring  the  efficiency  of  the  CBT/Simulator  training  intervention.  It  is  interesting  to  note  that  the  equipment 
owner  engineers  had  previously  asked  to  provide  training  time  requirements  by  the  programmer  initially  assigned  to 
the  project  and  their  responses  bore  no  relationship  to  the  actual  times  tracked  during  technician  training  in  the 
development  Fab.  One  engineer  reported  training  times  as  much  as  50%  in  excess  of  the  actual  time  required,  all 
other  engineers  reported  training  times  more  than  50%  below  the  actual  time  required.  The  instructional  designer, 
aware  of  the  reliability  and  value  of  self-reported  data,  created  the  time  tracking  instrument  and  carefully 
communicated  the  purpose  and  importance  of  tracking  time  spent  on  training.  Although  the  engineers  did  not  match 
the  specificity  of  time  tracking  achieved  with  the  CBT/Simulator  intervention,  the  times  reported  were  more 
accurate  and  therefore  a meaningful  basis  for  comparison. 

Usability  / Functionality  Testing 

Basic  usability  testing,  even  for  paper-based  manuals,  precedes  any  performance  testing;  functionality 
testing  can  easily  run  concurrently  with  usability  testing.  Usability  means  that  people  can  follow  the  directions,  use 
the  documentation  or  intervention  successfully,  not  get  lost  or  frustrated  while  trying  to  use  the  materials,  and  not 
have  trouble  figuring  out  what  to  do  next.  Functionality  testing  means  the  materials  work  consistently  like  they  are 
supposed  to:  the  exercises  or  practice  items  are  aligned  to  what  the  trainees  are  learning  to  do  and  anyone  who  has 
completed  the  instruction  should  be  able  to  complete  most  or  all  of  them  correctly;  the  answers  as  well  as  steps  for 
successful  completion  are  provided  as  appropriate  feedback  when  required;  scenarios  or  simulations  are  provided 
where  appropriate  to  learning  outcomes;  and  the  thing  works  (application  runs,  links  link,  manual  is  accurate  and  has 
complete  table  of  contents,  index,  and  sufficient  and  accurate  instructions/directions  to  users).  Once  people  can  get 
through  it  successfully,  then  a tryout  or  pilot  test  will  provide  critical  feedback  on  effectiveness  and  efficiency  of  the 
intervention. 

In  the  SortSoft  case,  content  experts  and  the  test  engineers,  using  an  explanatory  guideline  and  checklist 
created  for  that  purpose,  also  reviewed  the  intervention.  In  some  cases,  the  equipment  owner  engineers  requested 
that  the  content  be  restructured  or  reorganized  in  ways  that  made  it  more  useful  for  experts  (they  themselves)  but 
NOT  for  the  novices  who  made  up  the  target  audience,  and  those  changes  were  refused  (with  extensive 
communication  about  why). 

During  usability  and  functionality  testing,  we  identified  necessary  revisions.  We  installed  a dialog  box  to 
pop-up  when  the  trainee  selected  Exitfrom  the  application  menu,  so  a trainee  could  choose  to  return  to  the  CBT  if 
they  did  not  actually  want  to  exit  the  CBT.  Also,  information  and  content  were  presented  in  a different  colored  text 
than  actual  instructions  for  using  the  simulator  and  directions  for  completing  practice  items.  Some  small 
instructional  changes  were  made  to  revise  confusing  practice  items  or  add  detail  to  feedback  at  the  end  of  several 
modules.  At  the  request  of  the  equipment  owner  engineers,  two  different  alarms  which  routinely  occur  during 
typical  wafer  processing  were  added  in  two  the  final  practice  scenario  and  also  into  the  posttest  simulation  of  three 
wafer  lots. 

Most  surprising  of  all  was  the  behavior  of  all  the  testers  in  regard  to  the  interactivity  level  of  the  training. 
Trainees  were  required  to  complete  each  module  and  correctly  complete  the  related  scenarios  before  continuing  on 
to  the  subsequent  module.  Directions  at  the  beginning  of  the  CBT  and  the  beginning  of  each  module  explained  that 
clearly.  However,  every  single  trainee  ignored  the  requirements  for  interaction  and  practice,  and  simply  clicked  the 
Next  button  until  they  reached  the  end  of  the  first  module;  they  were  then  dismayed  to  see  instructions  directing 
them  back  to  the  beginning  of  the  module  so  they  could  complete  the  required  interaction  with  the  simulator  that 
provided  the  practice.  To  address  this  astonishing  outcome,  a short  module  was  designed  to  introduce  these  trainees 
to  performance-based  training.  By  requiring  them  to  actually  complete  a brief  simulation  of  the  Log  In  process  (no 
different  than  logging  into  their  network)  prior  to  beginning  the  first  module,  we  began  to  change  their  expectations 
about  interactive,  performance-based  training;  they  all  acquired  bad  habits  from  completing  electronic  page-turners 
(simple  presentations)  mis  -labeled  as  training. 

The  CBT  development  schedule  followed  the  SortSoft  development  schedule  closely,  so  after  additional 
tryouts  at  several  sites  with  members  of  the  target  audience,  and  revisions  required  by  software  changes  or  lessons 
learned  in  tryouts,  the  pilot  was  scheduled  at  the  newest  Fab.  The  entire  population  of  new  hires  training  for  Sort 
Operations  (all  shifts)  reported  to  a small  computer  lab  for  a dedicated  shift,  and  each  completed  the  CBT/Simulator 
intervention.  Trainees  worked  at  their  own  pace.  Trainees  could  log  out  to  take  breaks  as  each  chose;  the  total  time 
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did  not  include  breaks.  Time  stamps  throughout  the  program  tracked  the  completion  time  for  each  module  and  each 
scenario,  and  all  the  performance  data  (answers  to  questions,  selections  made  when  operating  the  simulator,  etc)  was 
also  tracked  electronically.  The  completion  time  varied  from  4 hours  10  minutes  to  7 hours  50  minutes,  with  92%  of 
the  trainees  finishing  in  less  than  6 hours  30  minutes. 

Quantitative  (performance)  Data 

Most  quantitative  assessments  in  training  involve  both  skill  and  knowledge  components;  these  trainees  had 
to  know  some  rules  regarding  safe  and  effective  operation  of  the  tester  itself  in  addition  to  knowledge  about  the 
operations  of  the  software.  The  skills  section  of  the  posttest  utilized  the  SortSoft  simulator,  and  when  the  trainee 
could  demonstrate  the  capability  to  process  three  lots  of  wafers  flawlessly  while  correctly  responding  to  all  alarms, 
the  equipment  owner  engineer  conducted  the  Skills  Certification  test.  The  Cert  test  simply  required  that  the  trainee 
flawlessly  sort  three  lots  of  wafers  while  correctly  responding  to  all  alarms,  although  the  checklist  included  the 
components  of  that  goal  in  more  detail,  which  enabled  engineers  to  determine  areas  of  failure  for  those  who  did  not 
complete  the  task  flawlessly;  those  components  are  also  part  of  the  SortSoft  posttest  and  performance  test. 

The  standards  for  the  CBT/Simu  lator  intervention  were  set  so  as  to  align  to  the  actual  performance  goal  in 
the  factory.  Each  equipment  owner  reported  the  same  results:  for  the  first  time  in  his/her  experience,  all  of  the 
trainees  achieved  certification  on  the  first  attempt,  which  they  described  as  a huge  saving  in  their  time  during  the 
most  critical  time  period  when  a new  wafer  Fab  is  ramping  up  to  begin  production. 

The  Level  2 included  both  the  Posttest  and  subsequent  Fab  Certification  for  each  trainee: 

Posttest  (two  sections): 

Knowledge  - trainee  can: 

Select  SortSoft’ s purpose  / objective,  environment  and  benefits  from  a list 
Select  the  correct  description  and  procedures  for  SortSoft  operations 
Label  SortSoft’ s Main  Screen  and  the  information  displays 

Access  / utilize  commands  in  SoitSoft’s  Menus  necessary  to  perform  basic  SortSoft  tasks 
Recognize  SortSoft’ s alarms  and  state  appropriate  responses  to  each 
Skills- trainee  can.- 

Log  in  to  the  tester  / SortSoft 

Run  a correlation  wafer  and  determine  next  steps 

Set  up  the  tester  for  a specified  wafer  lot  and  / or  probe  card 

Introduce  a wafer  lot 

Modify  a wafer  lot 

Begin,  pause  and  end  testing  (with  and  without  saving  the  data) 

Respond  to  / resolve  typical  operations  alarms  in  order  to  continue  or  restart  testing 
As  noted,  all  trainees  passed  the  posttest,  and  all  passed  the  Certification  Test  on  the  first  attempt.  Initial 
competency  levels,  performance  consistency,  and  response  to  alarms  were  among  the  factors  listed  by  our  clients 
when  we  requested  that  the  equipment  owner  engineers  assess  the  costs  / benefits  of  the  intervention.  Two  to  three 
shifts  (12  hour  shifts)  were  dedicated  per  trainee  for  instructor-led  training  (conducted  one-on-one  by  Technician 
Trainers  or  equipment  owners),  the  CBT/Simulator  freed  up  all  that  time  on  the  part  of  the  Trainer,  and  all  but  five 
or  six  hours  on  the  part  of  the  trainee,  fewer  Cert  Tests  also  freed  up  time  for  everyone.  More  importantly,  that  time 
was  freed  up  during  the  most  intense  times  in  semiconductor  Fab  work,  the  ramping  up  stage.  Equipment  owners 
noted  benefits  of  hands-on  training  without  risking  the  multi-million  dollar  S9K  tester  or  the  $25,000  probe  card 
used  to  conduct  the  sorting  tests.  Down  time,  even  for  training,  is  an  expensive  proposition  because  the  Fab  is  so 
expensive  to  operate  even  aside  from  the  capital  costs  of  the  testers.  Best  of  all,  according  to  the  engineers,  the 
hands-on  training  did  not  subject  wafers  (nearly  complete  in  fabrication  when  sorted,  and  therefore  very  expensive 
in  terms  of  processing  done)  to  damage  and  destruction  in  the  hands  of  novice  trainees. 

Unfortunately,  however,  the  engineers  had  developed  such  a profound  distrust  of  content  and  applications 
mis -labeled  as  training  that  we  were  never  able  to  persuade  them  to  allow  the  simulator-based  posttest  to  qualify  as 
certification  for  technicians  who  passed  flawlessly.  Even  the  successful  re-certifications  of  100%  of  those  trained 
using  CBT/Simulator  over  the  next  year  was  not  sufficient  to  overcome  the  additive  experience  of  so  many  bad 
training  experiences.  Their  responsibility  as  equipment  owners  includes  expensive  equipment  and  supplies  as  well 
as  expensive  product,  and  although  they  enthusiastically  acknowledged  the  success  of  the  training  and  posttest,  they 
wanted  to  see  trainees  perform  at  required  levels  before  they  certified  technicians  to  run  the  equipment  without 
supervision. 
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Qualitative  (opinion)  Data 


The  CBT/Simulator  intervention  was  very  different  from  the  typical  electronic  application  provided  to  the 
trainees  during  New  Hiring  training  (page-turners  organized  by  content  experts),  and  the  novelty  effect  likely 
contributed  to  the  100%  Level  1 response  rate.  Also,  the  Level  1 instrument  was  very  specific  to  the  content  of  the 
CBT  and  to  the  skills  required  for  the  posttest  and  certification,  so  there  was  a clear  link  between  the  responses 
trainees  gave  and  the  training  they  had  just  completed.  Several  trainees  commented  that  they  appreciated  the 
difference  between  the  generic  surveys  used  for  other  training  and  one  designed  to  elicit  meaningful  inputs  for  the 
specific  training  intervention. 

SortSoft  CBT  Level  1 Description 

The  Level  1 instrument  was  specifically  designed  to  measure  perceptions  of  motivation,  continuing 
motivation,  satisfaction,  and  confidence.  The  survey  items  were  specific  enough  to  both  content  and  skill  outcomes 
to  enable  revisions  where  necessary.  Not  all  comments  resulted  in  revisions;  for  example,  trainees  consistently 
complained  of  too  many  practice  items,  that  they  were  required  to  solve  more  problems  than  they  needed  to  in  order 
to  acquire  the  skill.  However,  during  tryouts  trainees  had  consistently  failed  the  posttest  performance  test  on  the 
simulator  until  we  increased  the  number  or  complexity  of  the  practice  problems.  Level  1 data  has  very  little  value 
until  it  is  correlated  with  Level  2 (performance)  data;  the  opinion  of  the  trainee  is  irrelevant  if  there  is  no 
measurable,  observable  improvement  in  job  skill  - after  all,  this  is  on-the-job  training  in  usually  competitive 
businesses,  not  entertainment.  All  of  the  trainees  agreed  or  strongly  agreed  to  the  statements  on  confidence  about 
their  performance  in  the  Fab,  and  that  the  intervention  provided  enough  information,  instruction  and  practice.  All 
but  one  trainee  would  recommend  the  training  to  their  colleagues  and  all  but  one  would  choose  a CBT/Simulator  to 
learn  to  operate  other  Fab  equipment.  The  trainee  who  expressed  a preference  for  instructor-led  training  was  the 
trainee  who  took  the  longest  to  complete  the  intervention;  language  skills  (reading  comprehension,  specifically) 
were  likely  a factor  in  both  the  completion  time  and  the  preference  for  instructor-led  training. 

Trainees  responded  to  a list  of  items  using  a 4-point  Likert -type  scale  attached  to  each:  Strongly  Agree, 
Agree,  Disagree,  and  Strongly  Disagree;  after  this  much  time  expended  on  training  so  critical  to  the  trainee  keeping 
a job,  the  trainee  can  reasonably  be  expected  to  have  an  opinion  about  each  statement  (an  online,  4 -point  Likert -type 
scale  was  implemented  with  course  release). 

The  Level  1 data  was  reported  out  as  counts  (the  number  of  trainees  who  agreed  with  a statement,  the 
number  who  disagreed  strongly  with  the  statement,  etc;  all  four  categories  for  each  statement).  I specifically  coach 
my  clients  to  pay  attention  to  how  practitioners  roll  up  qualitative  data,  as  descriptive  statistics  cannot  be  meaningful 
for  this  data.  The  numbers  assigned  to  the  categories  (typically  1 -4  or  1-5)  are  arbitrary,  or  artificially  determined, 
so  the  categories  remain  Strongly  Agree,  etc.  You  cannot  take  an  average  of  one  Agree  and  one  Disagree,  and  the 
distance  between  Agree  and  Disagree  is  not  numerically  equivalent  to  the  distance  between  Disagree  and  Strongly 
Disagree,  nor  equal  to  twice  the  distance  between  Agree  and  Strongly  Disagree.  Reporting  out  a mean  score  for 
opinion  data  is  just  meaningless,  and  it  destroys  a practitioner’s  credibility  with  anyone  who  has  a basic 
understanding  of  measurement. 

Level  3 (performance  on  the  job)  data  was  captured  subsequently  when  each  S9K  operator  was  successfully 
re-certified  to  current  specifications  at  appropriate  intervals  across  the  next  several  years.  A summative  evaluation 
conducted  after  three  years  yielded  a request  for  one  module  to  be  deleted  due  to  a change  in  procedure  for 
processing  wafers.  Unfortunately,  the  only  other  request  made  at  that  time  was  a request  to  disable  the  data  tracking 
and  writing  functionality  as  no  one  was  using  the  data  even  though  these  trainees  all  require  training  in  numerous 
other  areas  and  the  data  is  valuable  for  learning  about  that  critical  target  audience  who  routinely  handles  and 
processes  wafers  so  late  in  the  expensive  process. 

Conclusion 

Without  using  data  from  the  Level  2 and  Level  3 evaluations  of  the  SortSoft  CBT/Simulator,  we  could 
never  have  achieved  the  level  of  performance  described  above.  In  the  final  analysis,  we  did  a better  job  of  training 
new  technicians  than  the  equipment  owner  engineers;  because  ISD  applied  along  with  expertise  in  learning  and 
information  processing  is  more  powerful  than  subject  matter  expertise  alone  for  training.  If  we  had  released  the 
intervention  in  its  initial  form,  the  performance  levels  of  new  and  retrained  technicians  would  have  been  affected. 
The  company  would  have  paid  a huge  price  when  utilizing  less  efficient  training  that  required  far  more  technician 
training  time  over  the  years,  and  using  less  effective  training  requiring  more  technician  training  time  adds  higher 
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costs  due  to  more  engineer  training  time  and  certification  time.  The  costs  in  damaged  equipment  and  products  would 
be  added  on  also.  Overall,  it  is  always  cheaper  to  make  the  training  effective  and  efficient;  instructional  technologist 
time  is  much  cheaper  than  the  long-term  costs  of  bad  training  for  all  the  trainees  who  are  affected.  There  are  an 
infinite  number  of  excuses,  mostly  mythical  and  all  of  which  result  in  a lack  of  accountability  for  a practitioner.  A 
conglomeration  of  content,  even  if  carefully  formatted,  does  not  constitute  training;  a batch  of  information  handed 
out  (with  or  without  a PowerPoint  presentation)  is  almost  never  training  in  any  meaningful  sense  of  the  term.  In 
fact,  bad  training  is  such  a routine  occurrence  that  we  had  to  add  a special  section  teaching  trainees  to  actually 
complete  the  interactivity  before  attempting  to  continue  with  the  CBT.  Good  training  can  even  help  people  become 
better  learners;  bad  training  damages  profitability,  attitudes,  and  credibility  and  causes  trainees  to  develop  bad 
habits. 

As  professionals,  we  have  a responsibility  to  deliver  what  we  promise  to  deliver  and  what  we  get  paid  for. 
The  current  rates  of  change  in  both  technical  and  non-technical  arenas,  as  well  as  a slowing  economy  and  tightening 
budgets  contribute  to  a reality  where  fewer  and  fewer  people  can  afford  to  throw  away  hours  of  instruction  or 
training  on  non-functional  conglomerations  of  content.  The  consequences  of  failure  are  serious;  whether  the  training 
is  for  baggage  screeners  in  Boston’s  Logan  airport,  or  the  government  employees  who  help  families  fill  out  required 
forms,  or  the  people  who  repair  combat  equipment  for  Special  Forces;  the  commitment  is  to  ensure  learners  achieve 
performance  capability.  Evaluation  is  simply  not  optional  for  Instructional  Technology  professionals. 

One  Thing  To  Take  Away  With  You 

The  most  important  thing  anyone  practicing  in  the  performance  profession  can  learn  is  the  power  of  data  if 
applied  as  part  of  the  continuous  improvement  cycle.  The  following  paragraph  is  used  with  permission  of  the 
authors  (available  at  http://www.whidbev.com/frodo/isd.htm)  {emphasis  added): 

Perhaps  the  greatest  strength  of  the  ISD  process  is  the  evolutionary  nature  of  the  prescriptive,  research- 
based  model  itself  While  the  practice  of  ISD  still  retains  the  strengths  of  the  empirical  evaluation  and  revision 
cycles,  to  the  extent  research  and  experience  permit,  it  is  prescriptive.  That  is,  rather  than  depending  extensively  on 
the  test-revision  cycle  to  generate  effective  instruction  in  an  iterative  manner,  every  attempt  is  made  to  incorporate 
research  findings  and  past  experience  into  the  detailed  procedures  and  supporting  ISD  documentation  to  ensure  that 
the  instruction  developed  comes  as  close  to  the  mark  as  possible  the  first  time.  This  improves  the  validity  of  the 
process  while  also  improving  reliability.  This  has  proven  to  be  2l powerful  tool  in  large  scale  ISD.  In  addition,  as  the 
process  provides  more  data  from  the  constant  evaluation  process,  the  procedures  can  be  continually  improved. 
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